Quickstart
Introduction
Welcome to the Kurtosis quickstart!
If you arrived here by chance and you're curious as to what Kurtosis is, see here.
If you're ready to get going, this guide will take ~15 minutes and will walk you through building a basic Kurtosis package. The package that you build will start a Postgres server, seed it with data, put an API in front of it, and automate loading data into it.
You need to have Kurtosis installed (or upgraded to latest if you already have it), but you do not need any other knowledge.
If you get stuck at any point during this quickstart, there are many, many options available:
- Every Kurtosis command accepts a
-h
flag to print helptext - The
kurtosis discord
command will open up our Discord kurtosis feedback --github
will take you to opening a Github issuekurtosis feedback --email
will open an email to uskurtosis feedback --calendly
will open a booking link for a personal session with our cofounder Kevin
Don't suffer in silence - we want to hear from you!
Hello, World
First, create and cd
into a directory to hold the project you'll be working on:
mkdir kurtosis-quickstart && cd kurtosis-quickstart
All code blocks in this quickstart can be copied by hovering over the block and clicking the clipboard that appears in the right.
Next, create a Starlark file called main.star
inside your new directory with the following contents (more on Starlark in the "Review" section coming up soon):
def run(plan, args):
plan.print("Hello, world")
If you're using Vim, you can add the following to your .vimrc
to get Starlark syntax highlighting:
" Add syntax highlighting for Starlark files
autocmd FileType *.star setlocal filetype=python
Finally, run the script (we'll explain enclaves in the "Review" section too):
kurtosis run --enclave quickstart main.star
Kurtosis will work for a bit, and then deliver you results:
INFO[2023-03-15T04:27:01-03:00] Creating a new enclave for Starlark to run inside...
INFO[2023-03-15T04:27:05-03:00] Enclave 'quickstart' created successfully
> print msg="Hello, world"
Hello, world
Starlark code successfully run. No output was returned.
INFO[2023-03-15T04:27:05-03:00] ===================================================
INFO[2023-03-15T04:27:05-03:00] || Created enclave: quickstart ||
INFO[2023-03-15T04:27:05-03:00] ===================================================
UUID: a78f2ce1ca68
Enclave Name: quickstart
Enclave Status: RUNNING
Creation Time: Wed, 15 Mar 2023 04:27:01 -03
API Container Status: RUNNING
API Container Host GRPC Port: 127.0.0.1:62828
API Container Host GRPC Proxy Port: 127.0.0.1:62829
========================================== User Services ==========================================
UUID Name Ports Status
Congratulations - you've written your first Kurtosis code!
Review
We'll use these "Review" sections to explain what happened in the section. If you just want the action, feel free to skip them.
In this section, we created a .star
file that prints Hello, world
. The .star
extension corresponds to the Starlark language developed at Google, a dialect of Python for configuring the Bazel build system. Kurtosis uses Starlark for the same purpose of configuring builds, except that we're building a distributed application rather than binaries or JARs.
When you ran the Starlark, you got Created enclave: quickstart
. An enclave is a Kurtosis primitive that can be thought of as an ephemeral house for a distributed application. The distributed applications that you define with Starlark will run inside enclaves.
Enclaves are intended to be easy to create, easy to destroy, cheap to run, and isolated from each other. Use enclaves liberally!
Run Postgres
The heart of any application is the database. To introduce you to Kurtosis, we'll start by launching a Postgres server using Kurtosis.
Replace the contents of your main.star
file with the following:
POSTGRES_PORT_ID = "postgres"
POSTGRES_DB = "app_db"
POSTGRES_USER = "app_user"
POSTGRES_PASSWORD = "password"
def run(plan, args):
# Add a Postgres server
postgres = plan.add_service(
"postgres",
ServiceConfig(
image = "postgres:15.2-alpine",
ports = {
POSTGRES_PORT_ID: PortSpec(5432, application_protocol = "postgresql"),
},
env_vars = {
"POSTGRES_DB": POSTGRES_DB,
"POSTGRES_USER": POSTGRES_USER,
"POSTGRES_PASSWORD": POSTGRES_PASSWORD,
},
),
)
You're almost ready to run, but you still have the quickstart
enclave hanging around from the previous section. Blow it away and rerun:
kurtosis clean -a && kurtosis run --enclave quickstart main.star
This clean-and-run process will be your dev loop for the rest of the quickstart.
You'll see in the result that the quickstart
enclave now contains a Postgres instance:
UUID: a30106a0bb87
Enclave Name: quickstart
Enclave Status: RUNNING
Creation Time: Tue, 14 Mar 2023 20:23:54 -03
API Container Status: RUNNING
API Container Host GRPC Port: 127.0.0.1:59271
API Container Host GRPC Proxy Port: 127.0.0.1:59272
========================================== User Services ==========================================
UUID Name Ports Status
b6fc024deefe postgres postgres: 5432/tcp -> postgresql://127.0.0.1:59299 RUNNING
Review
So what actually happened?
- Interpretation: Kurtosis ran your Starlark to build a plan for what you wanted done (in this case, starting a Postgres instance)
- Validation: Kurtosis ran several validations against your plan, including validating that the Postgres image exists
- Execution: Kurtosis executed the validated plan inside the enclave to start a Postgres container
Note that Kurtosis did not execute anything until after Interpretation and Validation completed. You can think of Interpretation and Validation like Kurtosis' "compilation" for your distributed system: we can catch many errors before any containers run, which shortens the dev loop and reduces the resource burden on your machine.
We call this approach multi-phase runs. While it has powerful benefits, the stumbling point for new Kurtosis users is that you cannot reference Execution values like IP address in Starlark because they simply don't exist at Interpretation time. We'll see how to work around this limitation later.
Add some data
A database without data is a fancy heater, so let's add some.
Our two options for seeding a Postgres database are:
- Making a sequence of PSQL commands via the
psql
binary - Using
pg_restore
to load a package of data
Both are possible in Kurtosis, but for this tutorial we'll do the second one using a seed data TAR of DVD rental information, courtesy of postgresqltutorial.com.
Normally seeding a database would require downloading the seed data to your machine, starting Postgres, and writing a pile of Bash to copy the seed data to the Postgres server and run a pg_restore
. If you forgot to check if the database is available, you may get flakes when you try to use the seeding logic in a test.
You could try Docker Compose to volume-mount the data TAR into the Postgres server, but you'd still need to handle Postgres availability and sequencing the pg_restore
afterwards.
By contrast, Kurtosis Starlark scripts can use data as a first-class primitive and sequence tasks such as pg_restore
into the plan.
Let's see it in action, and we'll explain what's happening afterwards.
Replace your main.star
with the following:
data_package_module = import_module("github.com/kurtosis-tech/awesome-kurtosis/data-package/main.star")
POSTGRES_PORT_ID = "postgres"
POSTGRES_DB = "app_db"
POSTGRES_USER = "app_user"
POSTGRES_PASSWORD = "password"
SEED_DATA_DIRPATH = "/seed-data"
def run(plan, args):
# Make data available for use in Kurtosis
data_package_module_result = data_package_module.run(plan, struct())
# Add a Postgres server
postgres = plan.add_service(
service_name = "postgres",
ServiceConfig(
image = "postgres:15.2-alpine",
ports = {
POSTGRES_PORT_ID: PortSpec(5432, application_protocol = "postgresql"),
},
env_vars = {
"POSTGRES_DB": POSTGRES_DB,
"POSTGRES_USER": POSTGRES_USER,
"POSTGRES_PASSWORD": POSTGRES_PASSWORD,
},
files = {
SEED_DATA_DIRPATH: data_package_module_result.files_artifact,
}
),
)
# Wait for Postgres to become available
postgres_flags = ["-U", POSTGRES_USER,"-d", POSTGRES_DB]
plan.wait(
service_name = "postgres",
recipe = ExecRecipe(command = ["psql"] + postgres_flags + ["-c", "\\l"]),
field = "code",
assertion = "==",
target_value = 0,
timeout = "5s",
)
# Load the data into Postgres
plan.exec(
service_name = "postgres",
recipe = ExecRecipe(command = ["pg_restore"] + postgres_flags + [
"--no-owner",
"--role=" + POSTGRES_USER,
SEED_DATA_DIRPATH + "/" + data_package_module_result.tar_filename,
]),
)
Next to your main.star
, add a file called kurtosis.yml
with the following contents:
name: "github.com/john-snow/kurtosis-quickstart"
Rerun:
kurtosis clean -a && kurtosis run --enclave quickstart .
(Note that the final argument is now .
and not main.star
)
The output should also look more interesting as our plan has grown bigger:
INFO[2023-03-15T04:34:06-03:00] Cleaning enclaves...
INFO[2023-03-15T04:34:06-03:00] Successfully removed the following enclaves:
60601dd9906e40d6af5f16b233a56ae7 quickstart
INFO[2023-03-15T04:34:06-03:00] Successfully cleaned enclaves
INFO[2023-03-15T04:34:06-03:00] Cleaning old Kurtosis engine containers...
INFO[2023-03-15T04:34:06-03:00] Successfully cleaned old Kurtosis engine containers
INFO[2023-03-15T04:34:06-03:00] Creating a new enclave for Starlark to run inside...
INFO[2023-03-15T04:34:10-03:00] Enclave 'quickstart' created successfully
INFO[2023-03-15T04:34:10-03:00] Executing Starlark package at '/tmp/kurtosis-quickstart' as the passed argument '.' looks like a directory
INFO[2023-03-15T04:34:10-03:00] Compressing package 'github.com/YOUR-GITHUB-USERNAME/kurtosis-quickstart' at '.' for upload
INFO[2023-03-15T04:34:10-03:00] Uploading and executing package 'github.com/YOUR-GITHUB-USERNAME/kurtosis-quickstart'
> upload_files src="github.com/kurtosis-tech/awesome-kurtosis/data-package/dvd-rental-data.tar"
Files with artifact name 'howling-thunder' uploaded with artifact UUID '32810fc8c131414882c52b044318b2fd'
> add_service service_name="postgres" config=ServiceConfig(image="postgres:15.2-alpine", ports={"postgres": PortSpec(number=5432, application_protocol="postgresql")}, files={"/seed-data": "howling-thunder"}, env_vars={"POSTGRES_DB": "app_db", "POSTGRES_PASSWORD": "password", "POSTGRES_USER": "app_user"})
Service 'postgres' added with service UUID 'f1d9cab2ca344d1fbb0fc00b2423f45f'
> wait recipe=ExecRecipe(command=["psql", "-U", "app_user", "-d", "app_db", "-c", "\\l"]) field="code" assertion="==" target_value=0 timeout="5s"
Wait took 2 tries (1.135498667s in total). Assertion passed with following:
Command returned with exit code '0' and the following output:
--------------------
List of databases
Name | Owner | Encoding | Collate | Ctype | ICU Locale | Locale Provider | Access privileges
-----------+----------+----------+------------+------------+------------+-----------------+-----------------------
app_db | app_user | UTF8 | en_US.utf8 | en_US.utf8 | | libc |
postgres | app_user | UTF8 | en_US.utf8 | en_US.utf8 | | libc |
template0 | app_user | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/app_user +
| | | | | | | app_user=CTc/app_user
template1 | app_user | UTF8 | en_US.utf8 | en_US.utf8 | | libc | =c/app_user +
| | | | | | | app_user=CTc/app_user
(4 rows)
--------------------
> exec recipe=ExecRecipe(command=["pg_restore", "-U", "app_user", "-d", "app_db", "--no-owner", "--role=app_user", "/seed-data/dvd-rental-data.tar"])
Command returned with exit code '0' with no output
Starlark code successfully run. No output was returned.
INFO[2023-03-15T04:34:21-03:00] ===================================================
INFO[2023-03-15T04:34:21-03:00] || Created enclave: quickstart ||
INFO[2023-03-15T04:34:21-03:00] ===================================================
UUID: 995fe0ca69fe
Enclave Name: quickstart
Enclave Status: RUNNING
Creation Time: Wed, 15 Mar 2023 04:34:06 -03
API Container Status: RUNNING
API Container Host GRPC Port: 127.0.0.1:62893
API Container Host GRPC Proxy Port: 127.0.0.1:62894
========================================== User Services ==========================================
UUID Name Ports Status
f1d9cab2ca34 postgres postgres: 5432/tcp -> postgresql://127.0.0.1:62914 RUNNING
Does our Postgres have data now? Let's find out by logging into the database:
kurtosis service shell quickstart postgres
This will open a shell on the Postgres container. From there, listing the tables in the Postgres...
psql -U app_user -d app_db -c '\dt'
...will reveal that many new tables now exist:
List of relations
Schema | Name | Type | Owner
--------+---------------+-------+----------
public | actor | table | app_user
public | address | table | app_user
public | category | table | app_user
public | city | table | app_user
public | country | table | app_user
public | customer | table | app_user
public | film | table | app_user
public | film_actor | table | app_user
public | film_category | table | app_user
public | inventory | table | app_user
public | language | table | app_user
public | payment | table | app_user
public | rental | table | app_user
public | staff | table | app_user
public | store | table | app_user
(15 rows)
Feel free to explore the Postgres container. When you're done run either exit
or press Ctrl-D.
Review
So what did we just do?
Kurtosis' first-class data primitive is called a files artifact. Each files artifact is a TGZ of arbitrary files, living inside the enclave. So long as a files artifact exists, Kurtosis knows how to mount its contents on a service. We used this feature to mount the seed data into the Postgres instance via the ServiceConfig.files
option:
postgres = plan.add_service(
"postgres",
ServiceConfig(
# ...omitted...
files = {
SEED_DATA_DIRPATH: data_package_module_result.files_artifact,
}
),
)
But where did the data come from?
There are many ways to create files artifacts in an enclave. The simplest is to upload files from your local machine using the kurtosis files upload
command. A more advanced way is to upload files using the upload_files
Starlark instruction on the plan.
But... you never downloaded the seed data on your local machine. In fact, you didn't need you to because we leveraged one of the most powerful features of Kurtosis: composition.
Kurtosis has a built-in packaging/dependency system that allows Starlark code to depend on other Starlark code via Github repositories. When you created the kurtosis.yml
file, you linked your code into the packaging system: you told Kurtosis that your code is a part of a Kurtosis package, which allowed your code to depend on external Starlark code.
This line at the top of your main.star
...
data_package_module = import_module("github.com/kurtosis-tech/awesome-kurtosis/data-package/main.star")
...created a dependency on the external Kurtosis package living here.
Your code then called that dependency here...
data_package_module_result = data_package_module.run(plan, struct())
...which in turn ran the code in the main.star
of that external package. That Kurtosis package happens to contain the seed data, and it uses the upload_data
Starlark instruction on the plan to make the seed data available via a files artifact. From there, all we needed to do was mount it on the postgres
service.
This ability to modularize your distributed application logic using only a Github repo is one of Kurtosis' most loved features. We won't dive into all the usecases now, but the examples here can serve as a good source of inspiration.
Add an API
Databases don't come alone, however. In this section we'll add a PostgREST API in front of the database and see how Kurtosis handles inter-service dependencies.
Replace the contents of your main.star
with this:
data_package_module = import_module("github.com/kurtosis-tech/awesome-kurtosis/data-package/main.star")
POSTGRES_PORT_ID = "postgres"
POSTGRES_DB = "app_db"
POSTGRES_USER = "app_user"
POSTGRES_PASSWORD = "password"
SEED_DATA_DIRPATH = "/seed-data"
POSTGREST_PORT_ID = "http"
def run(plan, args):
# Make data available for use in Kurtosis
data_package_module_result = data_package_module.run(plan, struct())
# Add a Postgres server
postgres = plan.add_service(
service_name = "postgres",
ServiceConfig(
image = "postgres:15.2-alpine",
ports = {
POSTGRES_PORT_ID: PortSpec(5432, application_protocol = "postgresql"),
},
env_vars = {
"POSTGRES_DB": POSTGRES_DB,
"POSTGRES_USER": POSTGRES_USER,
"POSTGRES_PASSWORD": POSTGRES_PASSWORD,
},
files = {
SEED_DATA_DIRPATH: data_package_module_result.files_artifact,
}
),
)
# Wait for Postgres to become available
postgres_flags = ["-U", POSTGRES_USER,"-d", POSTGRES_DB]
plan.wait(
service_name = "postgres",
recipe = ExecRecipe(command = ["psql"] + postgres_flags + ["-c", "\\l"]),
field = "code",
assertion = "==",
target_value = 0,
timeout = "5s",
)
# Load the data into Postgres
plan.exec(
service_name = "postgres",
recipe = ExecRecipe(command = ["pg_restore"] + postgres_flags + [
"--no-owner",
"--role=" + POSTGRES_USER,
SEED_DATA_DIRPATH + "/" + data_package_module_result.tar_filename,
]),
)
# Add PostgREST
postgres_url = "postgresql://{}:{}@{}:{}/{}".format(
"postgres",
POSTGRES_PASSWORD,
postgres.ip_address,
postgres.ports[POSTGRES_PORT_ID].number,
POSTGRES_DB,
)
api = plan.add_service(
service_name = "api",
config = ServiceConfig(
image = "postgrest/postgrest:v10.2.0.20230209",
env_vars = {
"PGRST_DB_URI": postgres_url,
"PGRST_DB_ANON_ROLE": POSTGRES_USER,
},
ports = {POSTGREST_PORT_ID: PortSpec(3000, application_protocol = "http")},
)
)
# Wait for PostgREST to become available
plan.wait(
service_name = "api",
recipe = GetHttpRequestRecipe(
port_id = POSTGREST_PORT_ID,
endpoint = "/actor?limit=5",
),
field = "code",
assertion = "==",
target_value = 200,
timeout = "5s",
)
Now, run the same dev loop command as before (and don't worry about the result!):
kurtosis clean -a && kurtosis run --enclave quickstart .
We just got a failure, just like we might when building a real system!
> wait recipe=GetHttpRequestRecipe(port_id="http", endpoint="/actor", extract="") field="code" assertion="==" target_value=200 timeout="5s"
There was an error executing Starlark code
An error occurred executing instruction (number 6) at github.com/ME/kurtosis-quickstart[77:14]:
wait(recipe=GetHttpRequestRecipe(port_id="http", endpoint="/actor", extract=""), field="code", assertion="==", target_value=200, timeout="5s", service_name="api")
--- at /home/circleci/project/core/server/api_container/server/startosis_engine/startosis_executor.go:62 (StartosisExecutor.Execute.func1) ---
Caused by: Wait timed-out waiting for the assertion to become valid. Waited for '8.183602629s'. Last assertion error was:
<nil>
--- at /home/circleci/project/core/server/api_container/server/startosis_engine/kurtosis_instruction/wait/wait.go:263 (WaitCapabilities.Execute) ---
Error encountered running Starlark code.
Here, Kurtosis is telling us that the wait
instruction on line 77
of our main.star
(the one for ensuring PostgREST is up) is timing out.
The enclave state is usually a good place to start. If we look at the bottom of our output we'll see the following state of the enclave:
UUID: 5b360f940bcc
Enclave Name: quickstart
Enclave Status: RUNNING
Creation Time: Tue, 14 Mar 2023 22:15:19 -03
API Container Status: RUNNING
API Container Host GRPC Port: 127.0.0.1:59814
API Container Host GRPC Proxy Port: 127.0.0.1:59815
========================================== User Services ==========================================
UUID Name Ports Status
45b355fc810b postgres postgres: 5432/tcp -> postgresql://127.0.0.1:59821 RUNNING
80987420176f api http: 3000/tcp STOPPED
The problem is clear now: the api
service status is STOPPED
rather than RUNNING
. When we grab the PostgREST logs...
kurtosis service logs quickstart api
...we can see that the PostgREST is dying:
15/Mar/2023:01:15:30 +0000: Attempting to connect to the database...
15/Mar/2023:01:15:30 +0000: {"code":"PGRST000","details":"FATAL: password authentication failed for user \"postgres\"\n","hint":null,"message":"Database connection error. Retrying the connection."}
15/Mar/2023:01:15:30 +0000: FATAL: password authentication failed for user "postgres"
postgrest: thread killed
Looking back to our Starlark, we can see the problem: we're creating the Postgres database with a user called app_user
, but we're telling PostgREST to try and connect through a user called postgres
:
POSTGRES_USER = "app_user"
# ...
def run(plan, args):
# ...
# Add a Postgres server
postgres = plan.add_service(
service_name = "postgres",
ServiceConfig(
# ...
env_vars = {
# ...
"POSTGRES_USER": POSTGRES_USER,
# ...
},
# ...
),
)
# ...
postgres_url = "postgresql://{}:{}@{}:{}/{}".format(
"postgres", # <---------- THE PROBLEM
POSTGRES_PASSWORD,
postgres.ip_address,
postgres.ports[POSTGRES_PORT_ID].number,
POSTGRES_DB,
)
Replace that "postgres"
with POSTGRES_USER
in your main.star
file to use the correct username, and then rerun your dev loop:
kurtosis clean -a && kurtosis run --enclave quickstart .
Now at the bottom of the output we can see that the PostgREST service is RUNNING
correctly:
UUID: 11c0ac047299
Enclave Name: quickstart
Enclave Status: RUNNING
Creation Time: Tue, 14 Mar 2023 22:30:02 -03
API Container Status: RUNNING
API Container Host GRPC Port: 127.0.0.1:59876
API Container Host GRPC Proxy Port: 127.0.0.1:59877
========================================== User Services ==========================================
UUID Name Ports Status
ce90b471a982 postgres postgres: 5432/tcp -> postgresql://127.0.0.1:59883 RUNNING
98094b33cd9a api http: 3000/tcp -> http://127.0.0.1:59887 RUNNING
Review
In this section, we declared a new PostgREST service with a dependency on the Postgres service.
Yet... PostgREST needs to know the IP address or hostname of the Postgres service, and we said earlier that Starlark (the Interpretation phase) can never know Execution values. How can this be?
Answer: Execution-time values are represented at Interpretation time as future references - special Starlark strings like {{kurtosis:6670e781977d41409f9eb2833977e9df:ip_address.runtime_value}}
that Kurtosis will replace at Execution time with the actual value. In this case, the postgres_url
variable here...
postgres_url = "postgresql://{}:{}@{}:{}/{}".format(
POSTGRES_USER,
POSTGRES_PASSWORD,
postgres.ip_address,
postgres.ports[POSTGRES_PORT_ID].number,
POSTGRES_DB,
)
...used the postgres.ip_address
and postgres.ports[POSTGRES_PORT_ID].number
future references returned by adding the Postgres service, so that when postgres_url
was used as an environment variable during PostgREST startup...
api = plan.add_service(
service_name = "api",
config = ServiceConfig(
# ...
env_vars = {
"PGRST_DB_URI": postgres_url, # <-------- HERE
"PGRST_DB_ANON_ROLE": POSTGRES_USER,
},
# ...
)
)
...Kurtosis simply swapped in the correct Postgres container Execution-time values. While future references take some getting used to, we've found the feedback loop speedup to be very worth it.
Modifying data
Now that we have an API, we should be able to interact with the data.
Inspect your enclave:
kurtosis enclave inspect quickstart
Notice how Kurtosis automatically exposed the PostgREST container's http
port to your machine:
28a923400e50 api http: 3000/tcp -> http://127.0.0.1:59992 RUNNING
In this output the http
port is exposed as URL http://127.0.0.1:59992
, but your port number will be different.
You can paste the URL from your output into your browser (or Cmd+click it in iTerm) to verify that you are indeed talking to the PostgREST inside your quickstart
enclave:
{"swagger":"2.0","info":{"description":"","title":"standard public schema","version":"10.2.0.20230209 (pre-release) (a1e2fe3)"},"host":"0.0.0.0:3000","basePath":"/","schemes":["http"],"consumes":["application/json","application/vnd.pgrst.object+json","text/csv"],"produces":["application/json","application/vnd.pgrst.object+json","text/csv"],"paths":{"/":{"get":{"tags":["Introspection"],"summary":"OpenAPI description (this document)","produces":["application/openapi+json","application/json"],"responses":{"200":{"description":"OK"}}}},"/actor":{"get":{"tags":["actor"],"parameters":[{"$ref":"#/parameters/rowFilter.actor.actor_id"},{"$ref":"#/parameters/rowFilter.actor.first_name"},{"$ref":"#/parameters/rowFilter.actor.last_name"},{"$ref":"#/parameters/rowFilter.actor.last_update"},{"$ref":"#/parameters/select"},{"$ref":"#/parameters/order"},{"$ref":"#/parameters/range"},{"$ref":"#/parameters/rangeUnit"},{"$ref":"#/parameters/offset"},{"$ref":"#/parameters/limit"},{"$ref":"#/parameters/preferCount"}], ...
Now make a request to insert a row into the database (replacing $YOUR_PORT
with the correct PostgREST http
port from your enclave inspect
output)...
curl -XPOST -H "content-type: application/json" http://127.0.0.1:$YOUR_PORT/actor --data '{"first_name": "Kevin", "last_name": "Bacon"}'
...and then query for it (again replacing $YOUR_PORT
with your port)...
curl -XGET "http://127.0.0.1:$YOUR_PORT/actor?first_name=eq.Kevin&last_name=eq.Bacon"
...to get it back:
[{"actor_id":201,"first_name":"Kevin","last_name":"Bacon","last_update":"2023-03-15T02:08:14.315732"}]
Of course, it'd be much nicer to formalize this in Kurtosis. Replace your main.star
with the following:
data_package_module = import_module("github.com/kurtosis-tech/awesome-kurtosis/data-package/main.star")
POSTGRES_PORT_ID = "postgres"
POSTGRES_DB = "app_db"
POSTGRES_USER = "app_user"
POSTGRES_PASSWORD = "password"
SEED_DATA_DIRPATH = "/seed-data"
POSTGREST_PORT_ID = "http"
def run(plan, args):
# Make data available for use in Kurtosis
data_package_module_result = data_package_module.run(plan, struct())
# Add a Postgres server
postgres = plan.add_service(
service_name = "postgres",
ServiceConfig(
image = "postgres:15.2-alpine",
ports = {
POSTGRES_PORT_ID: PortSpec(5432, application_protocol = "postgresql"),
},
env_vars = {
"POSTGRES_DB": POSTGRES_DB,
"POSTGRES_USER": POSTGRES_USER,
"POSTGRES_PASSWORD": POSTGRES_PASSWORD,
},
files = {
SEED_DATA_DIRPATH: data_package_module_result.files_artifact,
}
),
)
# Wait for Postgres to become available
postgres_flags = ["-U", POSTGRES_USER,"-d", POSTGRES_DB]
plan.wait(
service_name = "postgres",
recipe = ExecRecipe(command = ["psql"] + postgres_flags + ["-c", "\\l"]),
field = "code",
assertion = "==",
target_value = 0,
timeout = "5s",
)
# Load the data into Postgres
plan.exec(
service_name = "postgres",
recipe = ExecRecipe(command = ["pg_restore"] + postgres_flags + [
"--no-owner",
"--role=" + POSTGRES_USER,
SEED_DATA_DIRPATH + "/" + data_package_module_result.tar_filename,
]),
)
# Add PostgREST
postgres_url = "postgresql://{}:{}@{}:{}/{}".format(
POSTGRES_USER,
POSTGRES_PASSWORD,
postgres.hostname,
postgres.ports[POSTGRES_PORT_ID].number,
POSTGRES_DB,
)
api = plan.add_service(
service_name = "api",
config = ServiceConfig(
image = "postgrest/postgrest:v10.2.0.20230209",
env_vars = {
"PGRST_DB_URI": postgres_url,
"PGRST_DB_ANON_ROLE": POSTGRES_USER,
},
ports = {POSTGREST_PORT_ID: PortSpec(3000, application_protocol = "http")},
)
)
# Wait for PostgREST to become available
plan.wait(
service_name = "api",
recipe = GetHttpRequestRecipe(
port_id = POSTGREST_PORT_ID,
endpoint = "/actor?limit=5",
),
field = "code",
assertion = "==",
target_value = 200,
timeout = "5s",
)
# Insert data
if args != None:
insert_data(plan, args)
def insert_data(plan, data):
plan.request(
service_name = "api",
recipe = PostHttpRequestRecipe(
port_id = POSTGREST_PORT_ID,
endpoint = "/actor",
content_type = "application/json",
body = json.encode(data),
)
)
Now clean and run, only this time with extra args to kurtosis run
:
kurtosis clean -a && kurtosis run --enclave quickstart . '[{"first_name":"Kevin", "last_name": "Bacon"}, {"first_name":"Steve", "last_name":"Buscemi"}]'
Using the new http
URL on the api
service in the output, query for the rows you just added (replacing $YOUR_PORT
with your correct PostgREST http
port number)...
curl -XGET "http://127.0.0.1:$YOUR_PORT/actor?or=(last_name.eq.Buscemi,last_name.eq.Bacon)"
...to yield:
[{"actor_id":201,"first_name":"Kevin","last_name":"Bacon","last_update":"2023-03-15T02:29:53.454697"},
{"actor_id":202,"first_name":"Steve","last_name":"Buscemi","last_update":"2023-03-15T02:29:53.454697"}]
Review
How did this work?
Mechanically, we first create a JSON string of data using Starlark's json.encode
builtin. Then we use the request
Starlark instruction to shove the string at PostgREST, which writes it to the database:
plan.request(
service_name = "api",
recipe = PostHttpRequestRecipe(
port_id = POSTGREST_PORT_ID,
endpoint = "/actor",
content_type = "application/json",
body = json.encode(data),
)
)
At a higher level, Kurtosis automatically deserialized the [{"first_name":"Kevin", "last_name": "Bacon"}, {"first_name":"Steve", "last_name":"Buscemi"}]
string passed as a parameter to kurtosis run
, and put the deserialized object in the args
parameter to the run
function in main.star
:
def run(plan, args):
Conclusion
And that's it - you've written your very first distributed application in Kurtosis!
Let's review. In this tutorial you have:
- Started a Postgres database
- Seeded it by importing a third-party Starlark package
- Added an API server
- Inserted & queried data via the API
- Parameterized data insertion
Along the way you've learned about several Kurtosis concepts:
But this was still just the intro to Kurtosis. To see examples that you can easily modify to be relevant to you, check out our awesome-kurtosis
repo. To explore real-scale Kurtosis packages delivering value, see the Ethereum package, Waku package, or NEAR package.
And now that you've reached the end, we'd love to hear from you - what went well for you, and what didn't? You can file issues and feature requests on Github...
kurtosis feedback --github
...or you can email us via the CLI...
kurtosis feedback --email
...and you can even schedule a personal session with our cofounder Kevin via:
kurtosis feedback --calendly
We use all feedback to fuel product development, so please don't hesitate to get in touch!
Finally, if liked what you saw and want to engage with us, you can:
- Star us on Github (this helps a lot!)
- Join our Discord (also available with the
kurtosis discord
CLI command) - Reach out to us on Twitter
Or you can simply dive deeper into the docs: