--- title: ServiceConfig sidebar_label: ServiceConfig --- The `ServiceConfig` is used to configure a service when it is added to an enclave (see [add_service][add-service-reference]). ```python config = ServiceConfig( # The name of the container image that Kurtosis will use to create and start the service's container. # This image can be pulled from a remote container registry, picked up from the local cache, or built by Kurtosis using # the underlying container engine. # If a string is provided, Kurtosis will by default detect if images exists locally, or pull from container registry if not. # The string form is syntactic sugar for ImageSpec with only image set # ImageSpec referring to private registries limited to Docker # Reach out to the team if you want to run Kurtosis with private images on Kubernetes # If an ImageBuildSpec is provided, Kurtosis will build the image. # MANDATORY image = "kurtosistech/example-datastore-server", OR image = ImageSpec( # The name of the image that needs to be pulled qualified with the registry # MANDATORY image = "my.registry.io/my-user/my-image", # The username that will be used to pull the image from the given registry # OPTIONAL username = "my-user", # The password that will be used to pull the image from the given registry # OPTIONAL password = "password", # The URL of the registry # OPTIONAL registry = "http://my.registry.io/" ) OR image = ImageBuildSpec( # Name to give built image # MANDATORY image_name="kurtosistech/example-datastore-server" # Locator to build context within the Kurtosis package # As of now, Kurtosis expects a Dockerfile at the root of the build context # MANDATORY build_context_dir="./server" # Name of the build file # The default value is "Dockerfile" # OPTIONAL build_file="" # Stage of image build to target for multi-stage container image # OPTIONAL target_stage="" # Arguments passed at build-time # OPTIONAL (Default: {}) build_args={ "BUILD_ARG_1": "VALUE_1", "BUILD_ARG_2": "VALUE_2", } ) OR image = NixBuildSpec( # The name of the container image that should be used # MANDATORY image_name = "hello-world-server", # Locator to build context within the Kurtosis package # This allows to select a sub-package where the context is going be used to build the image # MANDATORY build_context_dir = "./" # The relative path (from the `build_context_dir`) to the folder containing the flake.nix file # MANDATORY flake_location_dir = "./hello-go", # The selector for the Flake output with the image derivation. Fallbacks to the default package. flake_output = "containerImage", ) # The ports that the container should listen on, identified by a user-friendly ID that can be used to select the port again in the future. # Kurtosis will automatically perform a check to ensure all declared UDP and TCP ports are open and ready for traffic and connections upon startup. # You may specify a custom wait timeout duration or disable the feature entirely, learn more via PortSpec docs # If no ports are provided, no ports will be exposed on the host machine, unless there is an EXPOSE in the Dockerfile. # OPTIONAL (Default: {}) ports = { "grpc": PortSpec( # The port number which we want to expose # MANDATORY number = 3000, # Transport protocol for the port (can be either "TCP" or "UDP") # Optional (DEFAULT:"TCP") transport_protocol = "TCP", # Application protocol for the port # Optional application_protocol = "http", # Kurtosis will automatically perform a check to ensure all declared UDP and TCP ports are open and ready for traffic and connections upon startup. # You may specify a custom wait timeout duration or disable the feature entirely. # You may specify a custom wait timeout duration with a string: # wait = "2m" # Or, you can disable this feature by setting the value to None: # wait = None # The feature is enabled by default with a default timeout of 15s # OPTIONAL (DEFAULT:"15s") wait = "4s" ), }, # The deterministic public ports that Kurtosis will expose from the container to the machine # This only applies to Docker; the port ids here must be a subset of `ports` # If no ports are provided; normal ports behavior will happen and the exposed ports will be randomly allocated # This doesn't work on Kubernetes!!! # OPTIONAL (Default: {}) public_ports = { "grpc": PortSpec( # The port number which we want to expose # MANDATORY number = 3000, # Transport protocol for the port (can be either "TCP" or "UDP") # Optional (DEFAULT:"TCP") transport_protocol = "TCP", # Application protocol for the port # Optional application_protocol = "http" ) }, # Whether to publish UDP ports to the host machine. # By default, UDP ports are not published to the host to avoid port exhaustion issues, # as each published UDP port consumes a host port. # Set to True if you need to access UDP ports from outside the enclave. # This only applies to Docker. # OPTIONAL (Default: False) publish_udp = False, # A mapping of path_on_container_where_contents_will_be_mounted -> Directory object or file artifact name # For more info on what a Directory object is, see below # # OPTIONAL (Default: {}) files = { "path/to/files/artifact_1/": files_artifact_1, "path/to/files/artifact_2/": Directory( artifact_names=[files_artifact_2], ), "path/to/persistent/directory/": Directory( persistent_key="data-directory", ), }, # The ENTRYPOINT statement hardcoded in a container image's Dockerfile might not be suitable for your needs. # This field allows you to override the ENTRYPOINT when the container starts. # OPTIONAL (Default: []) entrypoint = [ "bash", ], # The CMD statement hardcoded in a container image's Dockerfile might not be suitable for your needs. # This field allows you to override the CMD when the container starts. # OPTIONAL (Default: []) cmd = [ "-c", "sleep 99", ], # Defines environment variables that should be set inside the Docker container running the service. # This can be necessary for starting containers from Docker images you don’t control, as they’ll often be parameterized with environment variables. # OPTIONAL (Default: {}) env_vars = { "VAR_1": "VALUE_1", "VAR_2": "VALUE_2", }, # ENTRYPOINT, CMD, and ENV variables sometimes need to refer to the container's own IP address. # If this placeholder string is referenced inside the 'entrypoint', 'cmd', or 'env_vars' properties, the Kurtosis engine will replace it at launch time # with the container's actual IP address. # OPTIONAL (Default: "KURTOSIS_IP_ADDR_PLACEHOLDER") private_ip_address_placeholder = "KURTOSIS_IP_ADDRESS_PLACEHOLDER", # The maximum amount of CPUs the service can use, in millicpu/millicore. # OPTIONAL (Default: no limit) max_cpu = 1000, # The minimum amout of CPUs the service must have, in millicpu/millicore. # CAUTION: This is only available for Kubernetes, and will be ignored for Docker. # OPTIONAL (Default: no limit) min_cpu = 500, # The maximum amount of memory, in megabytes, the service can use. # OPTIONAL (Default: no limit) max_memory = 1024, # The minimum amount of memory, in megabytes, the service must have. # CAUTION: This is only available for Kubernetes, and will be ignored for Docker. # OPTIONAL (Default: no limit) min_memory = 512, # This field can be used to check the service's readiness after the service has started, # to confirm that it is ready to receive connections and traffic # OPTIONAL (Default: no ready conditions) ready_conditions = ReadyCondition(...), # This field is used to specify custom labels at the container level in Docker and Pod level in Kubernetes. # For Docker, the label syntax and format will follow: "com.kurtosistech.custom.key": "value" # For Kubernetes, the label syntax & format will follow: kurtosistech.com.custom/key=value # Labels must follow the label standards outlined in [RFC-1035](https://datatracker.ietf.org/doc/html/rfc1035), # meaning that both the label key and label value must contain at most 63 characters, contain only lowercase # alphanumeric characters, dashes (-), underscores (_) or dots (.), start with an alphabetic character, and end with an alphanumeric character. # Empty value and capital letters are valid on label values. # OPTIONAL labels = { "key": "value", # Examples "name": "alice", } # The user id and group id to start the container with # Some containers are configured to start with users other than root by default; this allows you to override to root or other # users if you choose to. # Note that the `gid` field is optional # OPTIONAL user = User(uid=0, gid=0), # An array of Toleration # This refers to Kubernetes Tolerations https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ # This has no effect on Docker # As of 2024-01-24 Taints and Tolerations to work with Kubernetes you need at least one untainted node # Refer to the Toleration docs linked near the end of the page to learn more # OPTIONAL tolerations = [ Toleration( key = "test-key", value = "test-value", operator = "Equal", effect = "NoSchedule", toleration_seconds = 64, ) ], # A map of node selectors # This refers to Node Selectors in Kubernetes https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector # This has no effect on Docker # This is an experimental feature that might get replaced with a better abstraction suited for Kurtosis # OPTIONAL node_selectors = { "disktype": "ssd", }, # The tini_enabled field allows you to set the `--init` options when a container is started in Docker. # OPTIONAL # Default (true) tini_enabled = True, # The tty_enabled field enables TTY mode for the container, allowing interactive terminal sessions. # When set to True, the container will be created with TTY support enabled for interactive use. # OPTIONAL # Default (false) tty_enabled = False, # The devices field allows you to mount host devices into the container. # Each device path (e.g., "/dev/tpm0") will be mounted at the same path inside the container. # Devices are mounted with read-write-mknod permissions. # This is useful for accessing hardware devices like TPM modules, GPUs, or other specialized hardware. # OPTIONAL (Default: []) devices = [ "/dev/tpm0", "/dev/tpmrm0", ], # Linux capabilities to add to the container. # On Docker, these map to --cap-add flags. On Kubernetes, they are set via SecurityContext.Capabilities.Add. # This is useful for advanced networking (packet capture, traffic shaping), debugging, or accessing # privileged kernel features without running the container as fully privileged. # See https://man7.org/linux/man-pages/man7/capabilities.7.html for the full list of capabilities. # OPTIONAL (Default: []) capabilities = [ "NET_RAW", "NET_ADMIN", "SYS_PTRACE", "SYS_ADMIN", ], # Start the container with the Docker --privileged flag. # DANGEROUS: a privileged container has near-total access to the host kernel and can trivially # escape the container boundary. Only enable this for trusted images that genuinely require it # (for example, ethpandaops/disruptoor, which manipulates host networking). # Docker only — using this on the Kubernetes backend fails at interpretation time. # This field also requires an explicit run opt-in: pass --privileged on the CLI, # set allow-privileged-mode: true for the current cluster in kurtosis-config.yml, # or set allow_privileged_mode in the API request. The CLI flag/config/API field only # permits this ServiceConfig field; it does not make a service privileged by itself. # OPTIONAL (Default: False) privileged = False, # Bind-mount specific host paths into the container, mapping host_path -> container_path. # Today the only allowlisted host path is /var/run/docker.sock — any other host path is # rejected at interpretation time. This is intentionally narrow; the use case it exists for # is letting a container talk to the host's Docker daemon (for example, ethpandaops/disruptoor). # Docker only — using this on the Kubernetes backend fails at interpretation time. # This field also requires the same explicit run opt-in as privileged=True. # OPTIONAL (Default: {}) bind_mounts = { "/var/run/docker.sock": "/var/run/docker.sock", }, # Start the container in Docker's host PID namespace, equivalent to docker run --pid=host. # This lets the container see host/container PIDs under /proc. It is needed by tools that use # nsenter against sibling containers, such as ethpandaops/disruptoor. # Docker only — using this on the Kubernetes backend fails at interpretation time. # This field also requires the same explicit run opt-in as privileged=True. # OPTIONAL (Default: False) host_pid_namespace = False, # GPU configuration: device selection, shared-memory size, ulimits, and driver. # All GPU-related settings are bundled in a GpuConfig object since they only apply to GPU workloads. # OPTIONAL (Default: None — no GPU access) gpu = GpuConfig( # The number of GPUs to expose to the container. # Use -1 to expose all available GPUs, 0 for none, or a positive integer for a specific count. # On Docker, sets the DeviceRequests count for the configured driver. # On Kubernetes, sets a resource limit/request using the configured k8s resource name # (e.g. "nvidia.com/gpu"); only positive counts are supported on Kubernetes. # Cannot be used together with device_ids. # OPTIONAL (Default: 0) count = 2, # A list of specific GPU device IDs to pin to the container. # Use this when you need to assign particular GPUs by their device index or UUID # (e.g. ["0", "1"] or ["GPU-abc123"]). # On Docker, sets the DeviceRequests device IDs for the configured driver. # NOTE: device_ids is not supported on Kubernetes — use count instead. # Cannot be used together with count. # OPTIONAL (Default: []) device_ids = ["0", "1"], # The size of /dev/shm in megabytes. # On Docker, sets HostConfig.ShmSize (converted to bytes). # On Kubernetes, mounts a memory-backed emptyDir volume at /dev/shm of the given size. # OPTIONAL (Default: 0, which uses the runtime default — typically 64 MB) shm_size = 128, # Resource limits (ulimits) to apply to the container. # Keys are ulimit names (e.g. "memlock", "nofile") and values are the limit (soft=hard). # Use -1 for unlimited. Common: memlock=-1 for CUDA unified memory. # On Docker, maps to HostConfig.Ulimits. # NOTE: ulimits are not supported on Kubernetes. # OPTIONAL (Default: {}) ulimits = { "memlock": -1, "nofile": 65536, }, # The GPU driver to use. Accepts either: # - A string shorthand, e.g. "nvidia" or "amd". The Kubernetes resource name is # derived automatically as ".com/gpu" (e.g. "nvidia.com/gpu"). # - A dict with "docker" and/or "kubernetes" keys for explicit per-backend control, # e.g. {"docker": "amd", "kubernetes": "amd.com/gpu"} or # {"kubernetes": "gpu.intel.com/i915"} (unset keys use their defaults). # OPTIONAL (Default: "nvidia") driver = "nvidia", ), ) ``` ### Opt-in and update limitations `privileged`, `bind_mounts`, and `host_pid_namespace` give the resulting container elevated host access. They are denied by default and require an explicit run opt-in through `kurtosis run --privileged`, `kurtosis service add --privileged`, `kurtosis service update --privileged`, the per-cluster `allow-privileged-mode: true` config value, or the `allow_privileged_mode` API field. `--privileged` is an allow flag. It permits packages or JSON service configs that explicitly set `ServiceConfig(privileged=True)`, `bind_mounts={...}`, or `host_pid_namespace=True`; it does not set those fields automatically. `plan.set_service` and `kurtosis service update` currently preserve and can enable these fields, but cannot clear an existing `privileged=True`, remove existing bind mounts, or unset `host_pid_namespace=True`. To remove privileged access today, recreate the service without those fields. For a usage walkthrough and allowlist details, see the [Privileged containers & Docker socket guide](../../guides/running-privileged-containers.md). Note that `ImageBuildSpec` can only be used in packages and not standalone scripts as it relies on build context in package. More info on [`ImageBuildSpec`](./image-build-spec.md) here. More info can be found on [locators referring to local resources here][locators] and how to turn your script into a Kurtosis [package][package] here. The `ports` dictionary argument accepts a key value pair, where `key` is a user defined unique port identifier and `value` is a [PortSpec][port-spec] object. The `files` dictionary argument accepts a key value pair, where `key` is the path where the contents of the artifact will be mounted to and `value` is a [Directory][directory] object or files artifact name. Using a `Directory` object with `artifact_name` is strictly equivalent to directly using the files artifact name as the value of the dictionary. This is just to simplify usage. See [`NixBuildSpec`][nix-build-spec] for more information on how to use the Nix and Kurtosis together. You can view more information on [configuring the `ReadyCondition` type here][ready-condition]. :::note If you are experiencing issues with unsuccessful port check, try exposing the port on all network interfaces via `0.0.0.0` eg `--rpc.laddr tcp://0.0.0.0:36657`). See [here][port-ip-doc] for an in depth explanation. ```bash == FINISHED SERVICE 'service-a' LOGS =================================== Caused by: An error occurred while waiting for all TCP and UDP ports to be open Caused by: Unsuccessful ports check for IP '172.16.0.10' and port spec '{privatePortSpec:0x400071d0b0}', even after '240' retries with '500' milliseconds in between retries. Timeout '2m0s' has been reached ``` ::: :::tip If you are trying to use a more complex versions of `cmd` and are running into issues, we recommend using `cmd` in combination with `entrypoint`. You can set the `entrypoint` to `["/bin/sh", "-c"]` and then set the `cmd` to the command as you would type it in your shell. For example, `cmd = ["echo foo | grep foo"]` ::: :::tip Example of a rendered label If you have defined the pod label key:value pair in `ServiceConfig` to be: ```py config = ServiceConfig( ... labels = { "name": "alice", "age": "20", "height": "175" } ) ``` then the labels for the pods on Kubernetes will look like: ``` labels: kurtosistech.com.custom/name=alice kurtosistech.com.custom/age=20 kurotsistech.com.custom/height=175 ``` while on Docker, the container labels will look like: ``` labels: "com.kurtosistech.custom.name": "alice" "com.kurtosistech.custom.age": "20" "com.kurtosistech.custom.height": "175" ``` ::: The `user` field expects a [`User`][user] object being passed. The `tolerations` field expects a list of [`Toleration`][toleration] objects being passed. The `gpu` field expects a [`GpuConfig`][gpu-config] object being passed. See the [GpuConfig reference][gpu-config] for full details on GPU device selection, shared memory, ulimits, driver configuration, and Kubernetes limitations. [add-service-reference]: ./plan.md#add_service [directory]: ./directory.md [port-spec]: ./port-spec.md [ready-condition]: ./ready-condition.md [locators]: ../../advanced-concepts/locators.md [package]: ../../advanced-concepts/packages.md [user]: ./user.md [toleration]: ./toleration.md [nix-build-spec]: ./nix-build-spec.md [gpu-config]: ./gpu-config.md [port-ip-doc]: ../../advanced-concepts/public-and-private-ips-and-ports.md#gotchas