Defining Processes
Hydroplane is responsible for creating and destroying processes. Hydroplane defines a process as anything that runs inside a single container.
Process Specifications
When launching a process, you have to provide a process specification, or process spec for short. A process spec describes the process: it gives the process a name and specifies the location of the process’s container image and how the process’s container should be configured.
Here’s an example of a simple process spec:
{
"process_name": "test",
"container": {
"image_uri": "nginxdemos/hello",
"ports": [
{
"container_port": 80,
"host_port": 9090,
"protocol": "tcp"
}
]
}
}
This process spec describes a process named test that runs the container image nginxdemos/hello, which uses nginx to display a simple webpage.
Most of the process spec is optional, but you have the ability to change a lot of the details of how your process runs by changing its process spec. Here’s a detailed description of the process spec’s format:
- pydantic model hydroplane.models.process_spec.ProcessSpec
Provides all the details of the processs that you’d like to launch.
- field process_name: str [Required]
the name of the process to launch
- field group: Optional[str] = None
the name of the group to which the process belongs, if any
- field container: ContainerSpec [Required]
- field has_public_ip: bool = False
if true, the process has a publicly-exposed IP
- pydantic model hydroplane.models.container_spec.ContainerSpec
- Validators
- field image_uri: str [Required]
the URI of a Docker image that the process will run. Can be a fully-qualified URI, or as a shorthand (e.g. “python” or “nginxdemos/hello” )
- field username: Optional[Union[str, ProcessSecret]] = None
the username used to authenticate with the container registry
- field password: Optional[ProcessSecret] = None
the password used to authenticate with the container registry
- field ports: List[PortMapping] [Optional]
a list of ports that the container should expose
- field env: List[EnvironmentVariable] [Optional]
a list of environment variables that should be given to the container’s runtime
- field command: Optional[List[str]] = None
the command that the container should run
- field resource_request: Optional[ResourceSpec] = None
a request for physical resources that should be allocated to the container
- field resource_limit: Optional[ResourceSpec] = None
a limit on the amount of physical resources that the container is allowed to consume
- Validated by
- field node_selector: Optional[Dict[str, str]] = None
a collection of label name/label value pairs. if this collection is specified and the runtime supports it, this process will only be scheduled onto a node that has these labels
- validator requests_cannot_exceed_limits » resource_limit
If both resource requests and resource limits are specified, ensure that the resource request doesn’t exceed the resource limit.
- pydantic model hydroplane.models.container_spec.EnvironmentVariable
An environment variable that will be provided to the process’s runtime environment.
- field name: str [Required]
the name of the environment variable
- field value: Union[ProcessSecret, str] [Required]
the environment variable’s value; can be a literal, or a reference to a secret
- pydantic model hydroplane.models.container_spec.PortMapping
A port inside the container that will be exposed to the outside world.
- Validators
- field container_port: ConstrainedIntValue [Required]
the port that the container will be listening on
- Constraints
minimum = 1
maximum = 65535
- field host_port: Optional[ConstrainedIntValue] = None
the port that the host will expose. Some runtimes may not honor this setting.
- Constraints
minimum = 1
maximum = 65535
- Validated by
- field protocol: PortProtocol = PortProtocol.TCP
the protocol spoken by the bound port
- field name: Optional[str] = None
the name of the port binding; passed to some runtimes
- class hydroplane.models.container_spec.PortProtocol(value)
An enumeration.
- pydantic model hydroplane.models.container_spec.ResourceSpec
Encapsulates either a request or limit on the physical resources allocated to a process by the runtime.
- field cpu_vcpu: Optional[ConstrainedDecimalValue] = None
the number of abstract vCPUs
- Constraints
minimum = 0.001
- field memory_mib: Optional[ConstrainedIntValue] = None
the number of MiB of memory
- Constraints
minimum = 0
Process Groups
A process can optionally be made part of a process group when it’s created by setting the group field of the process spec. Hydroplane knows how to list the processes that are part of a group, as well as how to destroy a group of processes all at once.
Listing Processes
The Hydroplane server itself is stateless, but it can interrogate its runtime to list running processes. The way that the runtime is interrogated varies, but usually Hydroplane annotates the resources it manages with tags and uses those tags to distinguish processes it manages from other resources in the runtime.
Public and Private Processes
By default, processes are private, meaning that they can communicate with one another (and often other services running in that runtime) but cannot communicate with the outside world.
The has_public_ip field of the process spec controls whether a process is public or private. If the field is set to true, the service will be publicly routable, and its public IP address will be provided instead of its private one(s) when it’s listed.
Process Culling
If you’re running experiments in the cloud, running processes cost money, so you want to make sure you don’t leave processes sitting around accidentally. Hydroplane has a process culler that automatically stops any processes that are older than a certain age (specified in minutes).
The process culler’s settings are described below. Here’s an example configuration file showing process culler configuration:
---
secret_store:
secret_store_type: none
runtime:
runtime_type: docker
process_culling:
# Cull all processes older than two hours
max_age_minutes: 120
# Cull processes every 5 minutes
culling_interval_minutes: 5