`eks` - Amazon EKS 

The eks runtime integrates with EKS, AWS’s managed Kubernetes offering.

Contents

eks - Amazon EKS

Settings 

pydantic model hydroplane.runtimes.eks.Settings

field runtime_type: Literal['eks'] = 'eks'

field credentials: AWSCredentials [Required]: the credentials that will be used to authenticate with EKS

field cluster_name: str [Required]: the name of the EKS cluster

field region: str [Required]: the AWS region where the cluster is running (e.g. us-west-2)

field namespace: str = 'default': the Kubernetes namespace that Hydroplane will create pods and services within

Here’s an example of a configuration file that uses the eks runtime and the local secret store. This configuration sets up Hydroplane to talk to an EKS cluster named my-cool-cluster in the us-west-2 AWS region:

---
secret_store:
  secret_store_type: local
  store_location: ~/.hydro_secrets

runtime:
  runtime_type: eks
  cluster_name: my-cool-cluster
  region: us-west-2
  credentials:
    access_key:
      access_key_id:
        secret_name: aws-credentials
        key: AccessKeyId
      secret_access_key:
        secret_name: aws-credentials
        key: SecretAccessKey

Quickstart 

Step 1: Create an EKS cluster and IAM user 

In order to use the eks runtime, you first have to have an EKS cluster. We’ll use hydro-project/eks-setup to do that. Follow the instructions in the “Quickstart” section of that repo’s README and then come back here.

When you’re done with those instructions, you should have two things:

An EKS cluster (we’ll assume that the cluster’s name is hydro-eks and that it was created in us-west-2 in subsequent steps for brevity)
The access key and secret key for a user in the cluster’s cluster-admins group (if you added a user using eks-setup user add, that user is in the cluster-admins group)

Step 2: Configure Hydroplane 

Create a file named eks.yml with the following contents:

---
secret_store:
  secret_store_type: local
  store_location: ~/.hydro_secrets

runtime:
  runtime_type: eks
  cluster_name: hydro-eks
  region: us-west-2
  credentials:
    access_key:
      access_key_id:
        secret_name: aws-credentials
        key: AccessKeyId
      secret_access_key:
        secret_name: aws-credentials
        key: SecretAccessKey

If you haven’t already, initialize your local secret store:

bin/local-secret-store init

Make a note of the password you used when configuring the store; you’ll need it when Hydroplane starts.

Now, create a file called creds.json that looks like this:

{"AccessKeyId":"<put your user's access key ID here>","SecretAccessKey":"<put your user's secret access key here>"}

Add that file to the secret store:

bin/local-secret-store add -f creds.json aws-credentials

and delete the plaintext original afterwards:

rm creds.json

Step 3: Run Hydroplane 

Now you’ve (finally) got an EKS cluster and the ability to authenticate to it! Let’s launch Hydroplane:

bin/hydroplane -c eks.yml

You’ll be prompted for the password to your local secret store. Once you enter it, the server should bind and start accepting requests.

In a separate terminal, from the root of the hydroplane repo, let’s make sure we can list processes:

bin/hpctl list

You should get back an empty list, because we haven’t launched any processes yet.

Let’s try a simple example to make sure we can start something and access it remotely:

bin/hpctl start examples/nginx.json

If you list processes with bin/hpctl list now, you should see something like this:

[
  {
    "process_name": "nginx",
    "group": null,
    "socket_addresses": [
      {
        "host": "34.217.208.197",
        "port": 31491,
        "is_public": true
      }
    ],
    "created": "2022-12-01T22:40:33+00:00",
    "status": "RUNNING"
  }
]

Note that the process is exposing a single public port. In this example, it’s 34.217.208.197:31491, but your host and port will be different.

If you have jq installed, are on macOS, and want to open that server with a one-liner:

open http://$(bin/hpctl list | jq -r "(.[0].socket_addresses[0].host + \":\" + (.[0].socket_addresses[0].port|tostring))")

Otherwise, you can copy-paste the host and port into a browser window. Either way, you should see an nginx “hello world” page.

Now that we’ve verified that everything is working, let’s stop the process:

bin/hpctl stop nginx

Implementation and Configuration Details 

For details on how the eks runtime creates processes, see How Hydroplane Interacts with Kubernetes.

Authenticating with AWS 

In order for Hydroplane to launch processes on an EKS cluster, it must be able to authenticate with that cluster. To authenticate, you’ll need an AWS access key ID and corresponding secret access key. These credentials will need to be stored in Hydroplane’s secret store.

Authenticating with IAM Credentials 

Here’s an example configuration that uses a secret stored in Hydroplane’s secret store to authenticate with EKS. This config uses a secret named aws-creds that’s stored in the secret store as a JSON object like so:

{"AccessKeyId":"XXXXXXXXX","SecretAccessKey":"XXXXXXX"}

These related credentials are referenced individually using the key field.

runtime:
    runtime_type: eks
    cluster_name: my-cluster
    region: us-west-2
    credentials:
      access_key:
        access_key_id:
          secret_name: aws-creds
          key: AccessKeyId
        secret_access_key:
          secret_name: aws-creds
          key: SecretAccessKey

Authenticating with Role Assumption 

While nothing stops you from authenticating directly with an access key and secret, it’s generally considered a best practice to authenticate using temporary IAM credentials. These credentials are associated with a given IAM role, and expire after a certain period of time. Hydroplane will attempt to automatically renew these temporary credentials periodically.

Here’s an example of using the aws-creds secret from above to authenticate and then assume the role EKSAdmin. Most of this configuration will look familiar if you’ve looked at the configuration in the last section, but we’ve added an assume_role block to define the role assumption.

runtime:
    runtime_type: eks
    cluster_name: my-cluster
    region: us-west-2
    credentials:
      access_key:
        access_key_id:
          secret_name: aws-creds
          key: AccessKeyId
        secret_access_key:
          secret_name: aws-creds
          key: SecretAccessKey
       assume_role:
         role_arn: "arn:aws:iam::123456789012:role/EKSAdmin"

A Note on IAM User Permissions 

The authenticating IAM user must have enough permissions on the EKS cluster to create and destroy pods and services. If you’ve set up your cluster with hydro-project/eks-setup, then any user you add using that script will have the appropriate privileges.

AWS Authentication Configuration Reference 

pydantic model hydroplane.models.aws.AWSAccessKey

An AWS access key/secret key pair, used to authenticate with AWS.

field access_key_id: HydroplaneSecret [Required]

field secret_access_key: HydroplaneSecret [Required]

pydantic model hydroplane.models.aws.AWSAssumeRole

Information necessary for an AWS principal to temporarily assume a role (with associated temporary credentials).

field role_arn: str [Required]: the fully-qualified ARN of the role to assume

field external_id: Optional[HydroplaneSecret] = None: if needed, the external ID used to validate access to the role

field session_name: Optional[str] = None: the name of the temporary session created by the role assumption

field session_duration_seconds: Optional[ConstrainedIntValue] = 900

the duration of the temporary session, in seconds

Constraints

minimum = 900
maximum = 43200

pydantic model hydroplane.models.aws.AWSCredentials

Credentials used to authenticate with AWS.

field access_key: AWSAccessKey [Required]

field assume_role: Optional[AWSAssumeRole] = None: the role that Hydroplane will assume when communicating with AWS