Leverage AWS Secrets Manager in EKS Fargate with External Secrets Operator

---

1. Overview

2. Steps to Setup Fargate Profile using eksctl

3. Installing the External Secrets Operator

4. Consuming Secrets in Kubernetes Pods

5. Conclusion

6. References

---

Overview

When deploying applications on Amazon EKS (Elastic Kubernetes Service) using Fargate, managing sensitive data like API keys, database credentials, and other secrets securely is a critical consideration. AWS Secrets Manager or AWS Systems Manager (SSM) Parameter Store are ideal tools for storing these secrets, but how do you seamlessly integrate them with your Fargate workloads? In this blog post, we will explore how to leverage AWS Secrets Store in EKS Fargate by using the External Secrets Operator. We’ll walk through setting up the Fargate profile, installing the External Secrets Operator, and consuming secrets in your Kubernetes Pods.

The diagram below illustrates the sync process.

1. External Secrets will periodically call the AWS Secrets Manager API to retrieve, and copy specified secret values.

2. These copied values will then be used by External Secrets to create native Kubernetes Secrets.

3. The Kubernetes Secrets will be accessible to the designated applications running on your EKS cluster, with Kubernetes RBAC determining which applications can access the secrets.

4. Pods will access these secrets either as volume mounts or environment variables, as specified in their pod configurations.

   

   
   

Steps to Setup Fargate Profile using eksctl

Install eksctl

Before creating an EKS cluster, ensure you have the eksctl command-line tool installed. You can install eksctl by following the official instructions for your operating system. Here is a quick guide:

For macOS (using Homebrew):

brew tap weaveworks/tap brew install weaveworks/tap/eksctl

For Linux:

curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/0.144.0/eksctl_Linux_amd64.tar.gz" | tar xz -C /tmp sudo mv /tmp/eksctl /usr/local/bin

Create an EKS Cluster with Fargate

Once eksctl is installed, you can create an EKS cluster with Fargate support using the following command:

eksctl create cluster --name eks-fargate-cluster --region us-west-2 --fargate

This command creates a fully managed EKS cluster and automatically provisions a Fargate profile that runs your workloads on AWS Fargate.

Verify the Cluster and Fargate Profile

You can verify the creation of your cluster and the Fargate profile by using the following commands:

eksctl get cluster --name eks-fargate-cluster kubectl get fargateprofile --namespace kube-system

Once this is done, your EKS Fargate cluster is ready to be used.

Installing the External Secrets Operator

Add the Helm Chart Repository

To install the External Secrets Operator, first, ensure you have Helm installed. You can install Helm using the following commands:

For macOS (using Homebrew):

brew install helm

For Linux:

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

Once Helm is installed, add the External Secrets Operator Helm chart repository:

helm repo add external-secrets https://charts.external-secrets.io helm repo update

Install the External Secrets Operator

Now that the repository is added, you can install the External Secrets Operator using the following command:

helm install external-secrets external-secrets/external-secrets --namespace kube-system

This command installs the External Secrets Operator in the kube-system namespace, which is responsible for syncing secrets between external secret stores (like AWS Secrets Manager or SSM Parameter Store) and Kubernetes.

Verify the Installation

You can verify that the External Secrets Operator is running by checking the pods:

kubectl get pods -n kube-system | grep external-secrets

You should see the operator pod running, confirming the installation was successful.

Consuming Secrets in Kubernetes Pods

Now that we’ve set up the External Secrets Operator, it’s time to consume secrets stored in AWS Secrets Manager.

Create a Secret in AWS Secrets Manager

First, create a secret in AWS Secrets Manager that you want to reference in your Kubernetes application.

For example, you might store a database password:

aws secretsmanager create-secret --name my-database-secret --secret-string '{"username":"dbadmin","password":"supersecretpassword"}'

Define an ExternalSecret in Kubernetes

Next, create a manifest file that references the AWS Secrets Manager secret. This manifest will define how the secret is pulled into your Kubernetes pods.

Here’s an example of the manifest (my-database-secret.yaml):

apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: my-database-secret
spec:
  secretStoreRef:
    name: aws-secrets-manager
  target:
    name: my-database-secret
  data:
    - secretKey: db-username
      remoteRef:
        key: my-database-secret
        property: username
    - secretKey: db-password
      remoteRef:
        key: my-database-secret
        property: password

Apply the ExternalSecret Resource

Apply the manifest file to your Kubernetes cluster:

kubectl apply -f my-database-secret.yaml

This action tells the External Secrets Operator to fetch the my-database-secret from AWS Secrets Manager and inject it into a Kubernetes secret named my-database-secret.

Reference the Secret in Your Pod

Finally, reference the secret in your pod specification. Here’s an example of how to reference the secret in a Kubernetes deployment (my-app-deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: my-app-image:latest
        env:
        - name: DB_USERNAME
          valueFrom:
            secretKeyRef:
              name: my-database-secret
              key: db-username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: my-database-secret
              key: db-password

Conclusion

By leveraging AWS Secrets Manager with the External Secrets Operator on EKS Fargate, you can securely manage and inject secrets into your Kubernetes applications with minimal effort. This setup not only simplifies secret management but also ensures that sensitive data is never exposed in your Kubernetes manifests or hardcoded in your applications. With a few simple steps—setting up a Fargate profile, installing the External Secrets Operator, and configuring secrets consumption—you can significantly enhance the security posture of your EKS workloads.

References

• AWS Blog: Leverage AWS Secrets Stores from EKS Fargate with External Secrets Operator

• External Secrets Operator GitHub Repository

• AWS Secrets Manager Documentation