In Part 1 of this series, we set up HashiCorp Vault in an AKS cluster using Terraform, configured ExternalSecrets, and demonstrated how to fetch secrets from Vault's KV engine into Kubernetes.

Now, let's take it a step further. Static credentials are risky—they can be leaked, misused, or forgotten.🤯To mitigate this, Vault provides Dynamic Secrets, allowing credentials to be generated on-demand, time-bound, and auto-revoked after expiration.

✅In this article, we’ll deploy PostgreSQL in our AKS cluster using Helm, explore Vault's database secrets engine to generate short-lived credentials, set externalSecrets and vaultDynamicSecrets to natively sync those credentials in the cluster.

Let's jump right in.!🚀

GIF Credit

👉🏻 1) Setup PostgreSQL using Helm:

We will use the bitnami helm chart, and use the default parameters for the sake of simplicity. Fine-tuning parameters can be added in a separate values.yaml file and applied with the installation command.

helm install my-release oci://registry-1.docker.io/bitnamicharts/postgresql --set hostNetwork=true

This will generate the default Postgres user password and store it as a secret in the cluster. Use the below-mentioned command to fetch the password in decoded format.

kubectl get secret --namespace default my-release-postgresql -o jsonpath="{.data.postgres-password}" | base64 -d

📝Note: The configured password will be ignored on a new installation in case the previous PostgreSQL release was deleted through the helm command. In that case, old PVC will have an old password, and setting it through helm won't take effect. Deleting persistent volumes (PVs) will solve the issue.

Up next, let's create a non-root user in the database which we will be using for all the interactions between Postgres and Vault. Since we are aiming at credential rotations, keeping the root user intact will ensure we never lose access to the database.

# execute interactive bash shell with the database statefulset
kubectl exec -it my-release-postgresql-0 -- /bin/bash

# login to the database 
psql -h 127.0.0.1 -U postgres -p 5432 -W

#Create user with password
CREATE USER vault WITH PASSWORD 'vault123';

# Grant privileges
GRANT ALL PRIVILEGES ON DATABASE postgres TO vault;
ALTER USER vault WITH SUPERUSER;

We can see the vault user is created

👉🏻 2) Create Database Secret Engine from Vault UI:

The Vault database secrets engine dynamically generates database credentials based on configured roles, eliminating the need to hardcode credentials. It supports various databases through plugins and allows for both dynamic and static roles.

Vault's leasing mechanism assigns a Time To Live (TTL) to dynamic secrets and tokens, ensuring they are valid for a specified period. Once the TTL expires, Vault can revoke the secret or token, necessitating periodic lease renewals by clients to maintain access.

Navigate to the Vault UI and create a new secrets engine.

Give an appropriate name and adjust the default lease TTL and Max lease TTL if needed.

Next, select the database plugin as PostgreSQL and pass the connection URL postgresql://{{username}}:{{password}}@localhost:5432/database-name

💡Here, Postgresql is the connection method, username and password of the Vault user created in the previous step, and IP address of the ClusterIP service created while Postgres installation followed by the database as Postgres.

The reason for using private clusterIP is that the database is running the same cluster and can be accessed through the service connected with the statefulset.

To further enhance the security, TLS configuration can also be added in this step.

🤐Let's configure the role for this connection. Dynamic roles generate unique, time-limited database credentials for each service request. In contrast, static roles map Vault roles to existing database usernames, and Vault manages automatic password rotation for these static credentials.

⚠️ Static roles are not recommended for root credentials as rotating them will no longer keep the authentication between Vault and Postgres.

This is the main part. The role is attached to the database connection and it generates a dynamic username and password with 10 minutes of validity (Default TTL) and max TTL of 1 day.

Image Source

We have kept the validity to only 10 minutes to verify the rotation. Based on the sensitivity of the application, the TTL duration can be adjusted.

Here, creation statement and revocation statement consist of SQL queries which would be performed when new credential request is triggered. In our example, it will create a database role with password and expiry and grant select privileges on the given table(s). At the time of expiry, it will revoke all the permissions and drop the user.

As shown in the screenshot below, on clicking "Generate Credentials" from the vault UI, the short-lived credentials are displayed one-time.

Now let's log in with this user and verify if the granted permissions got applied or not. As shown below, it shows temporary users with their expiry dates added as attributes, and any other operation than select would not work.

Okay, so far so good. But how to natively fetch these secrets in the cluster?🤔🤔 There comes the **vaultDynamicSecrets** in the picture.

👉🏻 3) Configure VaultDynamicSecret resource:

Since we already have externalSecrets set up to natively fetch credentials from external secret stores like Vault, integrating these dynamic credentials with the same would make it more flexible to use with other Kubernetes resources through native secrets.

By default, the externalSecrets resource we used in part 1 only supports KV (Key-value) secret engine. Here, we will be using generators which allow to generate values from the given source. Generators can be defined as a custom resource and re-used across different ExternalSecrets.

The VaultDynamicSecret generator specifically integrates with HashiCorp Vault to retrieve dynamic secrets directly from Vault's database secrets engine.

apiVersion: generators.external-secrets.io/v1alpha1
kind: VaultDynamicSecret
metadata:
  name: db-credentials
  namespace: external-secrets
spec:
  path: "/database/creds/dynamicrole"
  method: "GET"
  provider:
    server: "<VAULT SERVER ADDRESS>"
    auth:
        tokenSecretRef:
          name: "vault-token"
          namespace: "external-secrets"
          key: "token"

This resource definition creates a VaultDynamicSecret, which dynamically retrieves database credentials from HashiCorp Vault.

The credentials are fetched from the specified path (/database/creds/dynamicrole) in Vault using the GET method.
The Vault server address and authentication are provided, with the token stored in a Kubernetes secret named vault-token within the external-secrets namespace.

This is one side of the bridge. On the next side, let's create externalSecret resource to connect with vaultDynamicSecrets resource and fetch the short-lived credentials natively.

Here, a native Kubernetes secret named db-credentials will be created keeping the refresh interval to 1 hour and referencing data source from the generator we created in the last step.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: "dynamic-external-secret"
  namespace: external-secrets
spec:
  refreshInterval: "1h"
  target:
    name: db-credentials
  dataFrom:
  - sourceRef:
      generatorRef:
        apiVersion: generators.external-secrets.io/v1alpha1
        kind: VaultDynamicSecret
        name: "db-credentials"

Ready.! Let's deploy the resources :)

Image Credits

Let's verify the changes by describing the secret resource in the namespace external-secrets. Two keys are stored with a username and password in it. This action is supported by the creation statement declared in the role attached to the database secrets engine.

How can we move ahead without verifying the rotation?😮‍💨 Run the below commands to fetch the decoded values of the secrets at intervals of 10 minutes and see the changes in action.

# Fetch username from the secret
kubectl get secret db-credentials -n external-secrets -o jsonpath="{.data.username}" | base64 --decode; echo

# Fetch password from the secret
kubectl get secret db-credentials -n external-secrets -o jsonpath="{.data.password}" | base64 --decode; echo

GIF Credit

And it worked. We got rotated credentials with fine-grained privileges which expire at every TTL duration.

If you are still reading, thanks for making it to the end.🥹 We've taken secrets management to the next level by introducing dynamic database credentials with Vault! Instead of relying on boring static, long-lived passwords, we now have ephemeral credentials that are automatic, time-bound with built-in rotation and seamlessly injected into Kubernetes pods via ExternalSecrets.! Wohoooo 🥳

This means no more hardcoded database passwords, reduced security risks from leaked credentials, and no manual rotation headaches—everything is automated! 🚀

With this, our Kubernetes workloads are now safer, more scalable, and fully automated in handling sensitive data.

💡 Let’s Connect!

Thanks for reading! 🎉 I hope this guide helped you understand dynamic secrets and automated credential rotation in Kubernetes. If you have feedback, suggestions, or want to discuss more, feel free to reach out! 💬

Find me on LinkedIn and check out the full project on GitHub. Let’s build smarter, more secure cloud solutions together! 🚀