Create a modified copy of existing secret using external-secrets operator

1. Technical Notes
2. K8s tips and tricks

Let's say you are setting up an application which also requires PostgreSQL database. Database cluster is create using CloudNativePG operator which automatically creates a secret for the application with connection string:

$ kubectl get secret -oyaml -n partdb partdb-cluster-app | yq .data.uri | base64 -d
postgresql://partdb:...@partdb-cluster-rw.partdb:5432/partdb

And then you discover that application actually expects some extra parameters in the end of connection string. Of course, you could take this secret, create a modified copy of it and manually add it to the cluster, but no.. this is not a gitops way of working. Besides CNPG also supports credential rotation, which might cause fun to debug issues.

Our automated gitops solution relies on using External Secrets operator.

Firstly, lets gather and map our requirements:

1. We would need to create a SecretStore using kubernetes provider targeting single namespace (in our case it's partdb)
2. ...and make sure we have a ServiceAccount with proper RBAC permissions (Role + RoleBinding) to retrieve the existing secret from this namespace.

So lets start by setting up the service account with proper RBAC:

Role with permissions to read Secret and create SelfSubjectRulesReview resources:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: eso-store-role
  namespace: partdb
rules:
- apiGroups: [""]
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - authorization.k8s.io
  resources:
  - selfsubjectrulesreviews
  verbs:
  - create

ServiceAccount:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: eso-user
  namespace: partdb

And RoleBinding assigning Role to ServiceAccount:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: eso-user
  namespace: partdb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: eso-store-role
subjects:
- kind: ServiceAccount
  name: eso-user
  namespace: partdb

At this point we have created enough "plumbing" to roll out SecretStore with kubernetes provider targeting our own cluster:

apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: secrets
  namespace: partdb
spec:
  provider:
    kubernetes:
      auth:
        serviceAccount:
          name: eso-user
      remoteNamespace: partdb
      server:
        caProvider:
          key: ca.crt
          name: kube-root-ca.crt
          type: ConfigMap
        url: kubernetes.default

And finally we can create the ExternalSecret which utilizes template to rewrite the values:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: partdb-fix-secret
  namespace: partdb
spec:
  refreshInterval: 24h
  secretStoreRef:
    kind: SecretStore
    name: secrets
  target:
    name: partdb-cluster-app-fixed
    creationPolicy: Owner
    template:
      engineVersion: v2
      data:
        uri: "{{ .uri }}/?serverVersion=16.4"
  data:
  - secretKey: uri
    remoteRef:
      key: partdb-cluster-app
      property: uri

Let's check it out:

$ kubectl get secret -oyaml -n partdb partdb-cluster-app-fixed | yq .data.uri | base64 -d
postgresql://partdb:...@partdb-cluster-rw.partdb:5432/partdb/?serverVersion=16.4