helm: keep the filer db credentials across uninstall and install (#12138)

* helm: keep the filer db credentials across uninstall and install

The db secret is kept on uninstall (`helm.sh/resource-policy: keep`), but it
is a pre-install hook, and Helm deletes an existing hook resource before
creating it again. The next install therefore wrote a new random password
and reset the user to YourSWUser, so the filer could no longer log in to a
database set up with the old ones, including credentials edited into the
kept Secret.

The template now reads the existing Secret with lookup and reuses its user
and password; a first install still generates them as before. A kind step
in helm_ci.yml installs, edits the Secret, uninstalls, installs again and
checks both values.

* helm: README, the db secret is kept and reused across a reinstall

* helm: make the db secret hook-delete-policy explicit

before-hook-creation is Helm's default for hook resources, but spell it
out so the reuse contract is obvious: lookup reads the kept Secret at
render time, the old copy is deleted before the hook recreates it, and
the stored credentials carry over.

---------

Co-authored-by: Chris Lu <chris.lu@gmail.com>

.github/workflows/helm_ci.yml

@@ -2034,6 +2034,38 @@ jobs:
           kubectl delete namespace "$NS"
           echo "SFTP host key lifecycle tests passed"
 
+      - name: Verify the filer db secret survives uninstall and install
+        run: |
+          set -e
+          CHART_DIR="k8s/charts/seaweedfs"
+          NS="filer-db-secret"
+          SECRET="db-seaweedfs-db-secret"
+          DB_ARGS="--set image.tag=$IMAGE_TAG --set-string filer.extraEnvironmentVars.WEED_MYSQL_ENABLED=true --set master.enabled=false --set volume.enabled=false"
+          kubectl create namespace "$NS"
+
+          echo "=== install generates the db credentials ==="
+          helm install db $CHART_DIR -n "$NS" $DB_ARGS
+          [ "$(kubectl get secret "$SECRET" -n "$NS" -o jsonpath='{.data.user}' | base64 -d)" = YourSWUser ] \
+            || { echo "FAIL: install did not create the default user"; exit 1; }
+          [ "$(kubectl get secret "$SECRET" -n "$NS" -o jsonpath='{.data.password}' | base64 -d | wc -c)" -eq 32 ] \
+            || { echo "FAIL: install did not generate a 32-character password"; exit 1; }
+
+          echo "=== credentials in the kept Secret survive uninstall and install ==="
+          PASSWORD='p"a\ss: #x {{ .Values }}'
+          kubectl patch secret "$SECRET" -n "$NS" --type merge \
+            -p "$(jq -n --arg p "$PASSWORD" '{stringData: {user: "seaweed_app", password: $p}}')"
+          helm uninstall db -n "$NS"
+          kubectl get secret "$SECRET" -n "$NS" >/dev/null || { echo "FAIL: uninstall removed the db secret"; exit 1; }
+          helm install db $CHART_DIR -n "$NS" $DB_ARGS
+          [ "$(kubectl get secret "$SECRET" -n "$NS" -o jsonpath='{.data.user}' | base64 -d)" = seaweed_app ] \
+            || { echo "FAIL: reinstall reset the db user"; exit 1; }
+          [ "$(kubectl get secret "$SECRET" -n "$NS" -o jsonpath='{.data.password}' | base64 -d)" = "$PASSWORD" ] \
+            || { echo "FAIL: reinstall replaced the db password"; exit 1; }
+
+          helm uninstall db -n "$NS"
+          kubectl delete namespace "$NS"
+          echo "Filer db secret reinstall test passed"
+
       - name: Verify JWT signing key persistence across upgrades
         run: |
           # chart-testing puts its pip-less venv first on PATH; use setup-python.

k8s/charts/seaweedfs/README.md

@@ -23,7 +23,7 @@ helm install --values=values.yaml seaweedfs seaweedfs/seaweedfs
 * master/filer/volume are stateful sets with anti-affinity on the hostname,
 so your deployment will be spread/HA.
 * leveldb2 is the default filer backend; a mysql-compatible database (memsql, ...) enables HA (multiple filer instances) and the backup/HA it can provide.
-* with `filer.extraEnvironmentVars.WEED_MYSQL_ENABLED` set to `"true"`, mysql user/password are created in a k8s secret (default: `<release>-seaweedfs-db-secret`) and injected to the filer with ENV. On any other store neither the secret nor the `WEED_MYSQL_*` env, plain or secret-backed, is rendered.
+* with `filer.extraEnvironmentVars.WEED_MYSQL_ENABLED` set to `"true"`, mysql user/password are created in a k8s secret (default: `<release>-seaweedfs-db-secret`) and injected to the filer with ENV. The secret is kept on `helm uninstall`, and a later install reuses the user and password in it, including ones you edited in. On any other store neither the secret nor the `WEED_MYSQL_*` env, plain or secret-backed, is rendered.
 * cert config exists and can be enabled, but not been tested, requires cert-manager to be installed.
 
 ## Prerequisites

k8s/charts/seaweedfs/templates/shared/secret-seaweedfs-db.yaml

@@ -1,4 +1,8 @@
 {{- if and .Values.filer.enabled (include "seaweedfs.filer.mysqlEnabled" .) }}
+{{- /* Kept on uninstall, but the pre-install hook recreates it on the next install:
+       reuse the stored user and password so the filer can still log in. */}}
+{{- $existing := lookup "v1" "Secret" .Release.Namespace (printf "%s-db-secret" (include "seaweedfs.fullname" .)) }}
+{{- $data := ($existing | default dict).data | default dict }}
 apiVersion: v1
 kind: Secret
 type: Opaque
@@ -8,13 +12,14 @@ metadata:
   annotations:
     "helm.sh/resource-policy": keep
     "helm.sh/hook": "pre-install"
+    "helm.sh/hook-delete-policy": "before-hook-creation"
   labels:
     app.kubernetes.io/name: {{ template "seaweedfs.name" . }}
     helm.sh/chart: {{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}
     app.kubernetes.io/managed-by: {{ .Release.Service }}
     app.kubernetes.io/instance: {{ .Release.Name }}
 stringData:
-  user: "YourSWUser"
-  # auto-generated password
-  password: {{ randAlphaNum 10 | sha256sum | b64enc | trunc 32 }}
+  user: {{ if $data.user }}{{ b64dec $data.user | quote }}{{ else }}"YourSWUser"{{ end }}
+  # generated on first install, reused afterwards
+  password: {{ if $data.password }}{{ b64dec $data.password | quote }}{{ else }}{{ randAlphaNum 10 | sha256sum | b64enc | trunc 32 }}{{ end }}
 {{- end }}