s3: tear down the emptied .versions directory on last-version delete; drain existing residue (#10278)

* s3: routed last-version delete removes the emptied .versions directory

The routed versioned delete (routedDeleteSpecificVersion) repoints the
latest pointer and deletes the version file, but unlike the lock-path
fallback (updateLatestVersionAfterDeletion) it never tears down the
.versions/ directory it just emptied. The residue keeps the key's read
path in the self-heal rescan loop: every GET of the deleted key logs
event=surfaced plus a GetObject error until the background
EmptyFolderCleaner gets to the directory — at least two minutes away on
its delay queue, and possibly never (the queue is in-memory, bounded,
and gated on the bucket's allow-empty-folders policy). Veeam's lock
arbitration probes deleted lock keys continuously, so those windows are
always open and the log spam is chronic.

ObjectMutation DELETE gains remove_empty_parent: after the child delete,
the filer best-effort removes the parent directory in the same locked
transaction. Non-recursive on purpose — a concurrent write that lands a
new version fails the removal instead of being lost with it. The routed
last-version delete sets it on the version-file DELETE, matching the
lock-path fallback's contract.

Claude-Session: https://claude.ai/code/session_014mMYAHXZySkCCUfpRFNtSv

* s3: drain empty .versions residue on read heal and in s3.versions.audit

Directories already stranded by pre-teardown deletes (or dropped from
the EmptyFolderCleaner's bounded in-memory queue) previously re-entered
the self-heal rescan on every GET forever: the heal only cleared the
pointer and nothing ever removed the directory, and s3.versions.audit
counted the state as clean.

When the heal rescan finds no remaining version, remove the directory
outright (non-recursive, so orphan children still block and fall back to
the pointer clear) and log event=healed mode=empty_dir_removed; the next
GET takes the clean not-found path. The audit gains an empty category so
the residue is visible, and -heal removes such directories in bulk.

Claude-Session: https://claude.ai/code/session_014mMYAHXZySkCCUfpRFNtSv

weed/s3api/s3api_object_versioning.go

@@ -1957,13 +1957,22 @@ func (s3a *S3ApiServer) healStaleLatestVersionPointer(bucket, normalizedObject s
 	}
 
 	if latestEntry == nil {
-		// Best-effort clear the stale latest-version pointer so subsequent
-		// reads short-circuit to ErrNotFound directly instead of replaying
-		// getLatestObjectVersion's read-retry loop and re-entering self-heal
-		// on every request. Orphan entries (files in .versions/ that lack
-		// the version-id extended attribute) remain in place; from the S3
-		// API perspective the object is correctly absent.
-		s3a.clearStaleLatestVersionPointer(bucket, normalizedObject, bucketDir, versionsObjectPath, versionsEntry, "healStaleLatestVersionPointer")
+		// No version remains: remove the residue directory outright so future
+		// reads take the clean ErrNotFound path instead of re-entering this
+		// self-heal (and its rescan + surfaced warning) on every request. The
+		// non-recursive rm makes this safe against a concurrent PUT — a new
+		// child fails the rm, and a directory removed just before that PUT's
+		// version file lands is recreated by the create's parent handling.
+		if rmErr := s3a.rm(bucketDir, versionsObjectPath, true, false); rmErr == nil {
+			versioningHealInfof("healed", "bucket=%s key=%s mode=empty_dir_removed", bucket, normalizedObject)
+		} else {
+			// Orphan entries (files in .versions/ that lack the version-id
+			// extended attribute) block the teardown and remain in place for an
+			// operator; fall back to clearing the stale pointer so reads at
+			// least short-circuit. From the S3 API perspective the object is
+			// correctly absent either way.
+			s3a.clearStaleLatestVersionPointer(bucket, normalizedObject, bucketDir, versionsObjectPath, versionsEntry, "healStaleLatestVersionPointer")
+		}
 		// Wrap filer_pb.ErrNotFound so callers can distinguish genuine
 		// object-absence (nothing left to promote) from scan failures
 		// (I/O errors during list) via errors.Is.