s3: keep a file promoted to a directory retrievable as an object (#10070)

* filer: treat a directory carrying object data as an S3 key object

A file promoted to a directory by a child write keeps its chunks, inline
content, or remote-tiered entry. Recognize that as a directory key object,
not only when a Mime is set, so the object still lists, demotes on delete,
and is not reclaimed by cleanup like the object it still is.

* filer: keep the empty-folder cleaner from reclaiming a promoted object

The cleaner skips directory key objects, but its check only looked at the
Mime. Mirror the chunks/content/remote check so a file promoted to a
directory is not deleted once its children are gone.

* s3: serve ranged GET for a directory that carries object data

Reject only zero-size directories so a file promoted to a directory streams
range requests instead of returning 404, while empty directories still 404.

* s3: return HEAD metadata for a directory that carries object data

HEAD now 404s a directory only when it has no data, so a promoted object is
retrievable while empty/implicit directories still fall back to LIST.

weed/s3api/s3api_object_handlers.go

@@ -158,8 +158,8 @@ func (s3a *S3ApiServer) parseAndValidateRange(w http.ResponseWriter, r *http.Req
 		return 0, totalSize, false, nil
 	}
 
-	// S3 semantics: directories (without trailing "/") should return 404
-	if entry.IsDirectory {
+	// Empty directory: 404. A file promoted to a directory keeps its data and stays retrievable.
+	if entry.IsDirectory && totalSize == 0 {
 		s3err.WriteErrorResponse(w, r, s3err.ErrNoSuchKey)
 		return 0, 0, false, newStreamErrorWithResponse(fmt.Errorf("directory object %s/%s cannot be retrieved", bucket, object))
 	}
@@ -2341,7 +2341,8 @@ func (s3a *S3ApiServer) HeadObjectHandler(w http.ResponseWriter, r *http.Request
 		// PyArrow may create 0-byte files when writing datasets, or the filer may have actual directories
 		if objectEntryForSSE.Attributes != nil {
 			isZeroByteFile := objectEntryForSSE.Attributes.FileSize == 0 && !objectEntryForSSE.IsDirectory
-			if objectEntryForSSE.IsDirectory {
+			// A directory with data (a promoted file) is retrievable; empty directories 404 for LIST fallback.
+			if objectEntryForSSE.IsDirectory && filer.FileSize(objectEntryForSSE) == 0 {
 				s3err.WriteErrorResponse(w, r, s3err.ErrNoSuchKey)
 				return
 			}