* filer/leveldb_handle: reopen a database whose journal latched a write error
goleveldb keeps the first journal write error and returns it for every
later write until the database is reopened, so one transient ENOSPC or
EIO fails all writes until a restart. The handle offers the leveldb.DB
methods the stores use; a write that fails on an OS error probes the
storage and, once it accepts writes and no call holds the database,
closes and reopens it and retries the write once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer: open the leveldb stores through leveldb_handle
leveldb, leveldb2 and leveldb3 now recover from a transient storage
write error without a restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer: leveldb stores return iterator errors from listings and deletes
A listing on a database that cannot be read reported an empty
directory, and DeleteFolderChildren wrote an empty batch, because the
iterator error was never checked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer/leveldb_handle: test a reopen that fails after the probe passes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer/leveldb_handle: probe storage before draining readers
A probe that stalls in Create, Write or Sync held mu exclusively and
blocked every read on the handle. Reopen attempts are now serialized by
their own mutex, and the probe runs before the drain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer/leveldb_handle: hold new calls back while a reopen drains
TryLock succeeds only at an instant with no reader, so overlapping
listings could keep a reopen from ever draining while each write spun
for the full drain timeout. The drain now holds new calls back for at
most drainTimeout; a call nested inside a running one waits that out
instead of deadlocking. A drain that times out holds off the next
attempt for busyReopenInterval, so new calls are not held back over and
over.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer/leveldb_handle: bound the wait for a running reopen
A write that hit the latched error waited on the reopen mutex for as long
as another caller's storage probe ran, so a stalled probe queued every
writer. Callers now wait at most reopenWait and return their error, and
still retry once a concurrent reopen succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* filer/leveldb_handle: tests wait for the drain and readers explicitly
The nested-read test polled for a drain that could start and end between
polls and then spun forever; it now syncs on the probe, widens the drain
and fails if the put returns first. The overlapping-readers test waits
until every reader holds an iterator before it writes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
