"no recoverable versions" on startup

[exarkun]

Seemingly every time a ZKAPAuthorizer-enabled Tahoe-LAFS client node starts this traceback is logged:

2022-02-01T09:43:14-0500 [-] Fuzzy timer service (lease maintenance service)
        Traceback (most recent call last):
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 501, in errback
            self._startRunCallbacks(fail)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
            self._runCallbacks()
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 1475, in gotResult
            _inlineCallbacks(r, g, status)
        --- <exception caught here> ---
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/_zkapauthorizer/controller.py", line 1030, in bracket
            result = yield between()
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/_zkapauthorizer/lease_maintenance.py", line 144, in renew_leases
            storage_indexes = yield iter_storage_indexes(visit_assets)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/python/failure.py", line 512, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/_zkapauthorizer/lease_maintenance.py", line 79, in visit_storage_indexes
            children = yield elem.list()
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/allmydata/mutable/filenode.py", line 446, in _maybe_retry
            failure.trap(NotEnoughSharesError)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/python/failure.py", line 460, in trap
            self.raiseException()
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/python/failure.py", line 488, in raiseException
            raise self.value.with_traceback(self.tb)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/nix/store/agy6yqqybdwh6scvmavkrsr1kx4dfrmm-python3-3.9.6-env/lib/python3.9/site-packages/allmydata/mutable/filenode.py", line 415, in _get_version
            raise UnrecoverableFileError("no recoverable versions")
        allmydata.mutable.common.UnrecoverableFileError: no recoverable versions

I believe this is just because the lease maintenance service tries to run too quickly and no storage servers have been contacted yet. ie, there is no data consistency or durability problem, nor really even an availability problem - only a sequencing problem where the lease maintenance service tries to use storage before the local node is really offering that service.

[meejah]

Sounds like the classic "ready" problem (from many other perspectives in Tahoe). IIRC, there are internal APIs for this ("connected enough" or similar)? Or maybe they were deleted along with magic-folders?