Provider pack store: same-machine-only support boundary
The boundary.~/.traycer’s provider pack store may be shared by any number
of Traycer host processes on one machine. It is not supported on storage
shared across machines — NFS, SMB, a roaming home directory, or a container
volume mounted into more than one node.
This is a support statement, not a warning about performance. Sharing the store
across machines breaks a correctness precondition, and it fails in a shape that
misdirects whoever investigates.
Why
Concurrent downloads of the same pack are serialized by a cross-process lease (traycer-host/src/domain/providers/provider-pack-lease.ts). A peer that finds
an existing lease decides whether to wait or to take it over by asking this
kernel whether the recorded pid is still running, and whether its
process-start identity still matches.
That question only means something for a process on the same machine. Across a
shared mount, a pid belonging to a live host on machine A either does not exist
on machine B, or exists as an entirely unrelated process. Either answer can read
as the owner is dead.
The symptom you will actually see first
verification is the alarming one, and it is not the likely one. The lease
election notices a stalled owner long before two appenders finish corrupting a
partial, so the first verdict this misconfiguration produces is usually
live-owner-stalled:
another Traycer process using this Traycer folder stopped making progress on the download. Try again to pick it up here.That sentence is true, and its “using this Traycer folder” is doing careful work: the lease record carries no machine identity, so the host genuinely cannot say the sibling is local. On a single-machine deployment the two read the same; in the one topology this document exists to describe, “on this device” — which is what the copy used to say — is false, and it points the reader at the wrong machine. So the escalation order is: repeated
live-owner-stalled on a store you did not
know was shared → adopt-and-append → verification. If a support thread opens
on verification, ask whether it was preceded by stalls.
Reading the evidence on disk
Everything needed to confirm or rule this out is in the partial’s sidecar,<store>/.tmp/dl-<digest>.lease:
The tell: a
pid that does not exist on the machine reading the lease, in a
store whose heartbeatAt keeps advancing. A dead owner’s heartbeat stops. An
owner that is alive on another node keeps stamping while this kernel insists the
pid is gone — and that combination cannot happen on a single machine.
Confirm the mount before anything else: df ~/.traycer (a network filesystem
type, or a device that is not local), or on Linux
findmnt -T ~/.traycer -o TARGET,SOURCE,FSTYPE.
Getting a stuck host moving again
- Stop every host sharing the store. Not just the one reporting the error — the whole point is that the other one is invisible to it.
- Delete the partials, not the store.
rm ~/.traycer/…/.tmp/dl-*. Both the in-flight bytes (dl-<digest>.<host-instance>) and the sidecar (dl-<digest>.lease) live under.tmp, so one glob covers them. Verified blobs are content-addressed and safe to keep; only the in-flight bytes are suspect. - Move the store off the shared mount before restarting, or point each
machine at its own
~/.traycer. Restarting without this reproduces the failure on the next convergence. - A
verificationverdict raised by this cause needs no registry investigation. That is the whole reason this page exists — confirm the mount first, and only escalate to supply chain if the store is local.
The store also stops reclaiming disk
Every reclaimer in the store — stranded download partials, and the content-addressed archives underblobs/ — runs only inside the
sole-participant barrier, at boot. That barrier is what makes deleting bytes
another process may be reading safe, and on a shared store it never reports
sole: the second host is always there, so the barrier returns
other-live-participant and both hosts skip.
So a shared store is not merely at risk of corrupt downloads. It never frees
anything, and it accumulates one multi-gigabyte archive per pack per pin bump,
on every machine’s view of the same directory, indefinitely. A store on a shared
mount that has been in use for a while is usually diagnosable by size alone.
A store that is local and still growing is a different problem — look for
provider pack blob reclamation skipped in the host log, which names the
retention root that could not be read.
Why it matters more than a failed download
Two appenders on one partial produce bytes that hash to nothing, and the install path classifies a digest mismatch asverification — a definitively-invalid
verdict whose entire meaning is “the published artifact does not match its
signature”.
So a deployment misconfiguration reaches the operator wearing the shape of a
supply-chain tampering alarm. That is the most expensive possible way to be
wrong about it: the honest response to a verification verdict is to stop and
investigate the registry, and none of that investigation will find anything,
because nothing is wrong with the artifact.
What is not enforced, and why
Nothing checks this at runtime. Liveness across a machine boundary is exactly the fact the mechanism cannot establish, so a guard would need the evidence it is missing. The boundary is documented instead.Upgrade path, if the boundary is ever lifted
Record a stable machine identity in the lease record alongside the pid, and have the liveness probe fail to indeterminate — never to “dead” — when the record’s machine is not this one.indeterminate already has a defined meaning throughout the lease module
(holdElection reports it rather than inventing progress), and it carries the
right bias: an unproven owner keeps its lease, so the worst case becomes a
download that waits rather than two that corrupt each other.
Deliberately not built today. Every supported deployment is single-machine, so
the machinery would sit unused, and an unused enforcement path is one nobody
notices breaking.