Skip to main content

Provider pack store: same-machine-only support boundary

The boundary. ~/.traycer’s provider pack store may be shared by any number of Traycer host processes on one machine. It is not supported on storage shared across machines — NFS, SMB, a roaming home directory, or a container volume mounted into more than one node. This is a support statement, not a warning about performance. Sharing the store across machines breaks a correctness precondition, and it fails in a shape that misdirects whoever investigates.

Why

Concurrent downloads of the same pack are serialized by a cross-process lease (traycer-host/src/domain/providers/provider-pack-lease.ts). A peer that finds an existing lease decides whether to wait or to take it over by asking this kernel whether the recorded pid is still running, and whether its process-start identity still matches. That question only means something for a process on the same machine. Across a shared mount, a pid belonging to a live host on machine A either does not exist on machine B, or exists as an entirely unrelated process. Either answer can read as the owner is dead.

The symptom you will actually see first

verification is the alarming one, and it is not the likely one. The lease election notices a stalled owner long before two appenders finish corrupting a partial, so the first verdict this misconfiguration produces is usually live-owner-stalled:
another Traycer process using this Traycer folder stopped making progress on the download. Try again to pick it up here.
That sentence is true, and its “using this Traycer folder” is doing careful work: the lease record carries no machine identity, so the host genuinely cannot say the sibling is local. On a single-machine deployment the two read the same; in the one topology this document exists to describe, “on this device” — which is what the copy used to say — is false, and it points the reader at the wrong machine. So the escalation order is: repeated live-owner-stalled on a store you did not know was shared → adopt-and-append → verification. If a support thread opens on verification, ask whether it was preceded by stalls.

Reading the evidence on disk

Everything needed to confirm or rule this out is in the partial’s sidecar, <store>/.tmp/dl-<digest>.lease: The tell: a pid that does not exist on the machine reading the lease, in a store whose heartbeatAt keeps advancing. A dead owner’s heartbeat stops. An owner that is alive on another node keeps stamping while this kernel insists the pid is gone — and that combination cannot happen on a single machine. Confirm the mount before anything else: df ~/.traycer (a network filesystem type, or a device that is not local), or on Linux findmnt -T ~/.traycer -o TARGET,SOURCE,FSTYPE.

Getting a stuck host moving again

  1. Stop every host sharing the store. Not just the one reporting the error — the whole point is that the other one is invisible to it.
  2. Delete the partials, not the store. rm ~/.traycer/…/.tmp/dl-*. Both the in-flight bytes (dl-<digest>.<host-instance>) and the sidecar (dl-<digest>.lease) live under .tmp, so one glob covers them. Verified blobs are content-addressed and safe to keep; only the in-flight bytes are suspect.
  3. Move the store off the shared mount before restarting, or point each machine at its own ~/.traycer. Restarting without this reproduces the failure on the next convergence.
  4. A verification verdict raised by this cause needs no registry investigation. That is the whole reason this page exists — confirm the mount first, and only escalate to supply chain if the store is local.

The store also stops reclaiming disk

Every reclaimer in the store — stranded download partials, and the content-addressed archives under blobs/ — runs only inside the sole-participant barrier, at boot. That barrier is what makes deleting bytes another process may be reading safe, and on a shared store it never reports sole: the second host is always there, so the barrier returns other-live-participant and both hosts skip. So a shared store is not merely at risk of corrupt downloads. It never frees anything, and it accumulates one multi-gigabyte archive per pack per pin bump, on every machine’s view of the same directory, indefinitely. A store on a shared mount that has been in use for a while is usually diagnosable by size alone. A store that is local and still growing is a different problem — look for provider pack blob reclamation skipped in the host log, which names the retention root that could not be read.

Why it matters more than a failed download

Two appenders on one partial produce bytes that hash to nothing, and the install path classifies a digest mismatch as verification — a definitively-invalid verdict whose entire meaning is “the published artifact does not match its signature”. So a deployment misconfiguration reaches the operator wearing the shape of a supply-chain tampering alarm. That is the most expensive possible way to be wrong about it: the honest response to a verification verdict is to stop and investigate the registry, and none of that investigation will find anything, because nothing is wrong with the artifact.

What is not enforced, and why

Nothing checks this at runtime. Liveness across a machine boundary is exactly the fact the mechanism cannot establish, so a guard would need the evidence it is missing. The boundary is documented instead.

Upgrade path, if the boundary is ever lifted

Record a stable machine identity in the lease record alongside the pid, and have the liveness probe fail to indeterminate — never to “dead” — when the record’s machine is not this one. indeterminate already has a defined meaning throughout the lease module (holdElection reports it rather than inventing progress), and it carries the right bias: an unproven owner keeps its lease, so the worst case becomes a download that waits rather than two that corrupt each other. Deliberately not built today. Every supported deployment is single-machine, so the machinery would sit unused, and an unused enforcement path is one nobody notices breaking.