ADR-0033 — A to-box and a from-box beside the checkout: two foldyard-owned mounts, one per direction¶
- Status: Accepted (2026-10-21), with the operator's decisions in
§ Decided with the operator, which override this
proposal where they differ: the names are
to-box/andfrom-box/; a running VM with a stale mount set is refused; transcripts get one release of dual reading; the to-box's first tenant isfy monitor hook add NAME --to-box; and on the podman backend nothing about the VM's mounts changes. Built for 0.6.1. - Sources: the current mount set (
machine.py,machine_backend.py,box.py,verify.pyas cited below); the transcript bind-out (transcripts.py,plugins/claude.py,plugins/codex.py); the guest monitor's hooks (monitorhooks.py) and the stopgap file sink being built on themonitor-file-sinkbranch; Lima 2.2.0 and QEMU sources as cited in § Read-only enforcement. Related: 0001 (repo-only mounts), 0004 (worktrees at equal trust), 0022 and 0023 (what the host may take from the mount), 0025 (gVisor), 0032 (the guest monitor).
Context¶
The VM mounts exactly two host paths, both writable, both at their host path in the guest: the
main checkout and the worktrees root (machine._volumes, src/foldyard/machine.py:45; Lima
renders them "writable": true in _set_expr, src/foldyard/machine_backend.py:667-689; the
podman backend passes --volume src:dst, machine_backend.py:353-354). verify's mount audit
exempts exactly those two mountpoints (machine.guest_mounts, machine.py:51, used at
src/foldyard/verify.py:545) and fails on any other line whose mountpoint matches the operator's
home or a foreign host root (verify.py:41-56, :252-265).
Every byte that crosses between the host and the box therefore goes through the checkout, in both directions:
| direction | what | where today | problem |
|---|---|---|---|
| box → host | Claude transcripts | <checkout>/<here>/.devbox-claude/projects, bound to ~/.claude/projects in the box (box.py:1391-1399, plugins/claude.py:162-165) |
grep noise in the checkout; git clean -fdx and worktree remove take it; box-written content inside a tree the host walks |
| box → host | Codex sessions | <checkout>/<here>/.devbox-codex/sessions (box.py:1400, plugins/codex.py:194-197) |
same |
| host → box | monitor events (planned file sink) | the checkout, written with no-follow opens + an HMAC (the stopgap) | the box can delete, truncate or replace the file; the HMAC detects tampering, cannot prevent it |
| host → box | posture mirror, proxy CA, staged wheel + server.py |
.dev-mode.json (devmode.py:227), .devbox-ca/ (plugins/proxy.py:161-184), .devbox-foldyard/ |
every host write into the mount needs mountwrite's no-follow walk, and the box can still swap what it reads |
Unix permissions can't make one directory inside a writable mount read-only to the box: root in
the box is the VM user (rootless podman), the VM user owns every file on the share (Lima maps the
host uid), and an owner can chmod. Lima's writable flag is per mount. So a read-only area
needs a mount of its own.
Decision¶
Two foldyard-owned host directories, each its own VM mount, one per direction.
| from-box | to-box | |
|---|---|---|
| host path | <state dir>/from-box/ (~/.foldyard/<project>/from-box/) |
<state dir>/to-box/ |
| VM mount | writable, host path == guest path (as today) | read-only (writable: false), host path == guest path — Lima vz/virtiofs and qemu/9p only (below) |
| box mount | its own slot only, rw, at the agent's native path | whole to-box :ro at a fixed path (/run/foldyard/to-box) |
| written by | the box | the host only |
| read by | the host — as untrusted input | the box (and anything in the VM) |
| first tenant | Claude transcripts, Codex sessions | the monitor's event file sink (fy monitor hook add NAME --to-box) |
| what it guards against | grep noise; transcripts lost to git clean / worktree remove |
a box that tampers with evidence or plants symlinks to redirect host writes — refused by the hypervisor/host share server, not by politeness |
- Layout mirrors the posture dirs (
config.posture_dir,config.py:1905):from-box/main/{claude,codex}/…andfrom-box/worktrees/<wt>/{claude,codex}/…, so a worktree literally namedmaincan't collide. The to-box is project-level with per-worktree slots where a tenant needs one:to-box/monitor/…,to-box/main/…,to-box/worktrees/<wt>/…. - The host creates from-box slots with the no-follow walk, never
Path.mkdir(parents=True). Before a worktree's firstfy box upthe host makesfrom-box/worktrees/<wt>/{claude,codex}/…, and any box can plantfrom-box/worktrees/<wt>as a symlink to a host directory ahead of time (next bullet). A followingmkdirwould create directories inside the target. The bind itself resolves in the guest, where the link points at nothing mounted, so nothing leaks back, but it is still a host write redirected by a box-planted symlink. So slot creation walks from the from-box root withO_NOFOLLOW | O_DIRECTORY, creating each missing component under the parent's fd: a directory-only twin ofmountwrite.write, refusing (ELOOP/ENOTDIR) on a link. - Per-worktree slots are organisation, not isolation. Any box holds the engine socket and
can start a sibling container mounting the whole from-box or to-box VM path. That is within
0004's equal-trust tier (a box can
already edit every worktree), but it means the to-box must hold only what every box of the
project may see, and nothing secret ever: no HMAC key, no
host.envvalue, no token. - The box's
:rois defence in depth, not the boundary. The boundary is the VM mount being read-only on the host side (next section). The box isn't--privileged, so it can't remount its:robind, but over the socket it can create a container with the same VM path bound rw — which then meets the VM mount's own read-only flag, and below that the host's. - The state dir stays private. Only the two subdirectories are mounted, never
~/.foldyard/<project>/(which holdshost.env,monitor-relay.key, the allow-store). The audit's exact-mountpoint exemption (below) is what catches a regression that mounts the parent.
Read-only enforcement by backend¶
The question that decides whether the to-box is worth a mount: can root in the guest (a
container escape that reaches VM root — the VM user itself has no root after boot, guest-boot.sh
narrows its sudo to shutdown) write through the read-only mount? Only enforcement outside the
guest kernel counts; a guest ro mount option is one mount -o remount,rw away for guest root.
Read from source (Lima v2.2.0, the version CI pins in foldyard-e2e.yml; sshocker v0.3.11;
QEMU master; vfkit, krunkit; podman v5.0–v6.1). Only the vz row has since been run live,
against guest root (§ The vz test, Lima 2.1.3); the others are read
from source only:
| backend / path | how writable: false reaches the hypervisor |
enforced outside the guest? | confidence |
|---|---|---|---|
Lima vz + virtiofs (macOS default) |
vz.NewSharedDirectory(location, !writable) — pkg/driver/vz/vm_darwin.go#L674, #L704; Apple: readOnly — "exposed as read-only to the guest" (VZSharedDirectory) |
yes — the flag is in the host's device config; the virtiofs server is in the host's Virtualization process | high — a documented contract, and the vz test (Lima 2.1.3) held against guest root |
Lima qemu + 9p (Linux and WSL2; 9p is the 2.2.0 default for qemu, qemu_driver.go#L239-L253) |
-virtfs …,readonly=on — pkg/driver/qemu/qemu.go#L990-L998 |
yes — QEMU's 9p server returns EROFS for every non-read op and for write/append/trunc opens (hw/9pfs/9p.c is_read_only_op, #L2186-L2191) |
high |
Lima qemu + virtiofs (Linux, opt-in) |
none: virtiofsd started without --readonly ("not supported … yet", qemu.go#L1000-L1001) |
no — guest ro only |
high |
Lima krunkit |
none (krunkit_darwin_arm64.go#L98) |
no | high |
| Lima reverse-sshfs (old instances / explicit) | host sftp-server -R (sshocker reversesshfs.go#L213-L216) |
writes yes, but -d is a chdir not a chroot: guest root can read the rest of the home (Lima's own warning) |
high — and disqualifying for every mount, not just this one |
podman applehv / libkrun (macOS) |
:ro parsed, then dropped: vfkit hard-codes NewSharedDirectory(dir, false) (vfkit virtio.go#L194); the guest unit doesn't even mount ro |
no — not even guest-side | high |
podman qemu (Linux, podman ≥ 5.2) |
virtiofsd without --readonly ("TODO: Honor hostmnt.readonly", virtiofsd.go#L81) |
no — guest ro only (5.0–5.1 used 9p readonly: yes) |
high |
| podman WSL | no volume mounts at all | n/a | high |
Lima also writes ro into the guest fstab via cloud-init for 9p and virtiofs
(cidata.go#L221-L226),
which is the guest-only layer the VM user meets first.
So:
- The to-box exists only on Lima
vz+virtiofs andqemu+9p. foldyard doesn't setmountTypetoday (_set_exprwrites mounts, sizing andvmType), so a Lima default change or an operator's[machine].vmtype = "krunkit"would silently downgrade it. Pin.mountTypewith the vmType (vz → virtiofs, qemu → 9p), and on any other pair, or on the podman backend, mount no to-box: tenants fall back to their signed-sink-in-the-checkout form (--file), andfy doctor'sto-boxrow (andfy monitor hook add --to-box) says why. A to-box the guest can write is worse than none, because it reads as tamper-proof. - Never nested in a writable mount. A read-only mount inside a writable one stays writable
through the parent (Lima's
--mounthelp: "do not specify directories that overlap"). WithFOLDYARD_STATE_DIRpointed inside the checkout or the worktrees root that is exactly what happens, soensurerefuses it. - Freshness: a read-only 9p mount defaults to
cache=fscache(defaults.go#L803-L811), so a reader in the guest may see host appends late. Fine for evidence, wrong for anything the box polls for liveness; pin9p.cacheif a tenant needs it. - Only the operator can widen it: Lima merges mounts across default, instance and
~/.lima/_config/override.yaml, and the most permissivewritablewins (defaults.go#L139-L141). The verify row below catches it, and so doesensure: it reads the instance's EFFECTIVE config (limactl list's mergedconfig), so an override that widens the to-box or swaps its share type is refused atfy uptoo.
verify: two exemptions, one new assertion¶
- Exemptions:
machine.guest_mountsreturns four mountpoints instead of two. Still exact mountpoint only, so the parent state dir, a sibling under it or a nested bind are still FAILs. The in-box run can't resolve the host's state dir (FOLDYARD_STATE_DIRmay move it), sofy box upbakes the two VM paths into the box env, as it bakesFY_HOST_HOMEfor the audit today (box.py,env_args). - New assertion (the to-box is read-only), from inside the box and from the host-side run:
- PID 1's mount table shows the to-box mountpoint with
roin its options (verify.py:221already reads it). Necessary, not sufficient — it's the guest's own flag. A guest that showsrwthere is a WARN while (2) still refuses (the vz test shows the host's enforcement holds without the flag), naming it as a sign that something in the guest remounted it; a FAIL only comes from (2). - A
--privilegedprobe container that binds the to-box VM path rw cannot create a file there, andmount -o remount,rwon it fails. The VM-level mount is in the VM's initial user namespace, so in the container's namespace its read-only flag should be locked (the row is what checks that expectation); a successful write is a FAIL that names the file it left. Any errno counts as refused (EROFS,EPERM,EACCES): which one depends on whether the guest's flag is still set (§ The vz test). - In the box:
/run/foldyard/to-boxrefuses a write (the:robind). Under gVisor the probe runs in the sandbox and (2)'s remount says nothing about the VM: report it as N/A like the PID-1 audit (verify.py:234-247); (1) is read host-side, (3) still holds. - What verify can't prove: host-side enforcement against guest ROOT. The box never has it,
so that proof is a host-tier e2e (next item), not a verify row.
fy docs securitysays so in "What verify doesn't prove". - The negative stays a negative.
test_verify_e2e's home-mounting VM mounts the home at/mnt/c, not at an exempted path, so it still FAILs. The second negative — the same VM with the to-box flippedwritable: true(lima_editon the stopped instance, restored in the fixture) FAILs the write row — rides the host-tier proof's red-first control below, which boots that VM anyway (one restart, not two).
Host-tier proof (new e2e, Lima on QEMU/9p in CI, vz/virtiofs on a macOS host)¶
Root in the guest is boot-time only, and the harness must not break that rule by adding a sudo
path. So the test records a test-only mode: system provision script under its own marker
(the 0032 precedent: never mentions the other
markers, always exits 0), which at boot, as root: mount -o remount,rw the to-box; tries to
create, append to, truncate, chmod, chown, rename and delete a host-seeded file; creates a
symlink and a hardlink; writes each result to /run/fy-test/to-box-ro. The host then asserts
every attempt was refused and that the seeded file's bytes, mode and inode are unchanged on
the host, then removes the script and restarts. Same harness, writable: true flipped, must show
the writes landing (the red-first control), and fy verify against it FAILs. Lives beside
test_verify_e2e.py as test_mounts_e2e.py; skips on a pair with no to-box (and the host tier
is lima-only, so never runs on the podman backend). First run by hand on vz/virtiofs, Lima 2.1.3:
every write refused, the seeded file unchanged.
Migration¶
One VM restart on Lima and one box recreate; on the podman backend, the box recreate alone.
- Mount set changes apply on a restart, not a recreate (Lima). There is no mount-set
staleness check today:
ensureonly warns when the worktrees root is missing (machine.py:725-728), andrecreate's docstring calls mount sets "init-only" (machine.py:928-931). Buttest_verify_e2e's fixture adds a mount withlimactl edit --seton the STOPPED instance and the next start honours it (tests/test_verify_e2e.py:76-98). So: a new_record_mounts()inensure, beside_record_provisioning()(machine.py:548): stopped →limactl editthe mount set; running with a different set → refuse withfy machine stop && fy up, as stale provisioning is refused. Named volumes survive. Needs a mount reader that sees thewritableflag (LimaBackend.mounts,machine_backend.py:531, reads only paths) — built asLimaBackend.mount_set, fromlimactl list's effective config. As built it compares and replaces only the two new entries and the pinnedmountType: the checkout and worktrees-root mounts stay as the VM was created (the host tier drives one VM from several example copies, and an operator's second clone of a project shares its VM — refusing those would be new breakage for nothing this ADR needs), and an entry someone else added is kept. A restart suffices becauselimactl editrefuses a running instance and otherwise rewriteslima.yaml, and vz and qemu both build their shares from it at every start (cmd/limactl/edit.go#L68-L69,vm_darwin.go#L660-L720,qemu.go#L982-L1003).recreate's "init-only" docstring is then true only of the podman backend. - podman backend: mounts are fixed at
podman machine init;podman machine sethas no volume flag (set.go#L48-L90). Decided: nothing about its mounts changes — no from-box even after a recreate, and never a to-box (above). Its boxes keep the legacy in-checkout bind-out, so the dual-read path is the only path there;fy monitor hook add --to-boxsays it needs the lima backend and points at--file. - The boot scripts don't change here. The mounts need no boot-script change. The deferred
rewording of
guest-boot.sh/machine-wall.sh("wall", "Mac"; CLAUDE.md) ships on its own (#114) in the same release (settled item 3), so its new# fy-provisionid and this mount set are both taken by the onefy machine stop && fy up. - Transcripts: the box binds
from-box/<slot>/claude/projectsinstead of<checkout>/<here>/.devbox-claude/projects(likewise Codex). For one release the host also reads the old bound-out dirs:sync_current, the supervisor sweep,fy transcriptsandworktree removearchive both, the new one first. The firstfy box upon the new layout archives the old dir once and prints that it can be deleted; it never moves it into the from-box.DEVBOX_TRANSCRIPTS/DEVBOX_CODEX_TRANSCRIPTSkeep working. The.devbox-claude//.devbox-codex/ignore lines (init.py:47-49) stay for that release, then go. worktree removearchives the slot, then deletes it (the checkout deletion no longer takes it). The delete must not follow symlinks the box planted:shutil.rmtreeonly whereshutil.rmtree.avoids_symlink_attacksis true, else refuse.- Monitor file sink:
fy monitor hook add NAME --to-boxwritesto-box/monitor/NAME.jsonl(the box reads it at/run/foldyard/to-box/monitor/NAME.jsonl) — opt-in, never a default (settled item 1). It keeps its HMAC: the to-box protects the file at rest on this host; the signature is what a reader re-verifies once a copy has left it. The no-follow writes stay (cheap, and on a backend with no to-box the sink is back in the checkout, where they matter). A sink path the operator points at a box-writable tree (the checkout, the worktrees root, the from-box) gets a warning. - Later to-box tenants, each its own change: the posture mirror, the staged proxy CA, the
staged wheel and
server.py. Each removes amountwritecall site and a file the box can swap.
From-box content is box-written: how the host reads it¶
Treat everything in the from-box like a request body from the box. The reader this needs already
exists: #109 replaced the transcript sync's rsync -rtu --no-perms with a no-follow walk by
directory fd (transcripts._open_source, _mirror). The from-box reuses it with the from-box root as
the anchor, so the from-box adds no new reader:
| hazard | before #109 (rsync -rtu of the bound-out dir) |
#109's walk, which the from-box reader keeps |
|---|---|---|
| symlinked file | skipped (no -l: "non-regular file") |
skipped |
symlinked directory on the path (…/claude → ~/.ssh) |
followed: rsync opened src/ through it and copied the target into ~/.claude/projects |
every component opened O_NOFOLLOW \| O_DIRECTORY relative to its parent's fd, from the anchor down; a link is refused, however late it is swapped in |
| FIFO / device | skipped (no -D) |
opened O_NONBLOCK, re-checked with fstat S_ISREG, else skipped (a FIFO blocks a plain open) |
| hardlink | n/a | can't cross the mount; harmless |
| size | unbounded | a per-file cap (MAX_FILE_BYTES) and a free-space floor (MIN_FREE_BYTES); a file over the cap is skipped and reported, never truncated silently |
| content | copied into ~/.claude/projects, which a HOST claude --resume loads |
unchanged by this ADR, but say it: resuming a box session on the host feeds box-written text to a host-privileged agent |
Disk exhaustion is unchanged: the box can already fill the host disk through the checkout.
Alternatives considered¶
| option | box → host | host → box tamper-proof | cost | verdict |
|---|---|---|---|---|
| bind-out in the checkout (today) | live, durable | no | grep noise; lost to git clean; host walks a box-writable tree |
the status quo being replaced |
| bind-out under the worktrees root | live | no | a reserved dir name inside a box-writable tree; the same tamper surface | rejected — a hack, and no to-box |
| dedicated mounts (this) | live | yes, where the backend enforces it | one Lima restart; two audit exemptions; a new e2e | proposed |
| named volume + tar pulled over ssh | seconds of loss window | n/a (one way) | the host parses a box-produced tar stream; a pull loop per worktree | rejected — loses the tail on a crash, adds a parser |
| read the VM's disk image on the host | only when stopped | n/a | no btrfs tooling on macOS; host parses an attacker-shaped filesystem | rejected |
| signed sink in the checkout, no mounts | — | detects, can't prevent | none now | the stopgap; kept as the signature, not as the place |
Consequences¶
fy docs security's boundary section ("mounts only your repo") gains two foldyard-owned directories, one of them read-only, and their verify rows.- The checkout stops carrying box-written content; grep and
git statussee only the code. - Evidence the host publishes can no longer be altered by the box on a host-enforcing backend; on a backend with no to-box it is no worse than today, and the HMAC still detects.
- One more thing the host must read defensively (the from-box) — but it is one place, not "anywhere in the checkout".
- The podman backend gets neither directory (decided item 6): no current podman provider enforces read-only outside the guest, applehv/libkrun don't apply it at all, and its mount set changes only with a machine recreate. Its transcripts stay bound out in the checkout.
Settled with the operator (2026-10-21)¶
- The box sees the monitor's events. The reader is the in-box harness: control-arena's flip experiment runs in the box, builds each step's kernel digest from the events file and sends it to the scoring model. What is observed is the BashArena task containers, which are the box's SIBLINGS on the engine and have no mounts. They can't read the to-box; only the box can, and it is the experimenter. A box observing ITSELF (a monitor over the box's own agent) is a different deployment. There the to-box would tell the watched what was caught, so a hook only ever delivers into the to-box because the operator configured it to, never by default.
- Your computer verifies; the box doesn't. No Ed25519. Tampering is unlikely here, so the
response is loud and manual rather than automatic:
fy monitor check FILEon your computer (#107) reports the first bad line, and the operator investigates, then sets the file aside and starts a fresh one. Two follow-ups belong to the sink, not to this ADR: - The supervisor could compare the file's last line with the recorded MAC before each append, so a cut or edit shows in the supervisor log, a notification and doctor, not only at check time. It would keep appending, so evidence isn't lost.
- A host-side
rotatethat records where a fresh file starts, so a legitimately rotated file passescheckand a deleted one still fails it. On the to-box the signature stays for portability. A reader of the results can re-verify, which "it was mounted read-only" doesn't let them do. - The deferred
guest-boot.sh/machine-wall.shrewording rides this restart. The operator runsfy machine stop && fy uponce for both. (It ships as its own change, #114, in the same release.) - A live vz test runs before building: guest root remounts read-write, remounts the virtiofs tag fresh, then writes. The result is recorded below.
Decided with the operator (2026-10-21)¶
The open questions this ADR was proposed with, answered — plus what building it settled:
- Names:
to-box/andfrom-box/, not inbox/outbox: the guest already has/var/lib/fy-monitor/inbox(the Tetragon delivery inbox,monitor.py:60, ADR-0032), which keeps its name. Host paths~/.foldyard/<project>/to-box/(read-only to the VM and the box) and~/.foldyard/<project>/from-box/(box-writable); in the box,/run/foldyard/to-box. - A running VM whose mount set is stale is REFUSED — fail closed, like stale provisioning:
fy upand friends stop withfy machine stop && fy up, volumes kept._record_mounts()sits beside_record_provisioning(), on the Lima backend only. - Transcripts: one release of dual reading. New boxes bind out into the from-box; the host sync reads the from-box and the legacy in-checkout bind-out for this release, and the next one drops the legacy path. #109's no-follow walk, #111's own-project-folder rule and #113's per-project Codex home apply to the from-box exactly as to the legacy path; slots are made with the no-follow mkdir walk; a symlinked slot is refused.
- The to-box's first tenant is the monitor's signed file sink:
fy monitor hook add NAME --to-boxwrites<state dir>/to-box/monitor/NAME.jsonland prints the box-side path. The signature stays (portable evidence);--filestays for other paths. Without a to-box it says why and points at--file. mountTypeis pinned per vmType (vz → virtiofs, qemu → 9p), and no to-box is mounted on a combination that doesn't enforce read-only outside the guest.- The podman backend changes nothing about the VM's mounts (some consumers run it, and there a mount-set change is a machine recreate): no to-box, no from-box, no stale-mount refusal. Its transcripts keep the legacy bind-out, and the dual-read path is simply the only path there.
The vz test (2026-10-21)¶
Setup: Lima 2.1.3, macOS 27.0.1, a Fedora 44 guest (kernel 6.19.10) cloned from the base VM, with
one scratch directory mounted writable: false and mountType: virtiofs. As root in the guest,
17 kinds of write were tried, among them create, append, overwrite, mmap, truncate, chmod,
chown, mtime, xattr, symlink, hard link, FIFO, mkdir, rename and rm. They ran three ways: on
the mount as Lima made it; after mount -o remount,rw; and on the virtiofs tag freshly mounted
-o rw. Every write failed every time, and the directory on the host was unchanged by stat,
sha256 and xattrs after each run. The same writes against a directory inside the guest all
succeeded, which shows the test could see a write.
Verdict: on vz, writable: false is enforced on the host side, against root in the guest.
Three things follow for verify and the e2e:
- The guest's mount table can say the to-box is writable when it isn't. Both the remount and
the fresh mount return 0 and show
rwin/proc/mounts. Only an actual write shows the truth, so the assertion is a write attempt, not theroflag. - The refusal isn't always
EROFS. It isEROFSwhile the guest'sroflag is set. Once the guest drops it, the host refuses withEPERM(create,chmod, links, rename,rm) orEACCES(opening for write). A probe that reads "refused" asEROFSalone would misjudge the case that matters. - Ownership seen in the guest says nothing. It reports the caller as owner (
root:rootto root, the VM user to the VM user).
QEMU with 9p still rests on reading the source. CI's host tier is where that gets proved.