The sandbox that killed my emulator: why I run Cuttlefish with --enable_sandbox=false

9 Aug 2026 · aospcuttlefishcrosvmseccompdebugginglinux

Last time I got the Cuttlefish host packages to build on Ubuntu 26.04. This time the device wouldn’t boot on the same machine — and the failure mode is worth writing down, because it’s not really about Cuttlefish. It’s about what happens to any seccomp-sandboxed program when the kernel and libc underneath it are newer than the sandbox’s allowlist. Everything below is against android-15.0.0_r36 on Ubuntu 26.04, kernel 7.0.0-generic, glibc 2.42.

The symptom: a launch that lies

First launch, by the book:

launch_cvd --daemon

No errors. Host processes running. And adb devices: empty. Nothing to attach to, nothing in logcat, no complaint anywhere I looked at first.

Trying again:

Instance directory files in use. Try `cvd reset`?
Observed PIDs: 9277, 9280, 9281, ...

That second error is actually the first useful clue: those PIDs were the previous launch, still alive. run_cvd, the modem simulator, the adb connector, log tees — the whole host-side entourage was up and holding the instance directory. The launcher considered the launch a success. Something more specific than “it didn’t work” was going on.

Narrowing it down

The Cuttlefish host stack is a crowd of cooperating processes, so “is it running?” is the wrong question — the right one is “which layer is dead?” Three checks, each one level deeper:

1. Is anything listening where adb should connect? Cuttlefish exposes the guest’s adbd on TCP 6520 via a proxy:

ss -ltnp | grep 6520
# nothing

The adb server was running; the device port had no listener. So the guest side of that proxy never came up.

2. What does the launcher log say? ~/cuttlefish/instances/cvd-1/logs/launcher.log was an infinite loop of two processes politely failing:

webRTC ... vsock_connection.cpp:222] Failed to connect: No such device
adb_connector ... transport message failed, response body: device '0.0.0.0:6520' not found

vsock is the host↔guest socket family — “No such device” from a vsock connect means there is no guest. The host services weren’t broken; they were waiting, forever, for a VM that didn’t exist.

3. Did the VM ever boot? The definitive check:

wc -l ~/cuttlefish/instances/cvd-1/logs/kernel.log
# 0

Zero bytes of kernel output. The guest never executed a single instruction. And pgrep crosvm — the actual hypervisor process — came back empty, while everything around it lived on.

The actual error, one second in

Grepping the launcher log for crosvm’s own lines found the death certificate, timestamped one second after launch:

ERROR crosvm] exiting with error 1: the architecture failed to build the vm
    failed to create a PCI root hub: failed to create proxy device:
    Failed to configure tube: failed to receive packet:
    Connection reset by peer (os error 104)

“Proxy device” is the tell. crosvm runs in multiprocess mode: each virtual device (block, net, serial, rng, …) is forked into its own child process, and each child is locked into a minijail sandbox with a seccomp policy — a per-device allowlist of syscalls, shipped as .policy files next to the crosvm binary:

out/host/linux-x86/usr/share/crosvm/x86_64-linux-gnu/seccomp/
├── block_device.policy
├── net_device.policy
├── serial_device.policy
└── ...

Under seccomp’s default-deny, a syscall that isn’t on the list kills the process with SIGSYS. No stderr, no exception, no exit message — the child just stops existing. The parent’s next read on the control socket gets ECONNRESET, which it can only report as “Connection reset by peer”. That’s why the error message describes the consequence (a dead pipe) and not the cause (a forbidden syscall).

And why were syscalls forbidden? Because the allowlist is frozen and my machine is not. Those policy files were written for the kernels and glibc versions Android’s build infrastructure ran when android-15.0.0_r36 was cut. glibc 2.42 on kernel 7.0 happily uses syscalls that are newer than the policy — modern glibc aggressively adopts things like fresh clone3/futex variants and new *at2-style entry points as the kernel grows them. The program didn’t change; the floor under it did. First device child makes an unlisted syscall during setup, SIGSYS, dead, and the VM build aborts.

I couldn’t even catch the SIGSYS directly — audit logging was disabled on the box, so the kill is invisible in the journal too. The diagnosis rests on the shape of the failure: multiprocess sandbox + instant child death + ECONNRESET at proxy-device creation + a distro years newer than the toolchain. Every piece points the same way.

The fix, and why it’s the right one for a dev box

launch_cvd --daemon --enable_sandbox=false

Cuttlefish passes --disable-sandbox down to crosvm; device processes run unjailed; the VM builds; the device boots and shows up as 0.0.0.0:6520.

Now the opinionated part: on a developer machine running a bleeding-edge kernel, this sandbox is not worth keeping. Not because sandboxing is bad — because this sandbox’s threat model and failure mode are both wrong for this machine:

The threat model isn’t yours. The seccomp jail protects the host from a compromised or malicious guest image — sensible when a fleet boots untrusted images, or when crosvm runs in ChromeOS with real tenants. My guest is an image I built from source an hour ago, on a box whose whole purpose is to run it. The sandbox defends me from myself.
A frozen allowlist is a bet that the world stays still. A seccomp policy is an exhaustive list of “everything this program will ever ask the kernel for,” compiled at release time. Run it on a kernel + libc from two years later and the bet quietly expires. And unlike a version check, it doesn’t expire with an error message — it expires with SIGSYS.
The failure mode is maximally misleading. This is the part that turns a design trade-off into an afternoon of debugging: the sandbox converts “unknown syscall” into “child vanished”, which surfaces as “connection reset”, which surfaces as “launcher running but adb empty”. Each layer reports the layer below it truthfully and uselessly. A security mechanism whose failure presents as an unrelated subsystem’s flakiness has negative value on a machine where you are the one holding the pager.

The general rule I’m taking from this: pinned userspace + rolling kernel = seccomp time bomb. AOSP is the extreme case — its whole philosophy is hermetic, pinned, prebuilt everything — but the same applies to any older sandboxed binary you carry onto a new distro: browsers’ zygotes, container runtimes with hand-rolled seccomp profiles, gVisor-ish shims. If a multiprocess program dies at startup with a broken-pipe-flavored error and its children use seccomp, suspect the allowlist before you suspect the code.

What I’m taking away

“Running” is not “booted”. The launcher, the adb connector, the modem simulator — none of them are the VM. kernel.log at zero bytes is the single fastest truth test for whether a guest ever existed.
Read errors as chains, not messages. “Connection reset by peer” was four causal steps away from the real problem. The useful skill isn’t knowing the answer, it’s refusing to stop at the first layer that produced text.
SIGSYS is silent by design. If audit logging is off, a seccomp kill leaves no trace but the corpse. When a child process dies instantly and wordlessly inside a jail, the jail is a suspect precisely because there’s no evidence.
Security mechanisms age like dependencies. A seccomp policy is a dependency on the kernel ABI staying enumerable. Treat “old pinned sandbox on new kernel” the way you’d treat “old pinned OpenSSL on new distro” — as a compatibility risk to check proactively, not a background invariant.
Know which machine you’re on. The same flag that’s a bad idea on a shared CI fleet booting arbitrary images is the obviously correct default on a single-user dev box booting its own build. Defense-in-depth is a budget; spend it where the threat is.