adb remount: the illusion of a writable /system

11 Aug 2026 · aospadboverlayfsavbcuttlefish

The platform-developer additions to adb you never need as an app developer: adb remount, adb sync. My Linux-shaped intuition said adb remount must be roughly mount -o remount,rw /system with ceremony around it. That intuition is about fifteen years out of date. On a modern Android device a read-write remount of /system is impossible for three independent reasons, and the command is actually a small orchestrator that defeats each one in turn.

Three locks on the door

Lock 1: dm-verity. With Verified Boot, system, vendor, product and friends aren’t mounted from the block device directly — they’re mounted through a device-mapper target that hash-verifies every block it reads against a Merkle tree, whose root hash is signed inside vbmeta.img. Write one block and every future read of it fails verification (or panics the device, depending on verity mode). Writability isn’t merely discouraged; the crypto makes written data unreadable.

Lock 2: the filesystem has no write path. Build systems right-size images and dedupe them at the block level, and increasingly they use filesystems that are read-only by construction. My device is a clean example — in device/google/cuttlefish/shared/device.mk:

TARGET_RO_FILE_SYSTEM_TYPE ?= erofs

and shared/BoardConfig.mk fans that out to system, vendor, product, odm and the dlkm partitions. EROFS (“Enhanced Read-Only File System”) has no write support at all. There is no rw to remount to — the kernel would laugh at you.

Lock 3: no room anyway. Even on targets that still use ext4, the image is generated with essentially zero free blocks, and it’s a logical partition packed inside super.img alongside its siblings. “Just grow the filesystem” is a whole dynamic-partition resize, not a flag.

So “make /system writable” can’t be a mount flag. What is it?

What actually runs

The adb remount you type doesn’t do the work inside adbd — the daemon just executes /system/bin/remount on the device (the dispatch is in packages/modules/adb/daemon/services.cpp). That binary lives in system/core/fs_mgr/fs_mgr_remount.cpp, and it’s a fun read: the same binary is disable-verity, enable-verity and set-verity-state too — it switches behavior on argv[0] (see the progname check around line 58). One tool, four names, which already tells you verity and remounting are two halves of one mechanism.

Its job, in order:

1. Insist on root. Hence adb root first, and hence userdebug/eng builds only — on a user build adbd can’t elevate, and the verity toggle is refused. This is most of the reason the course has us build userdebug at all.

2. Deal with verity. The verity check happens at mount time, so disabling it is a persistent, boot-time decision: disable-verity flips a user-settable flag in the AVB metadata via avb_user_verity_set() and asks for a reboot. adb remount -R is the ergonomic shortcut — disable verity and reboot in one step; after the device comes back you run adb remount again and it proceeds. (On a locked production device this flag can’t be set — lock 1 stays locked, by design.)

3. Probe each partition and pick a strategy. Straight from system/core/fs_mgr/README.overlayfs.md — an ext4 partition with genuine free space gets an honest MS_REMOUNT and we’re done. Anything read-only-by-construction (my EROFS case) or packed full gets the interesting path: overlayfs.

4. Conjure backing storage. Writes have to live somewhere, so remount creates a brand-new logical partition named scratch out of the leftover free space in super, formats it, and mounts it at /mnt/scratch (the setup logic is in system/core/fs_mgr/fs_mgr_overlayfs_control.cpp). No room in super? It falls back to backing the overlay with an image file on /data.

5. Mount the overlay. For each partition: the original, verified, untouched image becomes the read-only lower layer; a directory under /mnt/scratch/overlay/ becomes the writable upper layer. Writes are copy-up per file — modify one byte of framework.jar and the whole file is copied into scratch, where the modified copy shadows the original. The image itself is never written. Lock 2 and lock 3 aren’t so much picked as bypassed entirely.

The illusion is complete: adb push into /system works, but df gives the game away — you’ll see overlay filesystems mounted where ext4/erofs used to be. That’s the tell that a device is in the remount state.

The consequences nobody warns you about

It survives reboots. The overlay is re-attached in first-stage init, early enough that even pushed SELinux policy takes effect. One remount -R + remount and the device stays writable-looking until you explicitly undo it.

adb enable-verity is the undo — and the eraser. It re-enables verity and wipes the overlay, reverting every override back to the pristine image. Good: one command back to stock. Surprising: your “installed” changes were never installed anywhere real.

The stale-overlay trap. Flashing a partition through bootloader fastboot doesn’t know about the overlay, so your freshly flashed image can boot with months-old overridden files silently shadowing it. The README is blunt about this: enable-verity to wipe. If you’ve ever muttered “why isn’t my flash taking effect”, check df for overlays before you check anything else. (fastboot flashall and userspace fastbootd do clear scratch.)

OTAs refuse to play. update_engine may decline to run while overlay overrides are active — the two mechanisms both believe they own the difference between “what’s on disk” and “what should be”.

Scratch space is finite. It’s carved from whatever super had left over. Push something huge and it fills; later dynamic-partition resizes can fail because scratch is squatting in the free space. Same fix as everything else: enable-verity and start clean.

Why bother: the ten-second dev loop

All of this machinery exists for one workflow, and it’s the one that makes platform development bearable:

m framework-minus-apex             # rebuild just what you changed
adb root && adb remount            # first time: adb remount -R, wait, repeat
adb sync system                    # push changed files from $ANDROID_PRODUCT_OUT
adb shell stop && adb shell start  # restart the runtime

adb sync compares $ANDROID_PRODUCT_OUT against the device and pushes only what changed; stop/start bounces the Android runtime without rebooting the kernel. Total turnaround after a one-file framework change: seconds. The alternative — rebuild super.img, tear down Cuttlefish, relaunch — is minutes, every time. Multiply by every iteration of every experiment in the rest of the course, and this weird little verity-toggling, partition-conjuring, overlay-mounting orchestrator might be the single highest-leverage tool in the platform developer’s kit.

Reading list

If you have a tree synced, the whole mechanism is a pleasant afternoon of reading:

• system/core/fs_mgr/README.overlayfs.md — the canonical doc; short, and refreshingly honest about the caveats.
• system/core/fs_mgr/fs_mgr_remount.cpp — the four-named binary itself.
• system/core/fs_mgr/fs_mgr_overlayfs_control.cpp and fs_mgr_overlayfs_mount.cpp — scratch creation and overlay mounting.
• device/google/cuttlefish/shared/device.mk — makes the overlayfs path the only path on Cuttlefish.