Extending Operating Systems in a Changing Stack: Hardware, Kernels, and AI Agents Wanxiang (Shawn) Zhong Department of Computer Sciences University of Wisconsin-Madison

Abstract:

Software uses operating-system interfaces to access hardware resources and shared system state. These interfaces reflect assumptions about the stack: hardware costs below the kernel, kernel implementations within it, and the software that uses them above it. As the stack changes, an interface may leave unnecessary work on a critical path, hide the dependencies needed for portability, or fail to support a new type of client safely. This dissertation shows that these problems can be addressed with new support at each interface while remaining compatible with existing software.

We develop three systems for three such changes: faster hardware below the kernel, variation among kernel builds within it, and AI agents as a new type of client above it. These changes expose filesystem overhead, cause portability problems, and put user files at risk.

We first present MadFS, a library filesystem for persistent memory. Persistent memory lets applications access file data with load and store instructions, but metadata operations still go through the kernel. MadFS introduces per-file virtualization, which embeds selected metadata in each file. A userspace library manages metadata, crash consistency, and concurrency, while the kernel retains protection and storage allocation. MadFS achieves up to 3.6× the throughput of ext4-DAX without kernel changes.

We next present DepSurf, which finds the kernel dependencies that make eBPF programs difficult to run across kernel builds. It extracts the dependency surface of a kernel image and compares it with the dependencies of a program. Across 25 kernel images, we find that kernel source changes, configuration choices, and compiler optimizations all reshape what a kernel provides. The resulting mismatches affect 83% of the 53 real-world programs we analyze.

Finally, we study 290 public reports of AI agents misusing filesystems and find that agents and users lack information about filesystem effects and control over them. We introduce agent-native filesystems and build YoloFS. Staging holds an agent’s changes for review, snapshots preserve earlier states for recovery, and progressive permission refines access rules as the agent works. YoloFS enables agents to undo hidden destructive side effects in 8 of 11 tasks and matches the baseline success rate on 112 routine tasks with fewer user interactions and negligible I/O overhead.

Together, the three systems improve performance, diagnose portability problems, and protect user files while preserving the interfaces and services that existing software depends on.

Full Paper: PDF   BibTeX

Publications