White-Box Caching for Cloud Data Systems Xiangpeng Hao Department of Computer Sciences University of Wisconsin-Madison

Abstract:

Modern data systems disaggregate compute, storage, and memory across independent tiers, making data movement more expensive than computation. Yet caches still treat contents as opaque bytes: they accept blobs, index them by key, and evict each blob as a unit, without knowing its structure or which fields a query will access. This dissertation argues that this opacity has become a bottleneck and that caches should instead inspect their data.

We call this approach White-Box caching and develop it along three axes – representation, content, and granularity – each recovering information a conventional cache discards. Three systems instantiate one axis each.

LiquidCache attacks representation. It transparently transcodes Parquet into independently decodable dictionary, FastLanes, FSST, ALP, and frame-of-reference encodings, allowing predicates to use the most efficient layer. On ClickBench and TPC-H, it reduces cache CPU by up to 10× over Parquet pushdown and network traffic by two orders of magnitude over file-serving caches, while using memory comparable to compressed Parquet.

SqueezeCache attacks content. Profiling shows that up to 98% of cached bytes – even under Belady-optimal replacement – are never read by the queries that warmed them. Guided by lineage analysis, SqueezeCache replaces each entry with a compact, lossy subset selected per column across a four-state fidelity spectrum. On ClickBench and JSONBench, it improves hit ratios by up to 4×, lowers latency by up to 22×, and cuts I/O by up to 12× over an uncompressed cache; corresponding gains over a compression-only baseline are 2×, 2×, and 8×.

Bf-Tree attacks granularity for transactional workloads. Variable-length mini-pages decouple cache pages from disk pages, yielding one index that is 2× faster than RocksDB on scans, 6× faster than a B-Tree on writes, and 2× faster than both on point lookups.

LiquidCache and SqueezeCache form a 50,000-line open-source Rust artifact, while Bf-Tree is implemented separately. The work also contributes substantially to Apache DataFusion, Arrow, and Parquet.

Full Paper: PDF   BibTeX

Publications