The ADvanced Systems Laboratory (ADSL)
Software
| :: | Home |
| :: | Projects |
| :: | People |
| :: | Pictures |
| :: | Publications |
| :: | Software |
| :: | Sponsors |
| :: | Internal |
File-system Fault Injector for Distributed Systems
CORDS is a file-system fault injection framework tailored for distributed storage systems. CORDS includes the following key pieces: errfs, a user-level FUSE file system that systematically injects file-system faults and errbench, a suite of system-specific workloads which drives systems to interact with their local storage.Source and Documentation
You can find the source and documentation in this github repo. If you use this work in some way, please cite our paper.Bugs and Developer Interaction
Some bugs and interesting behaviors discovered using CORDS:
ZooKeeper cluster unavailable on disk full(ENOSPC), disk quota(EDQUOT), disk write error(EIO) errorsZooKeeper crash on detecting a corruption
Kafka data corruption or EIO leads to data loss
Redis Silent data corruption
LogCabin reaction to disk errors and corruptions
CockroachDB resiliency to disk corruption and storage errors
CockroachDB client hangs after one node hits disk errors
CockroachDB disk corruptions and read/write error handling
RethinkDB integrity of read results
RethinkDB silent data loss on metablock corruptions
Rethink note on handling data corruption
Cassandra silent data corruption and corruption propagation