Maintaining and checking parity in highly available Scalable Distributed Data Structures

Access to data stored in distributed main memory is much faster than access to local disks. Highly available, Scalable Distributed Data Structures (SDDS) utilize this fast access. They counteract the effects of failed or unavailable nodes by storing data redundantly. Since main memory per node is limited, they generate this redundancy by storing parity data calculated with erasure correcting codes instead of using replication. We present here a way to maintain parity that is about 10 times faster than using the traditional 2PC scheme. We also present a scheme that can diagnose a mismatch between parity and user data with very little network traffic.