Exploring Bitslicing Architectures for Enabling FHE-assisted Machine Learning

Homomorphic Encryption (HE) is the ultimate tool for performing secure computations even in untrusted environ-ments. Application of HE for Deep Learning (DL) inference is an active area of research, given the fact that DL models are often deployed in untrusted environments (e.g. third-party servers) yet inferring on private data. However, existing HE libraries (somewhat (SWHE), leveled (LHE) or fully homomorphic (FHE)) suffer from extensive computational and memory overhead. Few performance optimized high-speed homomorphic libraries are either suffering from certain approximation issues leading to decryption errors or proven to be insecure according to recent published attacks. In this paper, we propose architectural tricks to achieve performance speedup for encrypted DL inference developed with exact HE schemes without any approximation or decryption error in homomorphic computations. The main idea is to apply quantization and suitable data packing in the form of bitslicing to reduce the costly noise handling operation, Bootstrapping while achieving a functionally correct and highly parallel DL pipeline with a moderate memory footprint. Experi-mental evaluation on the MNIST dataset shows a significant (37X) speedup over the non-bitsliced versions of the same architecture. Low memory bandwidths (700MB) of our design pipelines further highlight their promise towards scaling over larger gamut of Edge-AI analytics use cases.