Homomorphic evaluation of tensor programs

Embodiments of the present invention provide systems, methods, and computer storage media directed to optimizing the generation, evaluation, and selection of tensor circuit specifications for a tensor circuit to perform homomorphic encryption operations on encrypted data. A computing device having an improved compiler and runtime configuration can obtain a tensor circuit and associated schema. The computing device can map the obtained tensor circuit to an equivalent tensor circuit, adapted to perform fully homomorphic encryption (FHE) operations, and instantiated based on the obtained associated scheme. The computing device can then monitor a flow of data through the equivalent FHE-adapted tensor circuit utilizing various tensor circuit specifications determined therefor. A cost of each tensor circuit specification can be determined by the computing device based on the monitored flow of data, so as to identify an optimal set of optimal tensor circuit specifications that can be employed by the obtained tensor circuit, to efficiently perform homomorphic encryption operations on encrypted data.