Abstract: The increasing adoption of smart devices and cloud services, coupled with limitations in local computing and storage resources, prompts numerous users to transmit private data to cloud servers for processing. However, the transmission of sensitive data in plaintext form raises concerns regarding users’ privacy and security. To address these concerns, this study proposes an efficient privacy-preserving secure neural network inference scheme based on homomorphic encryption and secure multi-party computation, which ensures the privacy of both the user and the cloud server while enabling fast and accurate ciphertext inference. First, we divide the inference process into three stages, including the merging stage for adjusting the network structure, the preprocessing stage for performing homomorphic computations, and the online stage for floating-point operations on the secret sharing of private data. Second, we propose an approach of merging network parameters, thereby reducing the cost of multiplication levels and decreasing both ciphertext–plaintext multiplication and addition operations. Finally, we propose a fast convolution algorithm to enhance computational efficiency. Compared with other state-of-the-art methods, our scheme reduces the linear operation time in the online stage by at least 11%, significantly reducing inference time and communication overhead.

[1]Chabanne H, de Wargny A, Milgram J, et al., 2017. Privacy-preserving classification on deep neural network. https://eprint.iacr.org/2017/035

[2]Chai XL, Wang YJ, Gan ZH, et al., 2022. Preserving privacy while revealing thumbnail for content-based encrypted image retrieval in the cloud. Inform Sci, 604:115-141.

[3]Chaudhari H, Rachuri R, Suresh A, 2020. Trident: efficient 4PC framework for privacy preserving machine learning. Network and Distributed System Security Symp.

[4]Cheon JH, Kim A, Kim M, et al., 2017. Homomorphic encryption for arithmetic of approximate numbers. Proc 23^rd Int Conf on Theory and Applications of Cryptology and Information Security, p.409-437.

[5]Cheon JH, Han K, Kim A, et al., 2019. A full RNS variant of approximate homomorphic encryption. Proc 25^th Int Conf on Selected Areas in Cryptography, p.347-368.

[6]Chou E, Beal J, Levy D, et al., 2018. Faster CryptoNets: leveraging sparsity for real-world encrypted inference. https://arxiv.org/abs/1811.09953

[7]Dowlin N, Gilad-Bachrach R, Laine K, et al., 2016. CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. Proc 33^rd Int Conf on Machine Learning, p.201-210.

[8]Halevi S, Shoup V, 2014. Algorithms in HElib. Proc 34^th Annual Cryptology Conf, p.554-571.

[9]Hesamifard E, Takabi H, Ghasemi M, 2017. CryptoDL: deep neural networks over encrypted data. https://arxiv.org/abs/1711.05189

[10]Iha NK, Ghodsi Z, Garg S, et al., 2021. DeepReDuce: ReLU reduction for fast private inference. Proc 38^th Int Conf on Machine Learning, p.4839-4849.

[11]Ioffe S, Szegedy C, 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift. Proc 32^nd Int Conf on Machine Learning, p.448-456.

[12]Ishiyama T, Suzuki T, Yamana H, 2020. Highly accurate CNN inference using approximate activation functions over homomorphic encryption. Proc IEEE Int Conf on Big Data, p.3989-3995.

[13]Juvekar C, Vaikuntanathan V, Chandrakasan AP, 2018. GAZELLE: a low latency framework for secure neural network inference. Proc 27^th USENIX Security Symp, p.1651-1669.

[14]Kim D, Guyot C, 2023. Optimized privacy-preserving CNN inference with fully homomorphic encryption. IEEE Trans Inform Forensic Secur, 18:2175-2187.

[15]Koti N, Pancholi M, Patra A, et al., 2021. SWIFT: super-fast and robust privacy-preserving machine learning. Proc 30^th USENIX Security Symp, p.2651-2668.

[16]Lai ZZ, Zhou YF, Zheng PJ, et al., 2024. Efficient privacy-preserving KAN inference using homomorphic encryption. https://arxiv.org/abs/2409.07751

[17]Li JS, Liu IH, Tsai CJ, et al., 2020. Secure content-based image retrieval in the cloud with key confidentiality. IEEE Access, 8:114940-114952.

[18]Li QF, Huang ZC, Lu WJ, et al., 2020. HomoPAI: a secure collaborative machine learning platform based on homomorphic encryption. Proc IEEE 36^th Int Conf on Data Engineering, p.1713-1717.

[19]Li Y, Yan HY, Huang T, et al., 2024. Model architecture level privacy leakage in neural networks. Sci China Inform Sci, 67(3):132101.

[20]Liu J, Juuti M, Lu Y, et al., 2017. Oblivious neural network predictions via MiniONN transformations. Proc ACM SIGSAC Conf on Computer and Communications Security, p.619-631.

[21]Lou Q, Shen YL, Jin HX, et al., 2021. SAFENet: a secure, accurate and fast neural network inference. Proc 9^th Int Conf on Learning Representations.

[22]Ma JPK, Tai RKH, Zhao YJ, et al., 2021. Let’s stride blindfolded in a forest: sublinear multi-client decision trees evaluation. Network and Distributed System Security Symp.

[23]Mishra P, Lehmkuhl R, Srinivasan A, et al., 2020. Delphi: a cryptographic inference system for neural networks. Proc Workshop on Privacy-Preserving Machine Learning in Practice, p.27-30.

[24]Ng LKL, Chow SSM, 2021. GForce: GPU-friendly oblivious and rapid neural network inference. Proc 30^th USENIX Security Symp, p.2147-2164.

[25]Riazi MS, Weinert C, Tkachenko O, et al., 2018. Chameleon: a hybrid secure computation framework for machine learning applications. Proc Asia Conf on Computer and Communications Security, p.707-721.

[26]Schroff F, Kalenichenko D, Philbin J, 2015. FaceNet: a unified embedding for face recognition and clustering. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.815-823.

[27]Shen M, Cheng GH, Zhu LH, et al., 2020. Content-based multi-source encrypted image retrieval in clouds with privacy preservation. Fut Gener Comput Syst, 109:621-632.

[28]Wang J, He DB, Castiglione A, et al., 2023. PCNNCEC: efficient and privacy-preserving convolutional neural network inference based on cloud-edge-client collaboration. IEEE Trans Netw Sci Eng, 10(5):2906-2923.

[29]Wang Y, Chen LQ, Wu G, et al., 2023. Efficient and secure content-based image retrieval with deep neural networks in the mobile cloud computing. Comput Secur, 128:103163.

[30]Xie TY, Yamana H, Mori T, 2022. CHE: channel-wise homomorphic encryption for ciphertext inference in convolutional neural network. IEEE Access, 10:107446-107458.

[31]Yagyu K, Takeuchi R, Nishigaki M, et al., 2023. Improving classification accuracy by optimizing activation function for convolutional neural network on homomorphic encryption. Proc 17^th Int Conf on Broadband Wireless Computing, Communication and Applications, p.102-113.