A Review on Penetration Testing for Privacy of Deep Learning Models
Salma Akther1, Wencheng Yang1,*, Song Wang2, Shicheng Wei1, Ji Zhang1, Xu Yang3, Yanrong Lu4, Yan Li1
1 School of Mathematics, Physics and Computing, University of Southern Queensland, Toowoomba, QLD, Australia
2 Department of Engineering, La Trobe University, Melbourne, VIC, Australia
3 School of Computer and Data Science, Minjiang University, Fuzhou, China
4 School of Security Science and Engineering, Civil Aviation University of China, Tianjin, China
* Corresponding Author: Wencheng Yang. Email:
(This article belongs to the Special Issue: Artificial Intelligence Methods and Techniques to Cybersecurity)
Computers, Materials & Continua https://doi.org/10.32604/cmc.2026.076358
Received 19 November 2025; Accepted 22 January 2026; Published online 13 February 2026
Abstract
As deep learning (DL) models are increasingly deployed in sensitive domains (e.g., healthcare), concerns over privacy and security have intensified. Conventional penetration testing frameworks, such as OWASP and NIST, are effective for traditional networks and applications but lack the capabilities to address DL-specific threats, such as model inversion, membership inference, and adversarial attacks. This review provides a comprehensive analysis of penetration testing for the privacy of DL models, examining the shortfalls of existing frameworks, tools, and testing methodologies. Through systematic evaluation of existing literature and empirical analysis, we identify three major contributions: (i) a critical assessment of traditional penetration testing frameworks’ inadequacies when applied to DL-specific privacy vulnerabilities, (ii) a comprehensive evaluation of state-of-the-art privacy-preserving methods and their integration with penetration testing workflows, and (iii) the development of a structured framework that combines reconnaissance, threat modeling, exploitation, and post-exploitation phases specifically tailored for DL privacy assessment. Moreover, this review evaluates popular solutions such as IBM Adversarial Robustness Toolbox and TensorFlow Privacy, alongside privacy-preserving techniques (e.g., Differential Privacy, Homomorphic Encryption, and Federated Learning), which we systematically analyze through comparative studies of their effectiveness, computational overhead, and practical deployment constraints. While these techniques offer promising safeguards, their adoption is hindered by accuracy loss, performance overheads, and the rapid evolution of attack strategies. Our findings reveal that no single existing solution provides comprehensive protection, which leads us to propose a hybrid approach that strategically combines multiple privacy-preserving mechanisms. The findings of this survey underscore an urgent need for automated, regulation-compliant penetration testing frameworks specifically tailored to DL systems. We argue for hybrid privacy solutions that combine multiple protective mechanisms to ensure both model accuracy and privacy. Building on our analysis, we present actionable recommendations for developing adaptive penetration testing strategies that incorporate automated vulnerability assessment, continuous monitoring, and regulatory compliance verification.
Keywords
Penetration testing; deep learning; homomorphic encryption; differential privacy; federated learning