A Survey of Interpretable and Black-Box Deep Neural Networks
Downloads
This paper provides a more detailed overview of deep neural network (DNN) models with an emphasis on the potential for interpretability. The provided analysis categorizes the existing models based on interpretability (i.e., interpretable, post-hoc interpretable, or fully black-box models). The type of approach chosen for conducting the survey follows a well-structured methodology from several academic databases to comprehensively analyze the selected studies. The paper also analyzes explainability techniques such as SHAP and LIME, and investigates the trade-off in performance vs. complexity vs. interpretability with the existing models. The results of the analysis indicate that even though the black-box models have dominated in terms of peak accuracy, interpretable methods are improving comparably, achieving competitive results. Finally, this survey identifies three primary areas for future research that require attention: increased research related to model efficiency, transparency and scalability; lack of research addressing integration of explainability techniques into existing models; and unified low-complexity interpretability methods/models.
M. Haenlein and A. Kaplan, "A brief history of artificial intelligence: On the past, present, and future of artificial intelligence," California management review, vol. 61, no. 4, pp. 5-14, 2019.
M. I. Jordan and T. M. Mitchell, "Machine learning: Trends, perspectives, and prospects," Science, vol. 349, no. 6245, pp. 255-260, 2015.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
C. Molnar, Interpretable machine learning. Lulu. com, 2020.
I. El Naqa and M. J. Murphy, "What are machine and deep learning? " in Machine and deep learning in oncology, medical physics and radiology: Springer, 2022, pp. 3-15.
X. Li et al., "Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond," Knowledge and Information Systems, vol. 64, no. 12, pp. 3197-3234, 2022.
T. K. Gupta and K. Raza, "Optimizing deep feedforward neural network architecture: A tabu search-based approach," Neural Processing Letters, vol. 51, no. 3, pp. 2855-2870, 2020.
R. Richman and M. V. Wüthrich, "LocalGLMnet: interpretable deep learning for tabular data," Scandinavian Actuarial Journal, vol. 2023, no. 1, pp. 71-95, 2023.
M. Q. Li, B. C. Fung, and A. Abusitta, "On the effectiveness of interpretable feedforward neural network," in 2022 International Joint Conference on Neural Networks (IJCNN), 2022: IEEE, pp. 1-8.
L. Balderas, M. Lastra, and J. M. Benítez, "Optimizing dense feed-forward neural networks," Neural Networks, vol. 171, pp. 229-241, 2024.
Z. Liu et al., "Ffsplit: Split feed-forward network for optimizing accuracy-efficiency trade-off in language model inference," arXiv preprint arXiv:2401.04044, 2024.
A. Charalampopoulos, N. Chatzis, F. Ntoulas-Panagiotopoulos, C. Papaioannou, and A. Potamianos, "Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Node," arXiv preprint arXiv:2405.16836, 2024.
S. Chatterjee and T. Sudijono, "Neural networks generalize on low complexity data," arXiv preprint arXiv:2409.12446, 2024.
B. Deng and L. Heath, "Toward Errorless Training ImageNet-1k," arXiv preprint arXiv:2508.04941, 2025.
Y. Gorishniy, A. Kotelnikov, and A. Babenko, "Tabm: Advancing tabular deep learning with parameter-efficient ensembling," arXiv preprint arXiv:2410.24210, 2024.
J. Yan, J. Chen, Q. Wang, D. Z. Chen, and J. Wu, "Team up gbdts and dnns: Advancing efficient and effective tabular prediction with tree-hybrid mlps," in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 3679-3689.
J. Martorell-Marugán et al., "Explainable deep neural networks for predicting sample phenotypes from single-cell transcriptomics," Briefings in Bioinformatics, vol. 26, no. 1, p. bbae673, 2025.
J. S. Jeya I, "Deep Learning based Feed Forward Neural Network Models for Hyperspectral Image Classification," The Open Biomedical Engineering Journal, vol. 18, no. 1, 2024.
A. L. Yaser, H. M. Mousa, and M. Hussein, "Improved DDoS detection utilizing deep neural networks and feedforward neural networks as autoencoder," Future Internet, vol. 14, no. 8, p. 240, 2022.
P. Singh, S. K. Borgohain, A. K. Sarkar, J. Kumar, and L. D. Sharma, "Feed‐Forward Deep Neural Network (FFDNN)‐Based Deep Features for Static Malware Detection," International Journal of Intelligent Systems, vol. 2023, no. 1, p. 9544481, 2023.
S. Häffner, M. Hofer, M. Nagl, and J. Walterskirchen, "Introducing an interpretable deep learning approach to domain-specific dictionary creation: A use case for conflict prediction," Political Analysis, vol. 31, no. 4, pp. 481-499, 2023.
D. Tran and A. W. Tham, "Accuracy comparison between feedforward neural network, support vector machine and boosting ensembles for financial risk evaluation," Journal of Risk and Financial Management, vol. 18, no. 4, p. 215, 2025.
G. Jeon, H. Jeong, and J. Choi, "Distilled gradient aggregation: Purify features for input attribution in the deep neural network," Advances in Neural Information Processing Systems, vol. 35, pp. 26478-26491, 2022.
A. Eslamian and Q. Cheng, "TabMixer: advancing tabular data analysis with an enhanced MLP-mixer approach," Pattern Analysis and Applications, vol. 28, no. 2, pp. 1-17, 2025.
O. Dağlı, M. O. Kaya, and C. Eyüpoğlu, "Feed-Forward Neural Network-Based Prediction of Flutter Speed of 3 DOF 2D Wing Model," Journal of Vibration Engineering & Technologies, vol. 13, no. 1, p. 47, 2025.
S.-I. Jang, M. J. Girard, and A. H. Thiery, "Explainable and interpretable diabetic retinopathy classification based on neural-symbolic learning," arXiv preprint arXiv:2204.00624, 2022.
C. Geng, X. Xu, A. Xing, Z. Zhao, and X. Si, "From Neural Representations to Interpretable Logic Rules," arXiv preprint arXiv:2501.08281, 2025.
D. S. R. Kawale1, P. K. , and D. S. Holyachi, "Deep Neural Network Approach for Early-Stage Diabetes Risk Prediction using Hybrid SMOTE-ENN and GAN with SHAP-Based Feature Explanations " Journal of Neonatal Surgery vol. Vol. 14, Issue 31s (2025) no. ISSN(Online): 2226-0439 pp. 861-874, 2025.
M. Di Giammarco et al., "Explainable Deep Learning for Breast Cancer Classification and Localization," ACM Transactions on Computing for Healthcare, vol. 6, no. 1, pp. 1-18, 2025.
K. Iftikhar, N. Javaid, I. Ahmed, and N. Alrajeh, "A Novel Explainable Deep Learning Framework for Accurate Diabetes Mellitus Prediction," Applied Sciences, vol. 15, no. 16, p. 9162, 2025.
