Visualizing Coatnet Predictions for Aiding Melanoma Detection
Downloads
Melanoma is considered to be the most aggressive form of skin cancer. At present, the evaluation of malignancy is performed primarily by invasive histological examination of the suspicious lesion. Developing an accurate classifier for early and efficient detection can minimize and monitor the harmful effects of skin cancer and increase patient survival rates. Due to the similar shape of malignant and benign cancerous lesions, doctors spend considerably more time when diagnosing these findings. However, using a deep learning approach as a computer vision tool can overcome some of the challenges. This paper proposes a multi-class classification task using the CoAtNet architecture, a hybrid model that combines the deep depthwise convolution matrix operation of traditional convolutional neural networks with the strengths of Transformer models and self-attention mechanics to achieve better generalization and capacity. The model was evaluated based on precision, recall, and AP. The proposed multi-class classifier achieves an overall precision of 0.901, recall 0.895, and AP 0.923, indicating high performance compared to other state-of-the-art networks. The proposed approach should provide a less complex framework to automate the melanoma diagnostic process and speed up the life-saving process.
Albawi, S., Mohammed, T. A. & Al-Zawi, S. (2017), Understanding of a convolutional neural network, in ‘2017 international conference on engineering and technology (ICET)’, Ieee, pp. 1–6.
Amann, J., Blasimme, A., Vayena, E., Frey, D. & Madai, V. I. (2020), ‘Explainability for artificial intelligence in healthcare: a multidisciplinary perspective’, BMC Medical Informatics and Decision Making 20(1), 1–9.
Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M. & Khan, M. K. (2018), ‘Medical image analysis using convolutional neural networks: a review’, Journal of medical systems 42(11), 1–13.
Apalla, Z., Nashan, D., Weller, R. B. & Castellsagué, X. (2017), ‘Skin cancer: epidemiology, disease burden, pathophysiology, diagnosis, and therapeutic approaches’, Dermatology and therapy 7(1), 5–19.
Bastian, B. C. (2014), ‘The molecular pathology of melanoma: an integrated taxonomy of melanocytic neoplasia’, Annual Review of Pathology: Mechanisms of Disease 9, 239–271.
Boyd, K., Eng, K. H. & Page, C. D. (2013), Area under the precision-recall curve: point estimates and confidence intervals, in ‘Joint European conference on machine learning and knowledge discovery in databases’, Springer, pp. 451– 466.
Buckland, M. & Gey, F. (1994), ‘The relationship between recall and precision’, Journal of the American society for information science 45(1), 12–19.
Carli, P., Massi, D., de Giorgi, V. & Giannotti, B. (2002), ‘Clinically and dermoscopically featureless melanoma: when prevention fails’, Journal of the American Academy of Dermatology 46(6), 957–959.
Chang, J. & Sha, J. (2016), ‘An efficient implementation of 2d convolution in cnn’, IEICE Electronics Express pp. 13–20161134.
Coit, D. G., Andtbacka, R., Bichakjian, C. K., Dilawari, R. A., DiMaio, D., Guild, V., Halpern, A. C., Hodi, F. S., Kashani-Sabet, M., Lange, J. R. et al. (2009), ‘Melanoma’, Journal of the National Comprehensive Cancer Network 7(3), 250–275.
Dai, Z., Liu, H., Le, Q. V. & Tan, M. (2021), ‘Coatnet: Marrying convolution and attention for all data sizes’, Advances in Neural Information Processing Systems 34, 3965–3977.
Elgendi, M., Nasir, M. U., Tang, Q., Smith, D., Grenier, J.-P., Batte, C., Spieler, B., Leslie, W. D., Menon, C., Fletcher, R. R. et al. (2021), ‘The effectiveness of image augmentation in deep learning networks for detecting covid-19: A geometric transformation perspective’, Frontiers in Medicine 8.
England, J. R. & Cheng, P. M. (2019), ‘Artificial intelligence for medical image analysis: a guide for authors and reviewers’, American journal of roentgenology 212(3), 513–519.
FDA (2021), ‘Good machine learning practice for medical device development: Guiding principles’.
Garg, R., Maheshwari, S. & Shukla, A. (2021), Decision support system for detection and classification of skin cancer using cnn, in ‘Innovations in Computational Intelligence and Computer Vision’, Springer, pp. 578–586.
Ghassemi, M., Oakden-Rayner, L. & Beam, A. L. (2021), ‘The false hope of current approaches to explainable artificial intelligence in health care’, The Lancet Digital Health 3(11), e745–e750.
Guo, Y., Li, Y., Wang, L. & Rosing, T. (2019), Depthwise convolution is all you need for learning multiple visual domains, in ‘Proceedings of the AAAI Conference on Artificial Intelligence’, Vol. 33, pp. 8368–8375.
Hamet, P. & Tremblay, J. (2017), ‘Artificial intelligence in medicine’, Metabolism 69, S36–S40.
He, T., Guo, J., Chen, N., Xu, X., Wang, Z., Fu, K., Liu, L. & Yi, Z. (2019), ‘Medimlp: using grad-cam to extract crucial variables for lung cancer postoperative complication prediction’, IEEE Journal of Biomedical and Health Informatics 24(6), 1762–1771.
Hida, T., Kamiya, T., Kawakami, A., Ogino, J., Sohma, H., Uhara, H. & Jimbow, K. (2020), ‘Elucidation of melanogenesis cascade for identifying pathophysiology and therapeutic approach of pigmentary disorders and melanoma’, International Journal of Molecular Sciences 21(17), 6129.
Holzinger, A., Langs, G., Denk, H., Zatloukal, K. & Müller, H. (2019), ‘Causability and explainability of artificial intelligence in medicine’, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 9(4), e1312.
Japkowicz, N. & Stephen, S. (2002), ‘The class imbalance problem: A systematic study’, Intelligent data analysis 6(5), 429–449.
Jensen, J. D. & Elewski, B. E. (2015), ‘The abcdef rule: combining the “abcde rule” and the “ugly duckling sign” in an effort to improve patient self-screening examinations’, The Journal of clinical and aesthetic dermatology 8(2), 15.
Kassani, S. H. & Kassani, P. H. (2019), ‘A comparative study of deep learning architectures on melanoma detection’, Tissue and Cell 58, 76–83.
Kvak, D., Bendik, M. & Chromcova, A. (2022), ‘Towards clinical practice: Design and implementation of convolutional neural network-based assistive diagnosis system for covid-19 case detection from chest x-ray images’, arXiv preprint arXiv:2203.10596
Li, Q., Cai, W., Wang, X., Zhou, Y., Feng, D. D. & Chen, M. (2014), Medical image classification with convolutional neural network, in ‘2014 13th international conference on control automation robotics & vision (ICARCV)’, IEEE, pp. 844–848.
Liu, W., Hill, D., Gibbs, A. F., Tempany, M., Howe, C., Borland, R., Morand, M. & Kelly, J. W. (2005), ‘What features do patients notice that help to distinguish between benign pigmented lesions and melanomas?: the abcd (e) rule versus the seven-point checklist’, Melanoma research 15(6), 549–554.
Matthews, N. H., Li, W.-Q., Qureshi, A. A., Weinstock, M. A. & Cho, E. (2017), ‘Epidemiology of melanoma’, Exon Publications pp. 3–22.
Mikołajczyk, A. & Grochowski, M. (2018), Data augmentation for improving deep learning in image classification problem, in ‘2018 international interdisciplinary PhD workshop (IIPhDW)’, IEEE, pp. 117–122.
Nachbar, F., Stolz, W., Merkle, T., Cognetta, A. B., Vogt, T., Landthaler, M., Bilek, P., Braun-Falco, O. & Plewig, G. (1994), ‘The abcd rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions’, Journal of the American Academy of Dermatology 30(4), 551– 559.
Rigel, D. S., Friedman, R. J. & Kopf, A. W. (1996), ‘The incidence of malignant melanoma in the united states: issues as we approach the 21st century’, Journal of the American Academy of Dermatology 34(5), 839–847.
Samek, W. & Müller, K.-R. (2019), Towards explainable artificial intelligence, in ‘Explainable AI: interpreting, explaining and visualizing deep learning’, Springer, pp. 5–22.
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. & Batra, D. (2017), Grad-cam: Visual explanations from deep networks via gradientbased localization, in ‘Proceedings of the IEEE international conference on computer vision’, pp. 618–626.
Shen, W., Zhou, M., Yang, F., Yang, C. & Tian, J. (2015), Multi-scale convolutional neural networks for lung nodule classification, in ‘International conference on information processing in medical imaging’, Springer, pp. 588– 599.
Shorten, C. & Khoshgoftaar, T. M. (2019), ‘A survey on image data augmentation for deep learning’, Journal of big data 6(1), 1–48.
Sokolova, M., Japkowicz, N. & Szpakowicz, S. (2006), Beyond accuracy, f-score and roc: a family of discriminant measures for performance evaluation, in ‘Australasian joint conference on artificial intelligence’, Springer, pp. 1015– 1021.
Tan, M. & Le, Q. V. (2019), ‘Mixconv: Mixed depthwise convolutional kernels’, arXiv preprint arXiv:1907.09595 .
Thoma, M. (2017), ‘Analysis and optimization of convolutional neural network architectures’, arXiv preprint arXiv:1707.09725 .
Wang, F., Kaushal, R. & Khullar, D. (2020), ‘Should health care demand interpretable artificial intelligence or accept “black box” medicine?’.
Yilmaz, E. & Aslam, J. A. (2006), Estimating average precision with incomplete and imperfect judgments, in ‘Proceedings of the 15th ACM international conference on Information and knowledge management’, pp. 102–111.
Yu, L., Chen, H., Dou, Q., Qin, J. & Heng, P. A. (2016), ‘Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos’, IEEE journal of biomedical and health informatics 21(1), 65–75.
