Do Age and Sex Help Deep Fundus Disease Classifiers? A Controlled Analysis of Metadata Fusion and Attention on ODIR-5K
DOI:
https://doi.org/10.62497/irabcs.278Keywords:
fundus imaging, multi-label classification, metadata fusion, explainable AI, Grad-CAM, ODIR-5KAbstract
Patient age and sex are routinely fused into medical image classifiers, and small accuracy gains are usually reported. Such gains are rarely compared with what the metadata can predict alone, or tested for whether the network actually uses the metadata. We study this on ODIR-5K (3,500 patients, 8 labels) using a patient-level split and three models trained identically with three seeds: metadata only, a dual-eye ResNet-50 on fundus images only, and the same network fused with age and sex embeddings. Age and sex alone reached a macro-AUC of 0.587; the image-only model reached 0.863 and the fused model 0.861 (difference −0.002, 95% CI −0.012 to 0.007), with no label improving after Holm correction. When test metadata were shuffled, macro-AUC fell by only 0.0015, and sweeping a patient’s age from under 30 to over 80 left the predicted cataract probability almost unchanged (0.064 to 0.065) although cataract prevalence rises from 2% to 37% over that range. Stacking age and sex on the image model’s outputs gave no gain either. Quantified Grad-CAM showed that fusion alters attention no more than retraining with a different random seed. For this task, image evidence makes age and sex redundant; we recommend that metadata-fusion studies report a metadata-only control and a metadata-reliance test.
Downloads
References
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410. doi:10.1001/jama.2016.17216
Li N, Li T, Hu C, Wang K, Kang H. A benchmark of ocular disease intelligent recognition: one shot for multi-disease detection. In: Benchmarking, Measuring, and Optimizing (Bench 2020). Lecture Notes in Computer Science. Vol. 12614. Springer; 2021. p. 177–193. doi:10.1007/978-3-030-71058-3_11
Huang SC, Pareek A, Seyyedi S, Banerjee I, Lungren MP. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digit Med. 2020;3:136. doi:10.1038/s41746-020-00341-z
Pacheco AGC, Krohling RA. The impact of patient clinical information on automated skin cancer detection. Comput Biol Med. 2020;116:103545. doi:10.1016/j.compbiomed.2019.103545
Kawahara J, Daneshvar S, Argenziano G, Hamarneh G. Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J Biomed Health Inform. 2019;23(2):538–546. doi:10.1109/JBHI.2018.2824327
Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. In: Advances in Neural Information Processing Systems (NeurIPS); 2018. p. 9505–9515. arXiv:1810.03292. doi:10.48550/arXiv.1810.03292
Ting DSW, Cheung CYL, Lim G, Tan GSW, Quang ND, Gan A, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211–2223. doi:10.1001/jama.2017.18152
Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV, Corrado GS, et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat Biomed Eng. 2018;2(3):158–164. doi:10.1038/s41551-018-0195-0
Korot E, Pontikos N, Liu X, Wagner SK, Faes L, Huemer J, et al. Predicting sex from retinal fundus photographs using automated deep learning. Sci Rep. 2021;11(1):10286. doi:10.1038/s41598-021-89743-x
Wagner SK, Fu DJ, Faes L, Liu X, Huemer J, Khalid H, et al. Insights into systemic disease through retinal imaging-based oculomics. Transl Vis Sci Technol. 2020;9(2):6. doi:10.1167/tvst.9.2.6
Arun N, Gaw N, Singh P, Chang K, Aggarwal M, Chen B, et al. Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging. Radiol Artif Intell. 2021;3(6):e200267. doi:10.1148/ryai.2021200267
Geirhos R, Jacobsen JH, Michaelis C, Zemel R, Brendel W, Bethge M, et al. Shortcut learning in deep neural networks. Nat Mach Intell. 2020;2:665–673. doi:10.1038/s42256-020-00257-z
He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 770–778. doi:10.1109/CVPR.2016.90
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2009. p. 248–255. doi:10.1109/CVPR.2009.5206848
Loshchilov I, Hutter F. Decoupled weight decay regularization. In: Proceedings of the International Conference on Learning Representations (ICLR); 2019. arXiv:1711.05101. doi:10.48550/arXiv.1711.05101
Efron B, Tibshirani RJ. An introduction to the bootstrap. New York: Chapman & Hall; 1993. doi:10.1201/9780429246593
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845. doi:10.2307/2531595
Sun X, Xu W. Fast implementation of DeLong’s algorithm for comparing the areas under correlated receiver operating characteristic curves. IEEE Signal Process Lett. 2014;21(11):1389–1393. doi:10.1109/LSP.2014.2337313
Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6(2):65–70. doi:10.2307/4615733
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV); 2017. p. 618–626. doi:10.1109/ICCV.2017.74
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2025 Safeer Gohar, Touqir Gohar, Sardar Ahamd Ali (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors retain full copyright and all publishing rights without restrictions. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License.
