Hybrid 3D CNN and Swin Transformer for Autism Classification Through Structural and Functional Brain MRI

Rafal Razzaq Al-Khalidi; Roaa Razaq Al-Khalidy; Sarah Zuhair Kurdi

doi:10.54133/ajms.v9i2.2559

Authors

Rafal Razzaq Al-Khalidi Department of surgery, College of Medicine, Jabir Ibn Hayyan University for Medical and Pharmaceutical Sciences, Najaf, Iraq https://orcid.org/0009-0009-2576-2339
Roaa Razaq Al-Khalidy Department of Computer Science, Faculty of Education, University of Kufa, Najaf, Iraq https://orcid.org/0009-0003-4442-2993
Sarah Zuhair Kurdi Department of Surgery, College of Medicine, University of Kufa, Najaf, Iraq https://orcid.org/0009-0005-7024-7679

DOI:

https://doi.org/10.54133/ajms.v9i2.2559

Keywords:

Autism, Hybrid 3D CNN, fMRI, sMRI, Swin transformer

Abstract

Background: Autism spectrum disorder is a prevalent neurodevelopmental condition characterized by social communication deficits and behavioral disturbance. Early diagnosis is essential for timely intervention and condition control. Yet current behavioral assessments are subjective and often delayed; neuroimaging provides objective insights into structural and functional brain alterations. Objective: To evaluate whether integrating structural MRI and functional MRI using a deep learning framework can improve the interpretability and diagnostic accuracy of autism spectrum disorders. Methods: In this retrospective diagnostic modeling study, a hybrid model architecture was proposed, combining 3D convolutional neural networks for structural MRI features with Swin Transformers for functional MRI representations. Features were fused through cross-attention and classified with a fully connected layer. The model was trained and validated on the Autism Brain Imaging Data Exchange II dataset (ABIDE II) Georgetown University site and externally tested on the larger ABIDE II dataset NYU-1 site. Performance metrics included accuracy, F1-score, and ROC-AUC. The hybrid model was compared to each model alone. Results: The model achieved 94.6% accuracy on the GU site, which also maintained a high testing performance on NYU_1. Attention-based fusion of structural MRI and functional MRI revealed brain regions connected to autism spectrum disorder, making the images easier to understand. Conclusions: Multimodal fusion of structural MRI and functional MRI demonstrates a clinically valuable AI tool for early autism spectrum disorder detection. This approach may increase diagnostic confidence and timely interventions.

Downloads

Download data is not yet available.

References

World Health Organization. Autism spectrum disorders (ASD). 15 Nov 2023 [Internet]. Available from: https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders

Centers for Disease Control and Prevention. Data & statistics on autism spectrum disorder (ASD). 2022 [Internet]. Available from: https://www.cdc.gov/autism/data-research/index.html

Maenner MJ, Shaw KA, Bakian AV, Bilder DA, Durkin MS, Esler A, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. MMWR Surveill Summ. 202;70(11):1-16. doi: 10.15585/mmwr.ss7011a1. DOI: https://doi.org/10.15585/mmwr.ss7011a1

Kadhim DA, Mohammed MA. Advanced machine learning models for accurate kidney cancer classification using CT images. Mesopotamian J Big Data. 2025;1–25. doi: 10.58496/MJBD/2025/001. DOI: https://doi.org/10.58496/MJBD/2025/001

Ayoub MJ, Keegan L, Tager-Flusberg H, Gill SV. Neuroimaging techniques as descriptive and diagnostic tools for infants at risk for autism spectrum disorder: A systematic review. Brain Sci. 2022;12(5):602. doi: 10.3390/brainsci12050602. DOI: https://doi.org/10.3390/brainsci12050602

Müller RA, Shih P, Keehn B, Deyoe JR, Leyden KM, Shukla DK. Review of neuroimaging in autism spectrum disorders: What have we learned and where we go from here? Mol Autism. 2011;2(1):4. doi: 10.1186/2040-2392-2-4. DOI: https://doi.org/10.1186/2040-2392-2-4

Garcia M, Kelly C. Toward building an interpretable predictive tool for ASD with 3D convolutional neural networks. medRxiv. 2022. doi: 10.1101/2022.10.18.22281196. DOI: https://doi.org/10.1101/2022.10.18.22281196

Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, et al. Swin Transformer: Hierarchical vision transformer using shifted windows. arXiv. 2021. doi: 10.48550/arXiv.2103.14030. DOI: https://doi.org/10.1109/ICCV48922.2021.00986

Müller RA, Fishman I. Brain connectivity and neuroimaging of autism spectrum disorders. Ann N Y Acad Sci. 2018;1431(1):38–56. doi: 10.1111/nyas.13715. DOI: https://doi.org/10.1111/nyas.13715

Li X, Zhang K, He X, Zhou J, Jin C, Shen L, et al. Structural, functional, and molecular imaging of autism spectrum disorder. Neurosci Bull. 2021;37(7):1051-1071. doi: 10.1007/s12264-021-00673-0. DOI: https://doi.org/10.1007/s12264-021-00673-0

Neuroimaging Informatics Tools and Resources Clearinghouse (NITRC). Autism Brain Imaging Data Exchange II (ABIDE-II). 2016 [Internet]. Available from: https://www.nitrc.org/frs/downloadlink.php/9068

ABIDE II – Site-specific datasets (GU, NYU_1). NITRC. [Internet]. 2025 [cited Sep 2025]. Available from: https://fcon_1000.projects.nitrc.org/indi/abide/abide_II.html

Di Martino A, Kelly C, Grzadzinski R, Zuo XN, Mennes M, Mairena MA, et al. Aberrant striatal functional connectivity in children with autism. Biol Psychiatry. 2011;69(9):847–856. doi: 10.1016/j.biopsych.2010.10.029. DOI: https://doi.org/10.1016/j.biopsych.2010.10.029

Zhang X, Li Y, Chen H, Zhou J. Classification and diagnosis of ASD using Swin Transformer. In: Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI). 2023. doi: 10.1109/isbi53787.2023.10230792. DOI: https://doi.org/10.1109/ISBI53787.2023.10230792

Alharthi H, Alzahrani A. Multi-slice generation of sMRI and fMRI with 3D-CNN and vision transformers. Brain Sci. 2023;13(11):1578. doi: 10.3390/brainsci13111578. DOI: https://doi.org/10.3390/brainsci13111578

Jahani N, Patel S, Thomas G. Twinned neuroimaging analysis for ASD. Sci Rep. 2024;14:71174. doi:10.1038/s41598-024-71174-z DOI: https://doi.org/10.1038/s41598-024-71174-z

Krajevski T, Müller H, Smith J. ASD classification from sMRI and fMRI. In: Communications in Computer and Information Science (CCIS). Springer; 2022. p. 212–224. doi: 10.1007/978-3-031-22792-9_14. DOI: https://doi.org/10.1007/978-3-031-22792-9_14

Pavelić M, Kovač D, Prpić T. Deep learning for ASD detection using rs-fMRI. In: Proceedings of the IEEE ELMAR Conference. 2024. doi: 10.1109/elmar62909.2024.10693972. DOI: https://doi.org/10.1109/ELMAR62909.2024.10693972

Gao J, Huang Y, Zhao F. Multi-task transformer for ASD detection. BMC Neurosci. 2024;25:870. doi: 10.1186/s12868-024-00870-3. DOI: https://doi.org/10.1186/s12868-024-00870-3

Wang Y, Liu M, Xu Z. Multi-dimension embedding-aware fusion transformer for ASD detection. arXiv. 2023. doi: 10.48550/arxiv.2310.02690.

Mellema C, Riaz A, Lee S. Reproducible neuroimaging features for ASD with machine learning. medRxiv. 2021. doi: 10.21203/RS.3.RS-1024223/V1. DOI: https://doi.org/10.21203/rs.3.rs-1024223/v1