Evaluation of Artificial Intelligence Effectiveness in Detection of Lumbosacral Spine Degenerative Diseases

Objective: comparative evaluation of output data of a set of trained convolutional neural network (CNN) models and interpretation of pathological changes in lumbar spine by radiologists during magnetic resonance imaging.
Material and methods. More than 12,000 anonymized archives were collected to generate training and test neural network datasets from patients aged over 18 years. Each archive consisted of a set of programs in two planes containing T2-TSE, T1-TSE and T2 sequences with fat suppression program. Subsequently, the selected studies were tagged in two steps, directly consisting of manual tagging and its validation by experts. CNN training was performed separately for normal analysis, qualitative detection of individual pathological changes, and quantitative analysis. The accuracy of the models was verified by comparing the protocols of five radiologists and the output of CNN models in two steps. The first, intermediate stage evaluated the accuracy of the neural networks in detecting disc bulges, protrusions and extrusions, spinal canal stenosis, lateral stenosis, foraminal stenosis, spondylolisthesis and facet joint arthrosis. In the final stage, in addition to the pathologies considered in the intermediate one, the accuracy of detecting degenerative changes of the occlusive plates, synovitis of intervertebral joints, intervertebral discs degeneration, osteophytes, transitional vertebrae, hypertrophy of yellow ligaments and Schmorl’s hernia was tested. The reference value for all pathological changes considered in this paper was determined by majority vote and, in case of disagreement, by an external radiologist. The radiologists’ interpretations were then compared with those of the trained model.
Results. The artificial intelligence (AI) showed comparable sensitivity and specificity values compared to the reference result in a group of experienced radiologists for binary classification (presence/absence) of individual lumbosacral spine degenerative changes. The sensitivity and specificity of AI results were 0.88 and 0.97 for extrusions, 0.81 and 0.94 for protrusions, 0.87 and 0.98 for central stenosis, 0.83 and 0.85 for lateral stenosis, 0.92 and 0.84 for foraminal stenosis, 0.85 and 0.5 for osteoarthritis, 0.73 and 0.96 for occlusive plates degeneration, 0.85 and 0.84 for intervertebral joint synovitis, 0.91 and 0.88 for osteophytes, 0.93 and 0.72 for intervertebral disc degeneration, 1.0 and 1.0 for transitional vertebrae, 0.8 and 1.0 for spondylolisthesis, 0.67 and 0.99 for yellow ligament hypertrophy, and 0.75 and 1.0 for Schmorl’s hernia, respectively. The accuracy of quantitative size characterization of lumbosacral spine protrusions and extrusions showed unsatisfactory results, but improvements in the quality of determination of these parameters are planned in future work.
Conclusion. AI models showed comparable performance to expert radiologists in detecting lumbosacral spine degenerative changes. Consistent improvement of CNN models based on comparative evaluation with radiologists improves the sensitivity and specificity of pathologic change detection.

1. Li H, Luo H, Huan W, et al. Automatic lumbar spinal MRI image segmentation with a multi-scale attention network. Neural Comput Appl. 2021; 33(18): 11589–602. https://doi.org/10.1007/s00521-021-05856-4.

2. Forsberg D, Sjöblom E, Sunshine JL. Detection and labeling of vertebrae in mr images using deep learning with clinical annotations as training data. J Digit Imaging. 2017; 30(4): 406–12. https://doi.org/10.1007/s10278-017-9945-x.

3. Zhou Y, Liu Y, Chen Q, et al. Automatic lumbar MRI detection and identification based on deep learning. J Digit Imaging. 2019; 32(3): 513–20. https://doi.org/10.1007/s10278-018-0130-7.

4. Natalia F, Young JC, Afriliana N, et al. Automated selection of mid-height intervertebral disc slice in traverse lumbar spine MRI using a combination of deep learning feature and machine learning classifier. PLoS One. 2022; 17(1): e0261659. https://doi.org/10.1371/journal.pone.0261659.

5. Wang X, Zhai S, Niu Y. Automatic vertebrae localization and identification by combining deep SSAE contextual features and structured regression forest. J Digit Imaging. 2019; 32(2): 336–48. https://doi.org/10.1007/s10278-018-0140-5.

6. Lewandrowski KU, Muraleedharan N, Eddy SA, et al. Feasibility of deep learning algorithms for reporting in routine spine magnetic resonance imaging. Int J Spine Surg. 2020; 14(s3): S86–97. https://doi.org/10.14444/7131.

7. Hallinan JTPD, Zhu L, Yang K, et al. Deep learning model for automated detection and classification of central canal, lateral recess, and neural foraminal stenosis at lumbar spine MRI. Radiology. 2021; 300(1): 130–8. https://doi.org/10.1148/radiol.2021204289.

8. Jamaludin A, Lootus M, Kadir T, et al. ISSLS Prize in Bioengineering Science 2017: Automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J. 2017; 26(5): 1374–83. https://doi.org/10.1007/s00586-017-4956-3.

9. Lehnen NC, Haase R, Faber J, et al. Detection of degenerative changes on mr images of the lumbar spine with a convolutional neural network: a feasibility study. Diagnostics. 2021; 11(5): 902. https://doi.org/10.3390/diagnostics11050902.

10. Tsai JY, Hung IYJ, Guo YL, et al. Lumbar disc herniation automatic detection in magnetic resonance imaging based on deep learning. Front Bioeng Biotechnol. 2021; 9: 708137. https://doi.org/10.3389/fbioe.2021.708137.

11. Han Z, Wei B, Leung S, et al. Automated pathogenesis-based diagnosis of lumbar neural foraminal stenosis via deep multiscale multitask learning. Neuroinformatics. 2018; 16(3-4): 325–37. https://doi.org/10.1007/s12021-018-9365-1.

12. Galbusera F, Casaroli G, Bassani T. Artificial intelligence and machine learning in spine research. JOR Spine. 2019; 2(1): e1044. https://doi.org/10.1002/jsp2.1044.

13. Azimi P, Yazdanian T, Benzel EC, et al. A review on the use of artificial intelligence in spinal diseases. Asian Spine J. 2020; 14(4): 543–71. https://doi.org/10.31616/asj.2020.0147.