Fully-automated deep learning prediction of spinal deformity alignment irrespective of image quality obtained via smartphone photographs

Abstract

Purpose: For facilitating communication, it is popular for spine specialists to take photos of radiographs with smartphones. An automatic tool to accurately detect vertebral landmarks and alignment provides easy and highly useful information for surgeons. Original X-rays are not easily accessible for telemedicine and existing deep learningbased automated Cobb angle (CA) predictions are not accurate on suboptimal quality X-rays. The aim of study was to develop an automated CA prediction system irrespective of image quality, with no restrictions on curve patterns to facilitate clinical practice and telemedicine. Methods: A total of 367 consecutive patients attending a scoliosis clinic were recruited prospectively and their coronal X-rays were re-captured using mobile phones. Fivefold cross-validation was conducted (five experiments, each with 294 images to train a deep neural network named Spine_HRNet for endplates landmarks and endvertebrae detection, and the remaining 73 images were used to test). The predicted heatmaps of the vertebral landmarks were visualized to enhance interpretabilityof Spine_HRNet. Per-landmark-absolute-errors and recall of the landmark detection results were calculated to assess the accuracy of the predicted landmarks. Further calculated CAs were quantitatively compared with the spine specialists measured ground truth (GT). Results: The average per-landmark absolute distance error and the recall of the detected endplates landmarks were 2.8 pixels and 0.99, indicating a highly accurate detection. The predicted CAs were all significantly correlated with GT (p<0.01). Compared with GT, the mean error was 3.73°–4.15° and standard error of the measurement was 0.8°–1.7° for the predicted CAs at different spinal regions. Conclusions: This is the first study using Spine_HRNet on non-original X-rays to automatically and accurately predict vertebral landmarks of the scoliotic spine. Spine_HRNet’s applicability is evidenced by our thorough crossvalidation, which can be used with telemedicine to facilitate fast reliable auto-diagnosis and follow-up.