Artificial intelligence shows potential for accurate assessment of nailfold changes in systemic sclerosis patients

New research presented this week at ACR Convergence 2022, the American College of Rheumatology's annual meeting, showed that the Vision Transformer, an open-source computer vision algorithm, could detect changes in nailfold capillaroscopy images from systemic sclerosis patients, as reliably as human rheumatologists.

Systemic sclerosis (SSc, scleroderma) is a rare connective tissue disease marked by tightening and hardening of the skin and damage to internal organs. A key feature is microvascular abnormalities, which can be visualized using nailfold capillaroscopy (NFC), a non-invasive imaging technique. Despite NFC's important role in diagnosing and monitoring the progression of systemic sclerosis, it comes with certain challenges, including assessor training and potential bias. This has led to an interest in automating and standardizing NFC image assessment using computer vision algorithms, such as the Vision Transformer (ViT). The goal of this study was to evaluate the reliability of a ViT model in reading NFC images and to compare its performance to that of rheumatologists.

The researchers used NFC images from nearly 300 adults enrolled in one of two systemic sclerosis registries, the European Scleroderma Trials and Research Group (EUSTAR) and Very Early Diagnosis of Systemic Sclerosis (VEDOSS). Included patients had physician visits between 2012 and 2021 and met the 2013 ACR/EULAR classification criteria for established systemic sclerosis, or the preliminary criteria for VEDOSS. All available images of both hands were included, regardless of artifacts.

During the learning phase, a ViT model was trained to identify enlarged capillaries, giant capillaries, capillary loss, microhemorrhages, and abnormal morphology.

"The information learned by ViT model during this phase is eventually used to assess NFC changes on new NFC images," explains Alexandru Garaiman, a Ph.D. candidate at the University of Zurich and a rheumatology resident at University Hospital, Zurich. Garaiman and Farhad Nooralahzadeh, a postdoctoral researcher at the University of Zurich, were co-investigators of the study.

The algorithm's performance was evaluated using cross-fold validation (k = five folds), a resampling method to train and test a machine learning model. The researchers used a randomly sampled reliability set to compare ViT model's performance to that of four rheumatologists, three of them trained annotators.

Analyzing more than 17,000 NFC images, ViT model performed well in identifying NFC changes in all five folds. The AUC ranged from 81.8% to 84.5% for identifying different microangiopathic changes and 83.2% for detecting the presence of scleroderma pattern (extremely low capillary density, or giant capillaries present). In the reliability set, the algorithm performed best at diagnosing giant capillaries (AUC of 92.6%) and enlarged capillaries (AUC of 90.2%).

The performance of the human assessors was variable, and one out of four rheumatologists showed equal or lower classification measures compared to ViT.

"We have always seen our ViT model as a possible instrument to assist rheumatologists in screening for SSc-related microangiopathy. We never aimed to outperform human assessors or to replace them, but rather ease their reporting of NFC images," Garaiman says.

"Our study gave strong signals that the model was a reliable instrument to assess NFC images and could easily serve as an assisting device for detecting NFC changes. We see it coming into play when the rheumatologist has to assess a plethora of NFC images. This should red flag the images with abnormalities so the rheumatologist only has to assess a few images, not all of them. And at this, we think our model is doing a superb job."

Before the model is ready for clinical use, Garaiman says, he would like to see at least three more milestones met first.

"We would like to test the model's performance in patients with Raynaud's phenomenon of unknown etiology to gain insights about its diagnostic accuracy in an otherwise unselected cohort. We would also like to address the model's performance in longitudinal studies, especially the ability to detect capillary loss over time."

"And we would like to expose our ViT model to NFC images from other systemic sclerosis centers or recorded on different devices to see how it deals with artifact. Hopefully, the results of these studies will confirm our current findings. But for the time being, we think the diagnosis of scleroderma pattern needs the final judgment of a rheumatologist with experience in capillaroscopy."