Expert agreement confirms that negative changes in hand and foot radiographs are a surrogate for repair in patients with rheumatoid arthritis

The objective of the present study was to test the hypothesis that experts recognize repair of erosions and, if so, to determine which, if any, morphologic features permitted them to recognize the repair. We also tested whether scoring by a standard method detected repair. Seven experienced readers of radiographs in rheumatoid arthritis were presented with 64 sets of single joints-of-interest at two time points, randomized and blinded for the correct sequence. The readers assessed which joint was better, and recorded whether any of six specific features were seen. Two independent readers, experienced in scoring by the van der Heijde-modified Sharp method who were not on the expert panel, then scored the complete films that included the joint-of-interest. The panel agreed very well on which of two joints was better, and, even though they did not know the true sequence, the panel accurately assigned a sequence slightly better than chance alone (58%) but worse than their agreement on which image was 'better or worse' (78%). The readers therefore indirectly assigned repair by choosing the second film as the best. Putative repair features were seen in cases of both repair and progression, and were not discriminatory. Similar results were obtained when the experts were presented with the entire hand or foot containing the joint-of-interest. In the third repair exercise, two independent readers who scored whole hands and feet using a standard method found a mean negative score in 22/60 joints-of-interest. All 22 joints were also scored as repair by the panel. Repair was detected reliably by a majority of the panel on viewing paired images based on a better/worse decision and assigning sequence in a set of images that were blinded for sequence by an independent project manager. In this test set of images, repair was manifested by a reduction in the size of erosion in many cases. Size was one feature that aided the experts to detect repair but cannot be the only one; the experts had to find other features to determine whether a smaller erosion was the first in a sequence of radiographs in a patient with progressive damage or was the second film in a patient exhibiting repair. The change in size of erosion was also picked up by independent readers applying the van der Heijde-modified Sharp scoring method and was reflected in their scores.