Data-Driven Definitions for Active and Structural MRI Lesions in the Sacroiliac Joint in Spondyloarthritis and their Predictive Utility

Objectives . To determine quantitative sacroiliac joint (SIJ) MRI lesion cut-offs that optimally define a positive MRI for inflammatory and structural lesions typical of axial spondyloarthritis (axSpA) and that predict clinical diagnosis. Methods . The ASAS MRI group assessed MRIs from the ASAS Classification Cohort in two reading exercises: A. 169 cases and 7 central readers; B. 107 cases and 8 central readers. We calculated sensitivity/specificity for the number of SIJ quadrants or slices with bone marrow edema (BME), erosion, fat lesion, where a majority of central readers had high confidence there was a definite active or structural lesion. Cut-offs with ≥95% specificity were analyzed for their predictive utility for follow-up rheumatologist diagnosis of axSpA by calculating positive/negative predictive values (PPV/NPV) and selecting cut-offs with PPV ≥95%. Results. Active or structural lesions typical of axSpA on MRI had PPV≥95% for clinical diagnosis of axSpA. Cut-offs that best reflect definite active lesion typical of axSpA were either ≥4 SIJ quadrants with BME at any location or at the same location in ≥3 consecutive slices. For definite structural lesion, the optimal cut-offs were any one of ≥3 SIJ quadrants with erosion or ≥5 with fat lesion, erosion at the same location for ≥2 consecutive slices, fat lesion at the same location for ≥3 consecutive slices, or presence of a ‘deep’ (>1cm) fat lesion. Conclusion. We propose cut-offs for definite active and structural lesions typical of axSpA that have high PPV for a long-term clinical diagnosis of axSpA for application in disease classification and clinical research.


INTRODUCTION
Magnetic resonance imaging (MRI) of the sacroiliac joints (SIJ) represents a major advance in the evaluation of axial spondyloarthritis (axSpA) because it permits detection of inflammatory and structural changes in the subchondral bone marrow that are not visible on conventional radiography of the SIJ [1]. These include bone marrow edema (BME), inflammation in an erosion cavity, and fat lesion [2]. In addition, it is far more sensitive than plain radiography for detection of erosions [3][4][5][6]. Consequently, MRI features of sacroiliitis have been incorporated into the ASAS classification criteria where patients diagnosed with axSpA can be classified as having axSpA according to an imaging arm. This requires the presence of either MRI or radiographic criteria for sacroiliitis and at least one clinical feature of axSpA [7].
A positive MRI indicative of sacroiliitis was first defined in 2009 according to a consensus opinion of ASAS experts in MRI as the presence of subchondral bone marrow edema (BME) that is "highly suggestive" of axSpA on STIR/fat-suppressed T2 (T2FS) or osteitis on contrastenhanced fat-suppressed T1 (T1FS+Gd) MRI [8]. This ASAS definition included a quantitative component, this being the presence of two BME lesions on a single semicoronal slice through the SIJ or a single lesion on two consecutive semicoronal slices. In a subsequent 2016 reappraisal of this ASAS definition by consensus opinion of ASAS experts it was agreed that subchondral BME "highly suggestive" of axSpA was the defining characteristic although a quantitative component was not stipulated [9]. However, what constitutes a lesion that is "highly suggestive of axSpA" may vary considerably even among expert readers. Some reports have applied the Spondyloarthritis Research Consortium of Canada (SPARCC) MRI SIJ inflammation scoring method, which records the presence/absence of BME in SIJ quadrants on consecutive semicoronal slices [10]. A SPARCC cut-off score of ≥2 SIJ quadrants with BME was reported to best reflect central reader consensus as to the presence of BME meeting the ASAS definition [11]. Several cross-sectional studies have since reported that false positive BME lesions meeting the ASAS definition may be observed in 20-40% of healthy individuals as well as those with non-specific back disorders [6,[12][13][14][15]. This is a potential concern because definitions intended for classification purposes may be misused for diagnosis.
Recent advances in our understanding of MRI lesions in the SIJ prompted the ASAS-MRI Working Group (WG) to conduct a reappraisal and validation of new and revised ASAS MRI lesion definitions in a multi-reader exercise of MRI scans from patients with undiagnosed back pain who were recruited to the ASAS classification cohort (ASAS-CC) [7]. Data from this cohort led to the development of the ASAS classification criteria and follow up of the ASAS-CC after 4.4 years indicated high positive predictive value of the criteria for rheumatologist diagnosis of axSpA [16]. We employed a novel study design that first determined lesion cut-offs that optimally define an active or structural MRI lesion considered typical of axSpA as determined by Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 the majority of central readers from the ASAS-WG. We then assessed the predictive utility of these lesion cut-offs for rheumatologist's diagnosis of axSpA at the follow up assessment of the ASAS-CC.

METHODS
The study cohort, local rheumatologist assessments, imaging assessments, and follow up of the ASAS-CC have been reported previously [2,7,16]. The ASAS-CC is an inception cohort of patients with undiagnosed back pain of ≥3 months duration and with onset <45 years referred to a rheumatologist with suspicion of axSpA. Central reader MRI evaluations were conducted after diagnosis had been ascertained by local rheumatologists both at baseline and after 4.4 years of follow up of this cohort.
ASAS eCRF for evaluation of MRI lesions in the SIJ. The online-available eCRF comprised two sections: A. Data was first entered into a global scoring page where central readers recorded the presence/absence of each type of active and structural lesion in iliac and sacral portions of each SIJ and whether the scan met the ASAS definition for a positive MRI (8,9). Central readers also provided a yes/no response and degree of confidence rating (-4 (definitely absent) to +4 (definitely present) to two questions from the original ASAS-CC eCRF (7)  Statistics. In order to generate optimal cut-offs that reflect active or structural lesions typical of axSpA, we first calculated sensitivity and specificity for the number of SIJ quadrants with BME, erosion, fat lesion, and sclerosis where central readers considered there was a definite active or structural lesion present on the MRI scan. A definite lesion was deemed present where a majority of central readers agreed as to the presence of an active or structural lesion typical of axSpA with high confidence (≥ +3 on the -4 to +4 rating scale) in their responses to Q1 and Q2, which Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 constituted the external standards. An additional external standard was the definite presence of an ASAS positive MRI (BME "highly suggestive" of axSpA as per the 2016 definition). We similarly analyzed optimal cut-offs for the number of consecutive MRI slices that had active or structural lesions. We pre-specified optimal cut-offs as requiring specificity of ≥95% for a definite lesion.
We then assessed the predictive utility of baseline MRI data by calculating the sensitivity, axSpA(yes/no) (Q2). We then tested the predictive utility of the optimal SIJ quadrants/slice cutoffs identified in the prior analysis as having specificity of ≥95% for a definite lesion.
Both specificity of ≥95% for a definite lesion as well as a PPV of ≥95% were considered requirements for designation of cut-offs for defining a positive MRI for active or structural lesions.

Patient characteristics and frequencies of MRI lesions
For exercise A, scans with DICOM sequences and rheumatologist diagnosis at baseline were available from 169 cases and for 91 cases that also had a rheumatologist diagnosis at follow up (Table 1). Cases diagnosed with axSpA at baseline (114 (67.4%)) had significantly more clinical SpA features (3.4 vs 1.6), higher frequency of B27 positive (52.6% vs 21.8%), elevated CRP (37.7% vs 14.5%), and definite radiographic sacroiliitis according to modified New York criteria (28.8% vs 1.9%). Similar differences in these patient characteristics according to diagnosis of axSpA were also present at follow up. Active lesions typical of axSpA were observed on baseline MRI by a majority of readers (≥4/7) in 35.8% and 33.3% of cases diagnosed with axSpA at baseline and follow up, respectively, as compared to only 2.0% and 5.9% diagnosed as non-axSpA. Structural lesions typical of axSpA were observed on baseline MRI scans by a majority of readers (≥4/7) in 32.4% and 36.5% of cases diagnosed with axSpA at baseline and follow up, respectively, as compared to 6.8% and 0% diagnosed as non-axSpA (Table 1)

AxSpA
The cut-off for number of SIJ quadrants with BME at any location that reflected majority central reader designation of a definite active lesion typical of axSpA or an ASAS positive MRI (BME that is "highly suggestive" of axSpA) according to the pre-defined specificity of ≥95% was ≥4 ( Table 2). The presence of BME at the same location (i.e. same SIJ quadrant) in ≥3 consecutive slices also attained specificity of ≥95% for a definite active lesion or an ASAS positive MRI by the majority of central readers ( Table 3).
The cut-offs for number of SIJ quadrants that best reflected a definite structural lesion typical of axSpA according to the pre-defined specificity of ≥95% was ≥3 for erosion, ≥5 for fat lesion, and ≥7 for sclerosis (Table 2). A single SIJ quadrant or slice with an ASAS-defined fat lesion >1cm Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 in depth had a specificity of ≥95% for a definite structural lesion and specificity of 97.5% when present at the same location in ≥2 consecutive slices (Tables 2 and 3). The presence of erosion at the same location in ≥2 consecutive slices had ≥95% specificity while a fat lesion required ≥3 consecutive slices. The combination of fat lesion >1cm in depth in ≥1 SIJ quadrant and erosion in ≥1 SIJ quadrant (not necessarily the same quadrant) increased specificity to 99.2%. Additional combinations of fat lesion and erosion cut-offs did not enhance specificity.

Predictive Utility of Baseline MRI for Axial Spondyloarthritis
Analysis of baseline MRI scans in exercise A demonstrated that both active and structural lesions typical of axSpA, as well as designation of an ASAS positive MRI, detected by a majority of readers (≥4/7) had very high PPV (≥95%) for a follow up diagnosis of axSpA (Table 4). PPV for an 'ASAS positive MRI' was very similar to the PPV of 'active lesions typical of axSpA (Q1)' even though the former assesses only BME while the latter includes all active lesions in the SIJ.
However, NPV was low (<50%) irrespective of the type of lesion recorded by central readers.
In exercise B, baseline MRI that was considered indicative of axSpA (Q3) by a majority of readers (≥5/8) had 100% predictive utility for follow up diagnosis of axSpA. Both active and structural lesions had comparably high PPV. NPV was low (<50%) irrespective of the type of lesion and also for overall MRI global evaluation.

AxSpA
The presence of BME in ≥4 SIJ quadrants or at the same location in ≥3 SIJ slices on baseline MRI had very high PPV (≥95%) for follow up diagnosis of axSpA assessed by the rheumatologist (Table 5). This was comparable to the performance of the baseline global MRI assessment for 'active lesions typical of axSpA' and 'ASAS positive MRI' (Table 4), though with a slightly lower NPV.
The presence of erosion in ≥3 SIJ quadrants or at the same location in ≥2 SIJ slices on baseline MRI had very high PPV (≥95%) for diagnosis of axSpA ( Table 5). The presence of fat lesion in ≥5 SIJ quadrants, or at the same location in ≥3 SIJ slices, or with depth >1cm in ≥2 SIJ quadrants also had PPV≥95%. The performance of these structural lesion cut-offs was very comparable to the performance of the baseline global MRI assessment for 'structural lesions typical of axSpA', though with a slightly lower NPV. Examples of a positive MRI for active or structural lesions in the SIJ are provided in Figure 1.

Predictive Validity of Combinations of SIJ Lesion Cut-offs for Rheumatologist Diagnosis of AxSpA
Overall, combining cut-offs for different SIJ lesions did not substantially enhance predictive utility when compared to the cut-offs for individual lesions (Supplementary Table S1, available at Rheumatology online). For active lesions, there was slight improvement in predictive utility when cut-offs for fat lesion were combined with cut-offs for BME, primarily because of enhanced sensitivity without substantial compromise of specificity. For structural lesions, there was also slight improvement in predictive utility when cut-offs for fat lesion were combined with cut-offs for erosion due to enhanced sensitivity.

DISCUSSION
We report several observations that are highly relevant to the interpretation of MRI scans for both diagnosis and classification of axSpA. First, both active and structural MRI lesions considered typical of axSpA had equally high predictive utility for rheumatologist's follow up diagnosis of axSpA in patients with undiagnosed back pain referred with suspicion of axSpA. Second, ASASdefined BME in ≥4 SIJ quadrants at any location or in the same location on ≥3 consecutive slices were optimal for defining a definite active lesion typical of axSpA. Third, ASAS-defined erosion in ≥3 SIJ quadrants or at the same location in ≥2 consecutive slices and ASAS-defined fat lesion in ≥5 SIJ quadrants or ≥3 consecutive slices provided similarly high specificities and PPVs for defining a definite structural lesion typical of axSpA. Finally, we demonstrated that all these MRI lesion cut-offs had very high (≥95%) PPV for rheumatologist's follow up diagnosis of axSpA that was comparable to the predictive utility of global MRI evaluation.
Our data-driven definitions for an active lesion typical of axSpA would be consistent with recent studies describing false positive BME in up to 40% of individuals without axSpA when the cutoff applied was BME in only ≥2 SIJ quadrants or in ≥2 consecutive slices [12][13][14][15]. One previous study that assessed patients diagnosed by a rheumatologist with axSpA and those with nonspecific back pain or healthy controls demonstrated that BME in ≥3 and ≥4 SIJ quadrants had much higher specificity for axSpA (89% and 92%, respectively) without substantial decrease in sensitivity (75% and 70%, respectively) compared to a cut-off of ≥2 SIJ quadrants (sensitivity 80%, specificity 76%) [15]. However, this latter study was only cross-sectional and therefore did not analyze predictive utility of MRI lesion cut-offs for rheumatologist diagnosis of axSpA. A more recent Danish study reported that no controls had a BME score of ≥4 SIJ quadrants except Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 for women who had recently delivered a child [23]. An MRI study of patients with DISH also demonstrated that while 15.8% of patients met the ASAS definition of a positive MRI there were no DISH patients with BME in ≥4 SIJ quadrants [24]. Moreover, a 2015 study demonstrated that a cutoff value of ≥2 SIJ quadrants with BME optimally reflects the 3-reader consensus judgment of a positive versus negative MRI according to the ASAS definition [11]. These findings are not surprising because the quantitative part of the ASAS definition requires two BME lesions on a single slice or one lesion in 2 consecutive slices and most likely reflects circular reasoning. Moreover, a cut-off of ≥4 SIJ quadrants with BME or BME at the same location in ≥3 consecutive slices was optimal for reflecting an ASAS positive MRI considered "highly suggestive" of axSpA. This change over time in expert opinion as to what constitutes BME "highly suggestive" of axSpA is not surprising as data has accumulated demonstrating that false positive BME in the SIJ is a frequent observation in patients without axSpA. The thresholds for BME reported in this manuscript should not be applied to women within 6 months of delivery as several reports have highlighted a high frequency of substantial BME in the early post-partum period [23,25,26].
There has been more limited analysis of MRI lesion cut-offs that might define a structural lesion typical of axSpA and such studies have been cross-sectional. One study that analyzed rheumatologist diagnosis of axSpA as external reference concluded that erosion in ≥2 SIJ quadrants was optimal with specificity being 97% [15]. A study from the SPACE cohort used the ASAS classification of axSpA as external reference for defining the presence of axSpA and Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 concluded that ≥3 erosion lesions or ≥3 fat lesions or ≥5 of these lesions in any combination, each individual lesion being defined as being present on ≥2 consecutive slices, was optimal [27].
However, cut-offs according to number of SIJ quadrants with these lesions were not reported and it is therefore unclear how size of lesion was incorporated into these cut-offs. Moreover, in both studies, the lesion definition for erosion differed from the recently published ASAS-definition of erosion [2] which explicitly requires not only loss of the dark signal from cortical bone, which was included in previous studies, but also loss of the bright signal from adjacent marrow matrix.
An erosion cut-off of ≥3 SIJ quadrants derived from our analysis was more recently shown to differentiate between axSpA and controls, including post-partum women, in a Danish cohort with ≥95% specificity [23]. This was similarly reported as optimal for differentiation of patients with axSpA from those with DISH [24].
Opinion regarding the utility of fat lesions has evolved as it has become clear that it reflects the resolution of inflammation and has morphological characteristics on MRI that distinguish it from physiological fat infiltration. These have been incorporated in the recent update to the ASAS definitions of MRI lesions in the SIJ [2]. The revised ASAS definition for fat lesion includes distinct border, homogenous increase in T1W fat signal, and proximity to subchondral bone. The validation of this revised ASAS definition also included assessment of a 'deep' fat lesion, defined as having horizontal depth of >1cm from subchondral bone, because ASAS readers considered this might increase its specificity for axSpA. This was confirmed in this analysis and our data also demonstrated that a single 'deep' fat lesion on a single slice was 100% specific for axSpA.
However, we caution against the use of a cut-off of "1" for any imaging-based observation of a single lesion on a single image as such a low threshold is more likely to be subject to reader misinterpretation or simulation by imaging artifact.
There are several study limitations. The high PPV values for MRI overall and MRI lesion cutoffs in diagnosis of axSpA reinforces the importance that rheumatologists assign to this imaging modality in ascertaining the diagnosis and some degree of circular reasoning could therefore account for the high PPV values. However, the primary aim of the analysis was to determine what MRI cut-offs best define active or structural MRI lesions typical of axSpA and not what cut-offs define a diagnosis of axSpA since diagnosis is a complex process that integrates clinical and imaging data. PPV values for specific MRI cut-offs did not always increase with a stricter definition because of the frequent co-occurrence of active and structural lesions in cases from this cohort, as reported previously [22]. Higher MRI lesion cut-offs for specific lesions did not necessarily eliminate cases that were negative for a diagnosis of axSpA because such patients had other MRI lesions that contributed to diagnostic ascertainment. The low NPV values indicate that the lack of MRI findings does not rule out axSpA and that rheumatologists do not rely only on MRI to make a diagnosis of axSpA. These cut-offs will require further validation in different practice settings where the prevalence of axSpA, which influences PPV, may differ from that observed in the ASAS-CC. The follow up of the ASAS-CC incurred substantial patient drop-out and a higher proportion of patients assessed at follow up had MRI inflammation than those lost to follow up [16]. This channeling bias may inflate PPV values for MRI parameters.
In conclusion, we employed a novel two-step study design and three external standards to identify optimal MRI SIJ lesion cut-offs that reflect definite active or structural MRI lesions Downloaded from https://academic.oup.com/rheumatology/advance-article/doi/10.1093/rheumatology/keab099/6125390 by University College London user on 12 February 2021 typical of axSpA or an ASAS positive MRI "highly suggestive" for axSpA. The presence of BME in ≥4 SIJ quadrants or in ≥3 consecutive slices was optimal for both a definite active lesion and an ASAS positive MRI. The presence of erosion in ≥3 SIJ quadrants or ≥2 consecutive slices or the presence of fat lesion in ≥5 SIJ quadrants or ≥3 consecutive slices was optimal for a definite structural lesion typical of axSpA. The contribution of these cut-offs to the performance of classification criteria and diagnosis of axSpA will require testing in further studies. However, it is our hope that this work will finally lay to rest the frequent misuse of the quantitative component of the ASAS definition of a positive MRI, comprising BME in ≥2 locations on one slice or in one location on ≥2 consecutive slices, as a diagnostic criterion for axSpA.