Tomlinson, Eve;
Holmes, Jude;
Rutjes, Anne WS;
Davenport, Clare;
Leeflang, Mariska;
Yang, Bada;
Mallett, Sue;
(2025)
Developing a framework for assessing the applicability of the target condition in diagnostic research.
Journal of Clinical Epidemiology
, Article 112059. 10.1016/j.jclinepi.2025.112059.
(In press).
Preview |
Text
1-s2.0-S0895435625003920-main.pdf - Accepted Version Download (895kB) | Preview |
Abstract
OBJECTIVE: Assessment of the applicability of primary studies is an essential but often challenging aspect of systematic reviews of diagnostic test accuracy studies (DTA reviews). We explored review authors' applicability assessments for the QUADAS-2 reference standard domain within Cochrane DTA reviews. We highlight applicability concerns, identify potential issues with assessment, and develop a framework for assessing the applicability of the target condition as defined by the reference standard. STUDY DESIGN AND SETTING: Methodological review. DTA reviews in the Cochrane Library that used QUADAS-2 and judged applicability for the reference standard domain as "high concern" for at least one study, were eligible. One reviewer extracted the rationale for the "high concern", this was checked by a second reviewer. Two reviewers categorized the rationale inductively into themes, a third reviewer verified these. Discussions regarding the extracted information informed framework development. RESULTS: We identified 50 eligible reviews. Five themes emerged: study uses different reference standard threshold to define the target condition (6 reviews), misclassification by the reference standard in the study such that the target condition in the study does not match the review question (11 reviews), reference standard could not be applied to all participants resulting in a different target condition (5 reviews), misunderstanding QUADAS-2 applicability (7 reviews), and insufficient information (21 reviews). Our framework for researchers outlines four potential applicability concerns for the assessment of the target condition as defined by the reference standard: different sub-categories of the target condition, different threshold used to define the target condition, reference standard not applied to full study group, and misclassification of the target condition by the reference standard. CONCLUSION: Clear sources of applicability concerns are identifiable, but several Cochrane review authors struggle to adequately identify and report them. We have developed an applicability framework to guide review authors in their assessment of applicability concerns for the QUADAS reference standard domain. Plain Language Summary What is the problem? Doctors use tests to help to decide if a person has a certain condition. They want to know how accurate the test is before they use it. This means how well it can tell people who have the condition from people who do not have it. This information can be found in "diagnostic systematic reviews". Diagnostic systematic reviews start with a research question. They bring together findings from studies that have already been done to try to answer this question. It is important for researchers to check that the studies match the review question. This is called an "applicability assessment". For example, if the review looks at children, it is important to check whether the studies look at children too. There is a tool named "QUADAS-2" that can be used to check how well the studies match the review question. This can be hard to do and there are not many examples available to help people. What did we do? We wanted to know more about how people judge applicability concerns using the QUADAS-2 tool. We also wanted to write guidance to support judgements about applicability. What did we find? We found examples of how people are doing applicability assessment. Many reviews have not done it right and we explain why this is. We also made guidance to help people with applicability assessment.
Archive Staff Only
 |
View Item |
