eprintid: 10205016
rev_number: 8
eprint_status: archive
userid: 699
dir: disk0/10/20/50/16
datestamp: 2025-02-20 11:57:43
lastmod: 2025-02-20 11:59:56
status_changed: 2025-02-20 11:57:43
type: article
metadata_visibility: show
sword_depositor: 699
creators_name: Sousa-Pinto, Bernardo
creators_name: Neumann, Ignacio
creators_name: Vieira, Rafael José
creators_name: Bognanni, Antonio
creators_name: Marques-Cruz, Manuel
creators_name: Gil-Mata, Sara
creators_name: Mordue, Simone
creators_name: Nevill, Clareece
creators_name: Baio, Gianluca
creators_name: Whaley, Paul
creators_name: Schwarzer, Guido
creators_name: Steele, James
creators_name: Stewart, Gavin
creators_name: Schünemann, Holger J
creators_name: Azevedo, Luís Filipe
title: Quantitative assessment of inconsistency in meta-analysis using decision thresholds with two new indices
ispublished: inpress
divisions: UCL
divisions: B04
divisions: C06
divisions: F61
keywords: GRADE; Heterogeneity; Inconsistency; Meta-analysis; Systematic review
note: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
abstract: Objective:
In evidence synthesis, inconsistency is typically assessed visually and with the I2 and the Q statistics. However, these measures have important limitations (i) if there are few primary studies of small sample sizes, or (ii) if there are multiple studies with precise estimates. In addition, with the increasing use of decision thresholds (DT), for example in GRADE Evidence to Decision frameworks, inconsistency judgments can be anchored around DTs. In this article, we developed quantitative measures to assess inconsistency based on DTs.
//
Study Design and Setting:
We developed two measures to quantify inconsistency based on DTs – the Decision Inconsistency (DI) and the Across-Studies Inconsistency (ASI) indices. The DI and the ASI are based on the distribution of the posterior samples studies’ effect sizes across interpretation categories defined by DTs. We developed these indices for the Bayesian context, followed by a frequentist extension.
//
Results:
The DI informs on the overall inconsistency of effect sizes across interpretation categories, while the ASI quantifies how different studies are compared to each other (in relation to interpretation categories) based on absolute effects. A DI≥50% and an ASI≥25% are suggestive of important unexplained inconsistency. We provide an R package (metainc) and a web tool (https://metainc.med.up.pt/) to support the computation of the DI and ASI, including in the context of sensitivity analyses assessing the impact of potential uncertainty in inconsistency.
//
Conclusion:
The DI and the ASI can contribute to quantitatively assess inconsistency, particularly as DTs are gaining recognition in evidence synthesis and health decision-making.
date: 2025-02-13
date_type: published
publisher: Elsevier BV
official_url: https://doi.org/10.1016/j.jclinepi.2025.111725
full_text_type: other
language: eng
verified: verified_manual
elements_id: 2362123
doi: 10.1016/j.jclinepi.2025.111725
pii: S0895-4356(25)00058-7
lyricists_name: Baio, Gianluca
lyricists_id: GBAIO87
actors_name: Flynn, Bernadette
actors_id: BFFLY94
actors_role: owner
full_text_status: restricted
publication: Journal of Clinical Epidemiology
article_number: 111725
event_location: United States
issn: 0895-4356
citation:        Sousa-Pinto, Bernardo;    Neumann, Ignacio;    Vieira, Rafael José;    Bognanni, Antonio;    Marques-Cruz, Manuel;    Gil-Mata, Sara;    Mordue, Simone;                                 ... Azevedo, Luís Filipe; + view all <#>        Sousa-Pinto, Bernardo;  Neumann, Ignacio;  Vieira, Rafael José;  Bognanni, Antonio;  Marques-Cruz, Manuel;  Gil-Mata, Sara;  Mordue, Simone;  Nevill, Clareece;  Baio, Gianluca;  Whaley, Paul;  Schwarzer, Guido;  Steele, James;  Stewart, Gavin;  Schünemann, Holger J;  Azevedo, Luís Filipe;   - view fewer <#>    (2025)    Quantitative assessment of inconsistency in meta-analysis using decision thresholds with two new indices.                   Journal of Clinical Epidemiology      , Article 111725.  10.1016/j.jclinepi.2025.111725 <https://doi.org/10.1016/j.jclinepi.2025.111725>.    (In press).   
 
document_url: https://discovery.ucl.ac.uk/id/eprint/10205016/1/Quantitative%20assessment%20of%20inconsistency%20in%20meta-analysis%20using%20decision%20thresholds%20with%20two%20new%20indices..pdf