Sims, Samuel;
Anders, Jake;
Zieger, Laura;
(2022)
The internal validity of the school-level comparative interrupted time series design: evidence from four new within-study comparisons.
Journal of Research on Educational Effectiveness
10.1080/19345747.2022.2051652.
(In press).
Preview |
Text
Sims_19345747.2022.pdf Download (2MB) | Preview |
Abstract
Comparative interrupted time series (CITS) designs evaluate impact by modeling the relative deviation from trends among a treatment and comparison group after an intervention. The broad applicability of the design means it is widely used in education research. Like all non-experimental evaluation methods however, the internal validity of a given CITS evaluation depends on assumptions that cannot be directly verified. We provide an empirical test of the internal validity of CITS by conducting four within-study comparisons of school-level interventions previously evaluated using randomized controlled trials. Our estimate of bias across these four studies is 0.03 school-level (or 0.01 pupil-level) standard deviations. The results suggest well-conducted CITS evaluations of similar school-level education interventions are likely to display limited bias.
| Type: | Article |
|---|---|
| Title: | The internal validity of the school-level comparative interrupted time series design: evidence from four new within-study comparisons |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1080/19345747.2022.2051652 |
| Publisher version: | https://doi.org/10.1080/19345747.2022.2051652 |
| Language: | English |
| Additional information: | © 2022 The Author(s). Published with license by Taylor & Francis Group, LLC This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. |
| Keywords: | Comparative interrupted time series; randomised controlled trials; withinstudy comparisons; internal validity |
| UCL classification: | UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Learning and Leadership > Centre for Education Policy and Equalising Opportunities UCL > Provost and Vice Provost Offices > School of Education UCL UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Learning and Leadership |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10144364 |
Archive Staff Only
 |
View Item |
