UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes

Outhwaite, L; Gulliford, A; Pitchford, N; (2019) A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes. International Journal of Research and Method in Education 10.1080/1743727X.2019.1657081. (In press). Green open access

[img]
Preview
Text
Outhwaite et al 2019.pdf - Published version

Download (1MB) | Preview

Abstract

Randomized control trials (RCTs) are commonly regarded as the ‘gold standard’ for evaluating educational interventions. While this experimental design is valuable in establishing causal relationships between the tested intervention and outcomes, reliance on statistical aggregation typically underplays the situated context in which interventions are implemented. Developing innovative, systematic methods for evaluating implementation and understanding its impact on outcomes is vital to moving educational evaluation research beyond questions of ‘what works’, towards better understanding the mechanisms underpinning an intervention’s effects. The current study presents a pragmatic, two-phased approach that combines qualitative data with quantitative analyses to examine the causal relationships between intervention implementation and outcomes. This new methodological approach is illustrated in the context of a maths app intervention recently evaluated in a RCT across 11 schools. In phase I, four implementation themes were identified; ‘teacher support’, ‘teacher supervision’, ‘implementation quality’, and ‘established routine’. In phase II, ‘established routine’ was found to predict 41% of the variance in children’s learning outcomes with the apps. This has significant implications for future scaling. Overall, this new methodological approach offers an innovative method for combining process and impact evaluations when seeking to gain a more nuanced understanding of what works in education and why.

Type: Article
Title: A new methodological approach for evaluating the impact of educational intervention implementation on learning outcomes
Open access status: An open access version is available from UCL Discovery
DOI: 10.1080/1743727X.2019.1657081
Publisher version: https://doi.org/10.1080/1743727X.2019.1657081
Language: English
Additional information: This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Keywords: Implementation, intervention, evaluation, mixed-methods, education
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Learning and Leadership
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Learning and Leadership > Centre for Education Policy and Equalising Opportunities
URI: https://discovery.ucl.ac.uk/id/eprint/10082816
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item