UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal

Carpentieri, JD; Mallows, D; Amorim, JP; (2020) Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal. Adult Literacy Education , Spring pp. 6-21. 10.35847/JCarpentieri.JAmorim.DMallows.PFreire.2.1.6. Green open access

[thumbnail of Credibility, relevance and policy impact_ New Opportunities Initiative.pdf]
Preview
Text
Credibility, relevance and policy impact_ New Opportunities Initiative.pdf - Accepted Version

Download (253kB) | Preview

Abstract

Adult basic education (ABE) policies aim to help adults improve their literacy, numeracy and information and communications technology skills, as well as their qualifications, often in pursuit of economic gains such as better employment and earnings. The large-scale improvement of skills and qualifications has been referred to as a wicked policy problem, suggesting that it is extremely difficult and perhaps even impossible to achieve success in this policy domain. Evaluations have highlighted these challenges, with many programs showing little or no impact. Between 2006 and 2012, the Portuguese government ran a large-scale adult education program, the New Opportunities Initiative (NOI), which focused on the recognition and validation of adults’ existing skills and the development of literacy and numeracy. The NOI was evaluated twice, in 2009 and in 2012. These two evaluations produced very different findings and outcomes: the first evaluation found the NOI to be a success, and led to continued investment, but the second evaluation reached more negative conclusions and was used as a rationale for de-funding the program. In this article we analyze these two sets of evaluations, investigating the reasons for their starkly different conclusions. We find that, while both evaluations had strengths, they also suffered from serious methodological and/or theoretical weaknesses. These weaknesses are part of a broader pattern of evaluation errors that characterize the field of ABE more generally and which make it more likely that ABE policies will continue to fail. Using the conflicting NOI evaluations as case studies, we offer potential solutions to ABE’s evaluation problem, emphasizing the need to collect long-term longitudinal evidence on the causal mechanisms through which policy goals may be achieved.

Type: Article
Title: Credibility, Relevance, and Policy Impact in the Evaluation of Adult Basic Skills Programs: The Case of the New Opportunities Initiative in Portugal
Open access status: An open access version is available from UCL Discovery
DOI: 10.35847/JCarpentieri.JAmorim.DMallows.PFreire.2.1.6
Publisher version: http://doi.org/10.35847/JCarpentieri.JAmorim.DMall...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Culture, Communication and Media
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Education, Practice and Society
URI: https://discovery.ucl.ac.uk/id/eprint/10090947
Downloads since deposit
7Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item