?url_ver=Z39.88-2004&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Adc&rft.title=The+impact+of+Feedback+on+student+attainment%3A+a+systematic+review&rft.creator=Newman%2C+M&rft.creator=Kwan%2C+I&rft.creator=Schucan+Bird%2C+K&rft.creator=Hoo%2C+HT&rft.description=Abstract+Meta-syntheses+have+reported+positive+impacts+of+feedback+for+student+achievement+at+different+stages+of+education+and+have+been+influential+in+establishing+feedback+as+an+effective+strategy+to+support+student+learning.+However%2C+these+syntheses+combine+studies+of+a+variety+of+different+feedback+approaches%2C+combine+studies+where+feedback+is+one+of+a+number+of+intervention+components+and+have+several+methodological+limitations.+For+example+the+lack+of+quality+appraisal+of+the+included+studies.+There+is+also+still+more+research+needed+to+investigate+the+impact+of+different+types+of+feedback+on+different+students+in+different+settings.+Objective+This+systematic+review+was+conducted+at+the+request+of+the+Education+Endowment+Foundation+to+provide+more+precise+estimates+of+the+impact+of+different+types+of+feedback+in+different+contexts+for+different+learners+aged+between+5+and+18.+The+review+analysis+sought+to+explore+potential+variations+in+the+impact+of+feedback+through+subgroup+analysis+of+the+characteristics+of+the+feedback%2C+the+educational+setting%2C+the+learners+and+the+subject.+This+review+provides+evidence+that+can+be+used+to+support+the+development+of+guidance+for+teachers+and+schools+about+feedback+practices.+Methods+design+A+systematic+review+was+undertaken+in+two+stages.+First%2C+a+systematic+map+identified+and+characterised+a+subset+of+studies+that+investigated+the+attainment+impacts+of+feedback.+Second%2C+an+in-depth+review+comprising+of+a+meta-analysis+was+performed+to+answer+the+review+questions+about+the+impact+of+interventions+that+comprised+of+feedback+only+and+to+explore+the+variety+of+characteristics+that+may+influence+the+impact+of+feedback.+Methods+search+We+used+the+Microsoft+Academic+Graph+(MAG)+dataset+hosted+in+EPPI-Reviewer+to+conduct+a+semantic+network+analysis+to+identify+records+related+to+a+set+of+pre-identified+study+references.+The+MAG+search+identified+23%2C725+potential+studies+for+screening.+Methods+study+selection+Studies+were+selected+using+a+set+of+pre-specified+selection+criterion.+Semi-automated+priority+screening+was+used+to+screen+the+title+and+abstract+of+studies+using+bespoke+systematic+review+software+EPPI-Reviewer.+The+title+and+abstract+screening+was+stopped+after+3%2C028+studies+and+745+were+identified+for+full-text+screening.+Reviewers+carried+out+a+moderation+exercise%2C+all+screening+a+selection+of+the+same+titles+to+develop+consistency+of+screening.+Thereafter%2C+single+reviewer+screening+was+used+with+referral+for+a+second+reviewer+opinion+in+cases+of+uncertainty.+Methods+data+collection+Studies+were+coded+using+a+bespoke+data+extraction+tool+developed+by+the+EEF+Database+Project.+Study+quality+was+assessed+using+a+bespoke+risk+of+bias+assessment+adapted+from+the+ROBINS-I+tool.+The+review+team+undertook+a+moderation+exercise+coding+the+same+set+of+studies+to+develop+consistency.+Thereafter%2C+single+reviewer+coding+was+used%2C+based+on+the+full+text+with+referral+for+a+second+opinion+in+cases+of+uncertainty.+Methods+synthesis+Data+from+the+studies+was+used+to+calculate+standardised+effect+sizes+(Standardised+Mean+Difference-+Hedge%E2%80%99s+g).+Effect+sizes+from+each+study+were+combined+to+produce+a+pooled+estimate+of+effect+using+Random+Effects+Meta-analysis.+Statistical+Heterogeneity+tests+were+carried+out+for+each+synthesis.+Sensitivity+analysis+was+carried+out+for+assessed+study+quality.+Subgroup+analysis+was+completed+using+meta-analysis+to+explore+outcomes+according+to+the+different+characteristics+of+feedback%2C+context+and+subjects.+Main+results+The+full+text+screening+identified+304+studies+to+include+in+the+initial+systematic+map%2C+of+which+171+studies+investigated+feedback+only.+After+applying+final+selection+criteria%2C+43+papers+with+51+studies+published+in+and+after+the+year+2000+were+included.+The+51+studies+had+approximately+14%2C400+students.+Forty+studies+were+experiments+with+random+allocation+to+groups+and+11+were+prospective+quantitative+experimental+design+studies.+The+overall+ecological+validity+was+assessed+as+moderate+to+high+in+40+studies+and+the+overall+risk+of+bias+assessed+as+low+to+moderate+in+44+studies.+The+interventions+took+place+in+curriculum+subjects+including+literacy%2C+mathematics%2C+science%2C+social+studies%2C+and+languages%2C+and+tested+other+cognitive+outcomes.+The+source+of+feedback+included+teacher%2C+researcher%2C+digital%2C+or+automated+means.+Feedback+to+individual+students+is+reported+in+48+studies+and+feedback+to+group+or+class+is+reported+in+four+studies.+Feedback+took+the+form+of+spoken+verbal%2C+non-verbal%2C+written+verbal%2C+and+written+non-verbal.+Different+studies+investigated+feedback+that+took+place+immediately+after+the+task%2C+during+the+task+and+up+to+one+week+after+the+task+(delayed+feedback).+Most+of+the+feedback+interventions+gave+the+learner+feedback+about+the+outcome+and+the+process%2Fstrategy.+Some+provided+feedback+on+outcome+only+and+two+provided+feedback+about+task%2Fstrategy+only.+On+the+main+research+question%2C+the+pooled+estimate+of+effect+of+synthesis+of+all+studies+with+a+low+or+moderate+risk+of+bias+indicated+that+students+who+received+feedback+had+better+performance+than+students+who+did+not+receive+feedback+or+experienced+usual+practice+(g+%3D+0.17%2C+95%25+C.I.+0.09+to+0.25).+However%2C+there+is+statistically+significant+heterogeneity+between+these+studies+(I2+%3D+44%25%2C+Test+for+Heterogeneity%3A+Q(df+%3D+37)+%3D+65.92%2C+p+%3D+0.002)%2C+which+suggests+that+this+may+not+be+a+useful+indicator+of+the+general+impact+of+feedback+on+attainment+when+compared+to+no+feedback+or+usual+practice.+The+heterogeneity+analysis+suggested+considerable+heterogeneity+between+studies+in+the+main+synthesis+and+all+the+subgroup+synthesis%2C+and+in+the+majority+of+the+cases+the+heterogeneity+is+statistically+significant.+This+means+caution+is+required+when+considering+the+results+of+the+synthesis.+The+results+of+the+subgroup+synthesis+suggest+that+a+variety+of+student+and+context+factors+may+have+an+effect+on+the+impact+of+feedback.+Conclusions+The+results+of+the+review+may+be+considered+broadly+consistent+with+claims+made+on+the+basis+of+previous+synthesis+and+meta-synthesis%2C+suggesting+that+feedback+interventions%2C+on+average%2C+have+a+positive+impact+on+attainment+when+compared+to+no+feedback+or+usual+practice.+The+limitations+in+the+study+reports+and+the+comparatively+small+number+of+studies+within+each+subgroup+synthesis+meant+that+the+review+was+not+able+to+provide+very+much+more+certainty+about+the+factors+that+affect+variation+in+the+impact+of+single+component+feedback+interventions+within+different+contexts+and+with+different+students.+More+research+is+needed+in+this+area+to+consider+what+may+moderate+the+impact+of+feedback.+However%2C+the+findings+further+support+the+conclusion+made+by+previous+studies+that+feedback%2C+on+average%2C+has+a+positive+impact+on+attainment%3B+moreover%2C+this+is+based+on+a+more+precise+and+robust+analysis+than+previous+syntheses.+This+suggests+that+feedback+may+have+a+role+to+play+in+raising+attainment+alongside+other+effective+interventions.+Findings+were+further+interpreted+by+a+panel+of+expert+practitioners+and+academics+to+produce+the+EEF%E2%80%99s+Teacher+feedback+to+improve+pupil+learning+guidance+report.+1.+Background+and+review+rationale+Feedback+can+be+defined+as+information+communicated+to+the+learner+that+is+intended+to+modify+the+learner%E2%80%99s+thinking+or+behaviour+for+the+purpose+of+improving+learning.+Meta-syntheses+have+reported+positive+impacts+of+feedback%2C+with+effect+sizes+ranging+from+d+%3D+0.70+to+d+%3D+0.79+for+student+achievement+at+different+stages+of+education+and+have+been+influential+in+establishing+feedback+as+highly+effective+with+regards+to+student+learning.+For+example%2C+the+EEF+Teaching+and+Learning+Toolkit+meta-synthesis+suggests+that+feedback+may+have+%E2%80%98very+high%E2%80%99+impact+(equivalent+to+eight+months%E2%80%99+additional+progress)+for+relatively+low+cost.+However%2C+caution+is+necessary+when+interpreting+the+findings+of+these+meta-syntheses+for+a+number+of+reasons.+Firstly%2C+the+average+effect+size+reported+in+the+EEF+Toolkit+is+based+on+combining+the+estimates+from+existing+meta-analyses+of+individual+studies%2C+which+may+contain+limitations+of+various+kinds+(see+the+list+below+for+examples)+that+may+mean+that+average+effect+sizes+identified+are+overestimates.+Second%2C+some+studies+included+in+syntheses+(such+as+Kluger+and+DeNisi%E2%80%99s+meta-analysis+)+suggest+that+some+feedback+interventions+may%2C+in+fact%2C+negatively+impact+pupils.+Third%2C+previous+meta-syntheses+have+not+explored+in+detail+the+impact+of+potential+moderating+factors%2C+such+as+different+types+of+feedback.+As+Ekecrantz+has+argued%2C+there+is+still+a+need+to+better+understand+how+and+under+what+circumstances+teacher+feedback+on+student+performance+promotes+learning+as+well+as%2C+to+question+the+generalised+claim+(that+feedback+improves+attainment)+itself.+For+example%2C+a+recent+meta-analysis+that+re-analysed+studies+included+in+the+original+synthesis+by+Hattie+and+Timperley+revised+down+the+average+effect+size+from+the+estimates+of+the+effects+of+feedback+from+their+originally+published+Standardised+Mean+Difference+of+d+%3D+0.79+to+d+%3D+0.48.+In+the+revised+meta-analysis%2C+17%25+of+the+effect+sizes+from+individual+studies+were+negative.+The+confidence+interval+ranged+from+d+%3D+0.48+to+d+%3D+0.62%2C+and+the+authors+found+a+wide+range+of+effect+sizes.+Different+moderators+were+also+investigated+to+explore+the+impact+of+different+characteristics+of+context+and+feedback.+Whilst+this+meta-analysis+offers+improvements+over+previous+meta-syntheses%2C+it+has+a+number+of+limitations%2C+including%3A+%E2%80%A2+It+only+included+studies+drawn+from+36+existing+meta-analyses%2C+the+most+recent+of+which+was+published+in+2015.+Eligible+studies+published+after+2015+or+not+included+in+these+meta-analyses+would+not+have+been+included.+%E2%80%A2+All+comparative+study+designs+were+included.+Less+robust+study+designs+may+have+overestimated+the+positive+effect+of+feedback.+%E2%80%A2+There+was+no+reported+study+quality+assessment%2Fmoderation+or+sensitivity+analysis%2C+which+may+have+led+to+an+overestimation+of+the+pooled+effect+sizes.+%E2%80%A2+The+meta-analyses+included+studies+with+high+levels+of+heterogeneity%2C+I2+%3D+80%25+or+more+(in+the+main+and+moderator+analysis).+This+suggests+that+the+synthesis+may+be+combining+studies%2Fcomparing+feedback+practices+inappropriately.+%E2%80%A2+The+meta-analysis+did+not+consider+all+potentially+relevant+moderating+factors.+It+may+also+be+the+case+that+the+impact+of+feedback+depends+on+factors+other+than+those+analysed%2C+including+the+ability+of+the+learner%2C+the+learning+context%2C+and%2For+the+frequency%2C+duration%2C+timing%2C+and+type+of+feedback.+This+systematic+review+was+conducted+at+the+request+of+the+EEF+to+try+and+provide+more+accurate+and+precise+estimates+of+the+impact+of+different+types+of+feedback+in+different+schooling+contexts.+The+review+examines+the+impact+of+single+component+feedback%2C+in+different+contexts%2C+and+for+different+learners+with+a+greater+degree+of+granularity+and+precision+than+is+currently+available+via+the+EEF+Teaching+and+Learning+Toolkit+strand+on+%E2%80%98Feedback%E2%80%99.+For+EEF%2C+the+purpose+of+the+systematic+review+is+to+provide+evidence+that+can+be+used+to+inform+guidance+for+teachers+and+schools+about+effective+feedback+practices.+The+systematic+review+methods+and+processes+were+developed+and+carried+out+conterminously+with+the+EEF+Database+project+with+a+view+to+facilitating+the+future+use+of+the+produced+resources+and+supporting+the+ongoing+work+of+the+Database+project.+1.1+Domain+being+studied%3A+Feedback+approaches+This+review+focuses+on+interventions+that+provide+feedback+from+teachers+to+learners+in+mainstream+educational+settings.+Feedback+is+defined+in+accordance+with+the+EEF+toolkit+definition%3A+%E2%80%98Feedback+is+information+given+to+the+learner+and%2For+teacher+about+the+learner%E2%80%99s+performance+relative+to+learning+goals+or+outcomes.+It+should+aim+to+produce+(and+be+capable+of)+producing+improvement+in+students%E2%80%99+learning.+Feedback+redirects+or+refocuses+either+the+teacher%E2%80%99s+or+the+learner%E2%80%99s+actions+to+achieve+a+goal%2C+by+aligning+effort+and+activity+with+an+outcome.+It+can+be+about+the+output+of+the+activity%2C+the+process+of+the+activity%2C+the+student%E2%80%99s+management+of+their+learning+or+self-regulation%2C+or+them+as+individuals.+This+feedback+can+be+verbal+or+written+or+can+be+given+through+tests+or+via+digital+technology.+It+can+come+from+a+teacher+or+someone+taking+a+teaching+role%2C+or+from+%E2%80%98peers%E2%80%99.%E2%80%99+This+initial+broad+definition%2C+whilst+conceptually+coherent%2C+does+create+challenges+both+in+practice+for+teachers+and+in+terms+of+identifying+and+distinguishing+between+practices+when+considering+research+evidence.+For+example%2C+what+is+the+difference+between+small+group+learning+and+%E2%80%98peer+feedback%E2%80%99%3F+It+seems+perfectly+reasonable+to+assume+that+small+group+learning+must+contain+conversations+between+students+about+their+work+and+the+task+they+have+been+asked+to+complete+and+thus+is+%E2%80%98feedback%E2%80%99.+However%2C+in+practice%2C+this+may+not+be+what+teachers+think+of+as+%E2%80%98feedback%E2%80%99+and+in+the+research+literature%2C+%E2%80%98small+group+learning%E2%80%99+is+investigated+both+as+a+unique+pedagogical+strategy+and+as+a+component+of+a+number+of+other+pedagogical+strategies.+As+the+development+of+the+understanding+of+the+scope+of+the+review+evolved%2C+the+working+definition+of+feedback+for+the+review+became+modified+practically+through+the+exclusion+of+certain+categories+of+intervention%2C+even+though+they+may+contain+an+element+of+feedback+practice.+The+inclusion+criteria+in+the+methods+section+outlines+the+revised+definition+that+the+review+team+used.+1.2+Conceptual+framework%2FTheory+of+Change+There+are+several+ways+in+which+feedback+is+conceptualised+as+improving+learner+performance%E2%80%94i.e.+as+a+Theory+of+Change.+The+%E2%80%98Feedback%E2%80%99+strand+in+the+EEF+Teaching+and+Learning+Toolkit+draws+most+explicitly+on+the+conceptualisation+of+Hattie+and+Timperley%E2%80%99s+(2007)+model.+This+model+emphasises+the+importance+of+systems+of+feedback+where+the+teacher+provides+feedback+to+the+specific+needs+of+individual+students.+The+searching+processes+used+in+this+review+are+consistent+with+this+model+as+the+studies+used+in+the+Feedback+strand+of+the+EEF+Teaching+and+Learning+Toolkit+were+used+to+%E2%80%98seed%E2%80%99+the+search.+However%2C+they+did+not+preclude+the+inclusion+of+studies+that+may+draw+on+other+%E2%80%98models%E2%80%99+of+feedback+which%2C+though+similar+to+Hattie+and+Timperley+(2007)%2C+may+be+argued+to+place+more+emphasis+on%2C+for+example%3A+developing+learner+self-regulation+(Nicole+and+Macfarlane-Dick%2C+2006)%3B+students%E2%80%99+intrinsic+motivation+(Dweck%2C+2016)%3B+and%2For+are+subject+specific%E2%80%94for+example%2C+%E2%80%98Thinking+Mathematically%E2%80%99+(Mason%2C+Burton+and+Stacey%2C+2010).+The+coding+tools+used+in+the+review+were+informed+by+the+model+(in+terms+of+coding+about+the+source+and+content+of+the+feedback%3B+see+Appendix+3).+1.3+Review+design+A+systematic+review+approach+was+used+to+investigate+the+research+questions.+The+review+was+undertaken+in+two+stages.+First%2C+a+systematic+map+identified+and+described+the+feedback+characteristics+of+a+subset+of+studies+that+investigated+the+attainment+impacts+of+feedback.+The+map+was+used+to+make+decisions+about+focusing+the+analysis+in+the+second+in-depth+systematic+review+stage.+At+the+second+stage+an+in-depth+review%2C+including+meta-analysis%2C+was+performed+on+a+subset+of+the+studies+identified+in+the+map+to+answer+the+review+questions+and+explore+the+variety+of+intervention+and+context+characteristics+that+may+influence+the+impact+of+feedback.+This+systematic+review+was+designed+to+complement+the+work+of+the+EEF+Database+project.+The+EEF+Database+project+is+currently+undertaking+a+programme+to+extract+and+code+the+individual+studies+from+the+meta-synthesis+used+in+the+EEF+Teaching+and+Learning+Toolkit.+The+search+strategy+used+in+this+review+was+%E2%80%98seeded%E2%80%99+from+studies+identified+as+being+about+%E2%80%98feedback%E2%80%99+in+the+database%2C+and+this+systematic+review+used+the+coding+tools+developed+by+the+Database+team+(see+Appendix+3).+The+studies+newly+identified+in+this+review+will+be+subsequently+included+in+the+EEF+Database.+This+systematic+review+was+also+designed+to+provide+additional+research+evidence+for+use+in+guidance+on+feedback+developed+for+schools+produced+by+the+EEF%2C+and+therefore+to+fit+with+a+particular+time+window+for+the+review%E2%80%99s+production.+The+results+of+the+meta-analyses+were+presented+to+an+advisory+panel+of+academics+and+teaching+practitioners%2C+who+used+the+results+%2C+their+own+expertise%2C+a+review+of+practice+undertaken+by+the+University+of+Oxford%2C+and+conceptual+models+(such+as+Hattie+and+Timperley)+to+draft+recommendations+for+practice.&rft.subject=Systematic+Review%2C+Feedback%2C+Education%2C+Effectiveness%2C+Meta-analysis&rft.publisher=Education+Endowment+Foundation&rft.date=2021-08-02&rft.type=Report&rft.language=eng&rft.source=++++Education+Endowment+Foundation%3A+London%2C+UK.+++++&rft.format=text&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10138571%2F1%2FSystematic-Review-of-Feedback-EPPI-2021.pdf&rft.identifier=https%3A%2F%2Fdiscovery.ucl.ac.uk%2Fid%2Feprint%2F10138571%2F&rft.rights=open