Two recent systematic reviews for development
Short Description
Download Two recent systematic reviews for development...
Description
Annette N. Brown, Deputy Director for AIES
TWO RECENT SYSTEMATIC REVIEWS FOR DEVELOPMENT International Initiative for Impact Evaluation
www.3ieimpact.org
Outline • • • • • •
Review questions Inclusion criteria Theory of change Search pipeline Results Fun methods finding
International Initiative for Impact Evaluation
www.3ieimpact.org
“Interventions in Developing Nations for Improving Primary and Secondary School Enrollment of Children: A Systematic Review” By Anthony Petrosino, Claire Morgan, Trevor A. Fronius, Emily E. Tanner-Smith, Robert F. Boruch, November 2012
International Initiative for Impact Evaluation
www.3ieimpact.org
Review questions • What are the effects of interventions implemented in developing countries on measures of students’ enrollment, attendance, graduation, and progression? • Within those studies that report the effects of an intervention on [the above measures], what are the ancillary effects on learning outcomes? International Initiative for Impact Evaluation
www.3ieimpact.org
Inclusion criteria Population/participants: Primary and secondary school aged children in LMICs Interventions: Intended to affect one of the four primary outcomes Comparison: No specific program comparison Outcomes: Enrollment, attendance, graduation, and progression Studies: RCTs and QEDs International Initiative for Impact Evaluation
www.3ieimpact.org
Typology of education interventions
Demand
Reducing costs
Providing information
CCTs, scholarships and non-fee subsidies
Supply
Increasing preparedness
Buildings
Early child development
Vouchers
Abolishing school fees and capitation grants
International Initiative for Impact Evaluation
Health/ nutrition
Teachers
Materials
Management School feeding
www.3ieimpact.org
International Initiative for Impact Evaluation
www.3ieimpact.org
International Initiative for Impact Evaluation
www.3ieimpact.org
International Initiative for Impact Evaluation
www.3ieimpact.org
International Initiative for Impact Evaluation
www.3ieimpact.org
International Initiative for Impact Evaluation
www.3ieimpact.org
“Farmer field schools for improving farming practices and farmer outcomes in low- and middle-income countries: a systematic review” By Hugh Waddington, Birte Snilstveit, Jorge Hombrados, Martina Vojtkova, Daniel Phillips, and Howard White, December 2012
International Initiative for Impact Evaluation
www.3ieimpact.org
Review questions • What is the impact of farmer field schools on their objectives in terms of ‘endpoint’ outcomes such as increased yields, net revenues and farmer empowerment, and intermediate outcomes such as capacity building and adoption of improved practices? • Under which circumstances and why: what are the facilitators and barriers to FFS effectiveness and sustainability? International Initiative for Impact Evaluation
www.3ieimpact.org
Inclusion criteria • Population/participants: Farm households in low and middle income countries • Intervention: Programs explicitly referred to as ‘farmer field school’ • Comparison: No specific program comparison • Outcomes: effectiveness across the causal chain – Knowledge → adoption → – Impact on yields, revenues, environment, health, empowerment
• Studies: – Effects: experimental, quasi-experimental with controlled comparison – Barriers/facilitators: qualitative (Critical Appraisal Skills Programme checklist 2006) International Initiative for Impact Evaluation
www.3ieimpact.org
Input 1 Training of trainers
Input 2 Field school
T of Change
Capacity building (FFS participants)
Capacity building (FFS neighbours)
Adoption (FFS participants)
Adoption (FFS neighbours)
Measured impacts:
International Initiative for
Yield, input-output ratio, income, empowerment, environmental Impact Evaluation outcomes, health
www.3ieimpact.org
Input 1 Training of trainers
- Facilitators adequately trained - Farmers and facilitators attend sufficient meetings - FFS synchronised with planting season
- Curriculum relevant to problems facing farmers -Farmer attitudes changed (convinced message appropriate) - Relative advantage over old techniques Initiative International
Input 2 Field school
Capacity building (FFS participants)
Capacity building (neighbours)
Adoption (FFS participants)
Adoption (neighbours)
for
- Field days/follow-up - High degree of social cohesion - Geographical proximity to other farmers (observation) or market (communication)
- New technology appropriate - Market access Measured impacts: - Favorable prices Yield, input-output ratio, - Environmental factors income, empowerment, including weather, soil environmental fertility Impact Evaluation www.3ieimpact.org outcomes. health
Effectiveness
27,866 titles screened
1453 abstracts screened
Causal Chain Analysis 1,112 abstracts screened 751 excluded
126 no access
369 full text obtained
186 excluded: 128 on relevance 58 on design (no comparison)
183 Extension impact papers: 134 FFS 49 non-FFS
134 FFS impact papers
80 individual FFS studies
International Initiative for Impact Evaluation
312 full text sought
49 no access
257 excluded
Qualitative Synthesis 25 qualitative papers
30 IE and sister papers
20 individual FFS studies
11 individual FFS studies
BB+ Synthesis
www.3ieimpact.org
Study ID
ES (95% CI)
FFS participants Huan et al., 1999 (Vietnam) Endalew, 2009 (Ethiopia) Price et al., 2001 (Philippines) Rao et al., 2012 (India) Reddy & Suryamani, 2005 (India) Kelemework, 2005 (Ethiopia) Mutandwa & Mpangwa, 2004 (Zimbabwe) Dinpanah et al., 2010 (Iran) Khan et al., 2007 (Pakistan) Bunyatta et al., 2006 (Kenya) Erbaugh, 2010 (Uganda) Rebaudo & Dangles, 2011 (Ecuador) Subtotal (I-squared = 93.3%, p = 0.000) . FFS neighbours Khan et al., 2007 (Pakistan) Reddy & Suryamani, 2005 (India) Ricker-Gilbert et al, 2008 (Bangladesh) Rebaudo & Dangles, 2011 (Ecuador) Subtotal (I-squared = 0.0%, p = 0.610) . NOTE: Weights are from random effects analysis
0.02 (-0.06, 0.10) 0.27 (-0.06, 0.60) 0.42 (-0.17, 1.01) 0.43 (-0.02, 0.87) 0.45 (-0.04, 0.94) 0.54 (-0.22, 1.29) 0.59 (0.25, 0.92) 0.67 (0.41, 0.92) 0.79 (0.29, 1.29) 1.03 (0.65, 1.41) 1.14 (0.93, 1.34) 1.79 (1.17, 2.41) 0.66 (0.33, 1.00)
Positive impacts on knowledge among participants
-0.13 (-0.68, 0.42) 0.05 (-0.45, 0.56) 0.17 (-0.25, 0.59) 0.38 (-0.15, 0.91) 0.13 (-0.12, 0.37)
-.5
0
.5 1 Favours intervention
International Initiative for Impact Evaluation
3
www.3ieimpact.org
Study ID
ES (95% CI)
FFS neighbours Pananurak, 2010 (India) Khan et al., 2007 (Pakistan) Feder et al, 2004 (Indonesia) Labarta, 2005 (Nicaragua) Pananurak, 2010 (China) Wu Lifeng, 2010 (China) Pananurak, 2010 (Pakistan) Yamazaki & Resosudarmo, 2007 (Indonesia) Subtotal (I-squared = 49.5%, p = 0.054) . FFS participants Pananurak, 2010 (India) Van Rijn, 2010 (Peru) Naik et al., 2008 (India) Huan et al., 1999 (Vietnam) Labarta, 2005 (Nicaragua) Rejesus et al, 2010 (Vietnam) Feder et al, 2004 (Indonesia) Wu Lifeng, 2010 (China) Ali & Sharif, 2011 (Pakistan) Pananurak, 2010 (China) Gockowski et al., 2010 (Ghana) Yang et al., 2005 (China) Hiller et al., 2009 (Kenya) Khan et al., 2007 (Pakistan) Cavatassi et al., 2011 (Ecuador) Davis et al, 2012 (Tanzania) Birthal et al., 2000 (India) Pananurak, 2010 (Pakistan) Dinpanah et al., 2010 (Iran) Wandji et al., 2007 (Cameroon) Mutandwa & Mpangwa, 2004 (Zimbabwe) Palis, 1998 (Philippines) Zuger 2004 (Peru) Carlberg et al., 2012 (Ghana) Yamazaki & Resosudarmo, 2007 (Indonesia) Van den Berg et al., 2002 (Sri Lanka) Davis et al, 2012 (Kenya) Pande et al., 2009 (Nepal) Dinpanah et al., 2010 (Iran) Orozco Cirilo et al., 2008 b) (Mexico) Todo & Takahashi, 2011 (Ethiopia) Subtotal (I-squared = 93.0%, p = 0.000) . NOTE: Weights are from random effects analysis
0.79 (0.63, 1.00) 0.97 (0.74, 1.26) 0.99 (0.97, 1.01) 1.00 (0.99, 1.01) 1.02 (0.98, 1.07) 1.03 (0.99, 1.08) 1.03 (0.86, 1.25) 1.43 (1.05, 1.96) 1.01 (0.98, 1.03)
0.80 (0.61, 1.05) 0.86 (0.63, 1.18) 0.89 (0.83, 0.96) 0.95 (0.92, 0.98) 0.97 (0.92, 1.02) 0.97 (0.72, 1.31) 0.98 (0.96, 1.01) 1.08 (1.03, 1.14) 1.09 (1.03, 1.15) 1.09 (1.04, 1.14) 1.14 (1.03, 1.25) 1.15 (0.94, 1.41) 1.17 (0.53, 2.56) 1.17 (0.97, 1.42) 1.22 (0.97, 1.53) 1.23 (1.00, 1.51) 1.24 (1.13, 1.36) 1.24 (1.01, 1.54) 1.32 (1.22, 1.42) 1.32 (1.07, 1.63) 1.36 (1.06, 1.73) 1.36 (0.97, 1.92) 1.44 (1.09, 1.92) 1.58 (1.19, 2.10) 1.67 (1.23, 2.26) 1.68 (1.30, 2.18) 1.81 (1.15, 2.84) 2.11 (1.25, 3.56) 2.52 (2.05, 3.11) 2.62 (2.23, 3.08) 2.71 (1.11, 6.60) 1.23 (1.16, 1.32)
.5
1
2
Increased yields among FFSbeneficiaries not neighbours
3
Favours intervention International Initiative for Impact Evaluation
www.3ieimpact.org
Reduced environmental risk factors Study ID
ES (95% CI)
FFS participants Pananurak, 2010 (India)
0.52 (0.32, 0.85)
Praneetvatakul & Waibel, 2006 (Thailand)
0.54 (0.39, 0.76)
Pananurak, 2010 (Pakistan)
0.55 (0.41, 0.75)
Cavatassi et al., 2011 (Ecuador)
0.82 (0.55, 1.23)
Subtotal (I-squared = 8.0%, p = 0.353)
0.59 (0.49, 0.71)
. FFS neighbours Pananurak, 2010 (India)
0.58 (0.24, 1.41)
Pananurak, 2010 (Pakistan)
0.64 (0.37, 1.10)
Cavatassi et al., 2011 (Ecuador)
0.69 (0.43, 1.11)
Praneetvatakul & Waibel, 2006 (Thailand)
1.04 (0.32, 3.40)
Subtotal (I-squared = 0.0%, p = 0.878)
0.68 (0.49, 0.93)
. NOTE: Weights are from random effects analysis
.1
.2
.5
1
2
Favours intervention
International Initiative for Impact Evaluation
www.3ieimpact.org
Summary of quantitative findings • FFS increase knowledge and improve adoption of the FFS practices • On average increasing yields and/or incomes • Suggestions of farmers feeling empowered • Limited, if any, spillovers • Neighbours do not adopt the practices consistently International Initiative for Impact Evaluation
www.3ieimpact.org
Study ID
ES (95% CI)
High risk of bias Naik et al., 2008 (India) Huan et al., 1999 (Vietnam) Labarta, 2005 (Nicaragua) Gockowski et al., 2010 (Ghana) Yang et al., 2005 (China) Hiller et al., 2009 (Kenya) Khan et al., 2007 (Pakistan) Birthal et al., 2000 (India) Dinpanah et al., 2010 (Iran) Wandji et al., 2007 (Cameroon) Mutandwa & Mpangwa, 2004 (Zimbabwe) Palis, 1998 (Philippines) Zuger 2004 (Peru) Carlberg et al., 2012 (Ghana) Van den Berg et al., 2002 (Sri Lanka) Pande et al., 2009 (Nepal) Dinpanah et al., 2010 (Iran) Orozco Cirilo et al., 2008 b) (Mexico) Subtotal (I-squared = 95.4%, p = 0.000) . Medium risk of bias Pananurak, 2010 (India) Van Rijn, 2010 (Peru) Rejesus et al, 2010 (Vietnam) Feder et al, 2004 (Indonesia) Wu Lifeng, 2010 (China) Ali & Sharif, 2011 (Pakistan) Pananurak, 2010 (China) Cavatassi et al., 2011 (Ecuador) Davis et al, 2012 (Tanzania) Pananurak, 2010 (Pakistan) Yamazaki & Resosudarmo, 2007 (Indonesia) Davis et al, 2012 (Kenya) Todo & Takahashi, 2011 (Ethiopia) Subtotal (I-squared = 81.0%, p = 0.000) . Overall (I-squared = 93.0%, p = 0.000) NOTE: Weights are from random effects analysis
0.89 (0.83, 0.96) 0.95 (0.92, 0.98) 0.97 (0.92, 1.02) 1.14 (1.03, 1.25) 1.15 (0.94, 1.41) 1.17 (0.53, 2.56) 1.17 (0.97, 1.42) 1.24 (1.13, 1.36) 1.32 (1.22, 1.42) 1.32 (1.07, 1.63) 1.36 (1.06, 1.73) 1.36 (0.97, 1.92) 1.44 (1.09, 1.92) 1.58 (1.19, 2.10) 1.68 (1.30, 2.18) 2.11 (1.25, 3.56) 2.52 (2.05, 3.11) 2.62 (2.23, 3.08) 1.35 (1.19, 1.52) 0.80 (0.61, 1.05) 0.86 (0.63, 1.18) 0.97 (0.72, 1.31) 0.98 (0.96, 1.01) 1.08 (1.03, 1.14) 1.09 (1.03, 1.15) 1.09 (1.04, 1.14) 1.22 (0.97, 1.53) 1.23 (1.00, 1.51) 1.24 (1.01, 1.54) 1.67 (1.23, 2.26) 1.81 (1.15, 2.84) 2.71 (1.11, 6.60) 1.10 (1.03, 1.17)
Sensitivity analysis: Yields by risk of bias status High risk of bias studies overestimate impacts
1.23 (1.16, 1.32)
.5
1
2 3 Favours intervention
International Initiative for Impact Evaluation
www.3ieimpact.org
www.3ieimpact.org
THANK YOU!
International Initiative for Impact Evaluation
www.3ieimpact.org
View more...
Comments