Microsoft PowerPoint - the NCRM EPrints Repository

Beyond test scores: the role of primary schools in improving multiple child outcomes Claire Crawford and Anna Vignoles Institute of Education, University of London

Motivation • League tables measure school performance in terms of academic achievement e.g. test scores • But Every Child Matters agenda suggests schools should have a wider remit than just increasing educational attainment – e.g. improving health, socio-emotional skills, etc

Research Questions • Are schools at the top of the league tables also the best at improving child welfare? Or is there a trade-off? • What methodological approaches are most appropriate when measuring multiple outcomes from education?

Previous research • Small (but positive) impact of schools on educational attainment once variation in pupil characteristics is taken into account – e.g. Goldstein and Sammons 1997; Reynolds et al. 1996; Leckie, 2008; Hanushek 1997

• Limited work on role of schools in promoting wider outcomes, such as well being – e.g Gibbons and Silva, 2009; Feinstein & Gutman 2008

• But different disciplines have used different methods and comparisons of results is difficult – e.g. educationalists relied on multi level models/ economists tend to use fixed effect models

Outline of today’s talk • Use simple value-added model of educational attainment to compare RE and FE models – Are assumptions underlying RE/multi level model satisfied? – Use results to inform later model development

Outline of today’s talk • Document development of cognitive and non-cognitive skills in primary school – Can we think of these relationships as causal?

• What are our next steps? – How to consider joint determination of cognitive and non-cognitive skills

RE vs. FE • Start with a model of the following form: Yist = α + βYist-1 + γXist-1 + δs + εist-1

• Where: – Y: Key Stage maths points – X: vector of pupil characteristics (including family composition, parents’ education, parents’ socio-economic status, etc)

• RE model assumes: E[Xist-1|δs] = 0

– i.e. pupil characteristics are unrelated to the school effects

• If E[Xist-1|δs] ≠ 0, RE model produces biased estimates – Rely on FE model instead – Use Hausman test to judge whether condition has been met • Future work will explore other tests as well

Determinants of KS2 maths score OLS

FE

RE

KS1 maths score

0.113** [0.003]

0.114** [0.003]

0.113** [0.003]

2nd SES quintile

-0.015 [0.027]

-0.009 [0.028]

-0.015 [0.027]

3rd SES quintile

-0.013 [0.028]

-0.010 [0.029]

-0.004 [0.028]

4th SES quintile

-0.045 [0.029]

-0.030 [0.030]

-0.037 [0.029]

Most deprived SES quintile

-0.127** [0.032]

-0.070** [0.033]

-0.104** [0.032]

Number of schools

• Hausman test rejects RE model

414

Modelling cognitive and noncognitive skills • Focus on value-added models between ages 7 and 11: KSist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1 SDQist = α + β1KSist-1 + β2SDQist-1 + γXist-1 + δs + εist-1

• KS: Key Stage test results – Standardised average of reading, writing, maths at age 7 – Standardised average of English, maths, science at age 11

• SDQ: Strengths & Difficulties score – Teacher reports – 5 questions on 5 topics: pro-social, hyperactivity, emotional symptoms, conduct problems, peer problems – Invert pro-social and combine to create total “difficulties” score – Standardise total “difficulties” score and invert so SDQ measures “strengths” rather than “difficulties”

Model controls • X is a vector of pupil characteristics, including: – Month of birth – Ethnicity – Family composition (including twin status, number of older and younger siblings, whether parents are married or cohabiting) – Parents’ education – Parents’ socio-economic status – Birthweight and whether child was breastfed – Whether parents often read to the child at age 3

• δ represent school fixed effects

Model estimation • Estimate using seemingly unrelated regression analysis – Takes account of the fact that the error terms across the two equations may be correlated – Can extend to consider more outcomes • e.g. may additionally want to consider health?

Development of cognitive and noncognitive skills No school effects

School fixed effects

Std KS2 APS Std KS1 APS

0.546** [0.010]

0.542** [0.010]

Std SDQ score (age 7)

0.097** [0.009]

0.115** [0.009]

Std SDQ score (age 11) Std KS1 APS

0.221** [0.013]

0.219** [0.014]

Std SDQ score (age 7)

0.267** [0.011]

0.304** [0.013]

Causal effects? • Instrument KS1 attainment using month of birth to identify causal effects of KS1 on KS2 and SDQ scores Std KS2 APS

Std SDQ score

Using std KS1 APS Std KS1 APS

0.542** [0.010]

0.219** [0.014]

Std SDQ score (age 7)

0.115** [0.009]

0.304** [0.013]

Using MOB as instrument for std KS1 APS Std KS1 APS (instrument using MOB)

0.559** [0.011]

0.075 [0.083]

Std SDQ score (age 7)

0.109** [0.010]

0.328** [0.018]

Summary • Tested RE model with rich set of controls – Rejected RE model, so focus on FE models instead

• Investigated development of cognitive and noncognitive skills between ages 7 and 11 – Found strong statistical relationships • Inclusion of school fixed effects marginally increases the relationship between SDQ scores at 7 and 11

– Impact of attainment at age 7 on KS2 APS appears causal; but impact of attainment at age 7 on SDQ score at age 11 is not

Next steps • Further robustness checks surrounding rejection of RE model, using other models and data • Use other statistical software to run seemingly unrelated IV regressions • Consider first difference models of KS and SDQ scores – Instrument change in SDQ score with death of a parent/close family member