How do we know? - Children`s University

Evaluating progress and impact

John MacBeath University of Cambridge

A third space The aim of teaching is not produce learning but to provide the conditions for learning to take place (James Bradburne) Painting outside the frame (Ger Graus) ‘New skills’ – communication and teamwork (Megan and Taylor)

Having the tools to figure it out

When fleas are captured for the flea circus they are placed in jars and the lids are screwed on. When the fleas jumped in the jars, they would hit their heads on the lids. They still wanted to jump so they learned to jump just high enough so that they wouldn't hit their heads. The trainer then comes back and takes the fleas out of the jars and puts them in the circus. Even though the fleas now have the whole sky above them, they still do not jump past their now self-imposed limits. Even though the fleas are now free, they have made the limits truly theirs by refusing to go beyond them.

The definition of insanity

To go on doing the same thing and expecting different results

The peer and neighbourhood effect ‘You can take the child out of the neighborhood but much more difficult to take the neighbourhood out of the child’. ‘Neighbourhood, peer effect and its associated values are hard to eradicate, but for children and their families ‘locked in neighborhoods of corrosive, concentrated poverty expanding opportunities, enhancing social networks and peer groups are all essential aspect of substantive improvement in academic and social environments and their inter-relationship. (Smaker and Betey, 2011)

A world of learning Ten to eleven years olds:

• • •

Hours in school 900 Hours at home 1,277 Hours in the virtual world 1,934

(Baroness Susan Greenfield, 2010)

Inside the black box The effectiveness perspective

Attainment in

Attainment out

What knowledge is of most worth? We couldn’t find a mechanism to show we valued the things we didn’t test. That was the problem. We always valued the other things but we couldn’t find a way of showing it, that’s the problem. We need to get to a situation where there’s a way of showing how much we value dancing, music, sport and PE; how much we value how much improvement children make in the widest sense and that really gets into the public consciousness. (Estelle Morris, Education Minister)

NESTED LIVES Children and young people live nested lives, so that when classrooms do not function as we want them to, we go to work on improving them. Those classrooms are in schools, so when we decide that those schools are not performing appropriately, we go to work on improving them, as well. But those young people are also situated in families, in neighbourhoods, in peer groups who shape attitudes and aspirations often more powerfully than their parents or teachers. (David Berliner, 2005)

School as an anchor My thinking [was] that if you were really interested in development of the education of the child then you had to think of schools being a kind of anchor but an anchor orchestrating sets of experiences beyond school. (Sir Tim Brighouse, in Bangs, MacBeath & Galton, 2010)

Construction sites

internet

Peer group Home (s) neighbourhood

media Parents(s)

School Teacher(s) classroom

Extra curricular activities

Learning destinations

Four big questions  Does the Children’s University make a difference?  In what way?  How do we know?  What do we do about it?

Attendance

Achievement

Attitudes

Attendance 329 measures between 2007and 2010, comparing C.U. and non-C.U. pupils, On only 23 of these measures is there no difference (or a difference in favour of non-CU pupils).

Blackpool Canterbury Chesterfield Doncaster Havant Leicester Somerset` Sheffield Suffolk Sunderland Warwickshire

Term

No. pupils

Differ auth

Differ unauthor

Differ total

1 5 9 1 1 9 1 5 9 5 1 5 9 5 9 1 5 9 1 9 1 5 9 5 9

277 275 269 23 98 95 11 16 9 26 158 157 155 240 239 87 87 85 60 59 58 58 58 90 43

-1.5 -0.6 -0.9 -2.8 -1.5 -1.7 -3.3 -2.2 -2.5 -2.8 -1.7 -1.8 -1.5 -1.8 -1.4 -1.2 -2.9 -3.4 -2.0 -1.8 -0.2 -1.2 -1.1 -3.1 -2.6

-1.5 -1.8 -1.4 -0.5 -0.6 -0.4 1.1 -0.3 -1.6 -0.2 -1.2 -2.1 -2.9 -0.6 -0.6 -1.3 -1.0 -1.2 -0.4 -0.6 -2.0 -1.7 -2.9 -0.4 -0.6

-3.1 -2.4 -2.3 -3.3 -2.1 -2.1 -2.2 -2.5 -4.1 -3.0 -2.9 -3.9 -4.4 -2.4 -2.1 -2.5 -3.9 -4.5 -2.4 -2.4 -2.2 -2.8 -4.0 -3.4 -2.0

KS1

No.

EST

Actual

diff

2009-10 Maths level 3+

105

19%

27%

8.0

diff by school 8.4

2009-10 Maths level

105

2.60

2.69

0.10

0,10

2009-10 Writing level 3+

105

25%

30%

5.5

5.4

2009-10 Science level

105

2.56

2.64

0.07

0.06

2009-10 Reading level 3+

105

12%

9%

-3.5%

-2.3%

2009-10 English level

854

2.46

2.55

0.09

0.08

2009-10 Maths level 2+

854

89%

92%

3.1%

3.3%

2009-10 Maths level 2B+

854

71%

77%

5.3%

4.2%

2009-10 Maths level 3+

854

17%

19%

1.4%

1.5%

2009-10 Maths level

854

2.56

2.62

0.06

0.06

2009-10 Reading 2+ level

854

84%

89%

5.1%

5.3%

2009-10 Reading L2B+

854

71%

76%

5.6%

5.2%

2009-10 Writing 3+

854

23%

26%

2.8%

2.3%

2009-10 Writing L2+

854

80%

86%

6.6%

5.5%

2009-10 Writing

854

2.56

2.62

0.06

0.06

2009-10 Writing L2B+

854

58%

65%

6.8%

4.6%

2009-10 Reading L3+

854

10%

11%

0.9%

-0.2%

2010-11 Writing 3+

124

2.52

2.64

0.12

0.16

2010-11 Maths level 2++

124

91%

97%

5.9%

6.7%

KS2

436

84%

78%

-3.7%

-2.0%

436

4.58

4.52

-0.05

0.01

2009-10 Science level 5+ 436

38%

31%

-7.0

-3.4%

2009-10 Science level

436

4.68

4.60

-0.08

-0.02%

2009-10 English level

1031

4.43

4.48

0.01

0.05

2009-10 Maths 2 level progress 2009-10 Maths level4 +

1031

82%

83%

2.6%

3.2%

1031

79%

83%

3.2%

3.2%

2009-10 Maths level5 +

1031

30%

33%

2.6%

2.2%

2009-10 Maths level

1031

4.49

4.60

0.06

0.06

2009-10 English 2 levels progress 2009-10 English level

KS3

2009-10 Eng&Maths 5+

50

57%

70%

10.7%

8.6%

2009-10 Eng&Maths 6+

50

22%

28%

9.4%

9.6%

2009-10 English level progress 2009-10 English level 5

50

31%

42%

15.5%

12.1%

50

68%

78%

8.2%

8.8%

2009-10 English level 6+

50

26%

36%

13.6%

11.1%

2009-10 English level

50

5.28

5.58

0.33

0.28

2009-10 Maths 2 levels progress 2009-10 Maths level 5+

50

42%

58%

18.8%

18.6%

50

68%

76%

7.2%

3.7%

2009-10 Maths level 6+

50

37%

50%

11.6%

14.1%

2009-10 Maths level

50

5.48

5.85

0.35

0.29

2009-10

Science level 5+

50

67%

80%

12%

12.2%

2009-10

Science level 6+

50

28%

40%

16.1%

17.2%

2009-10

Science level

50

5.32

5.68

0.38

0.40

2009-10

373

64%

69%

5.3%

5.8%

373

29%

28%

373

29%

39%

11.5%

11.7%

2009-10

English & Maths Level 5+ English & Maths level 6+ English 2 levels progress English level 5+

373

72%

79%

8.7%

6.9%

2009-10

English level 6+

373

33%

37%

5.5%

4.8%

2009-10

English level

373

5.46

5.59

0.17

0.15

2009-10

373

52%

51%

0.9%

2,5%

2009-10

Maths level 2 progress Maths level 5+

373

74%

76%

3.0

2.9

2009-10

Maths level 6+

373

48%

46%

-2.4

-1.9

2009-10

Maths level

373

5.76

5.77

0.02

0.04

2009-10

Science level 5+

373

74%

78%

5.4

2.2

2009-10

Science level 6+

373

37%

44%

8.1%

5.3%

2009-10

Science level

373

5.57

5.66

0.13

0.04

2009-10 2009-10

-0.2%

0.6%

Data is for dialogue Professional accountability is based on data, not as a final judgment but as part of the toolkit for understanding current performance and formulating plans for reasonable action…. not as a static numerical accounting but as a conversation, using data to stimulate discussion, challenge ideas, rethink directions, and monitor progress, providing an ongoing image of [learning] as it changes, progresses, stalls, regroups, and moves forward again. (Earl and Katz, 2006: 13).

“Numbers are like people. Torture them enough and they will tell you anything” (Stephen Gorard)

 Case stories  Interviews  Focus groups  Observation and participation  Travellers’ tales  Partners , patrons and products

 CONNECT

What do I already know or can do?

 EXTEND

How does this extend my knowledge or skill?

 CHALLENGE

How does this challenge my knowledge or skill?

Curriculum and assessment

Beyond curriculum and Assessment smart

 What are the connections?  Who makes them?  How are they made?  How do we know?