Average Standardised Residual

January 13, 2018 | Author: Anonymous | Category: Math, Statistics And Probability, Statistics
Share Embed Donate


Short Description

Download Average Standardised Residual...

Description

Using CEM Data: Target Setting, Monitoring & Reporting Belfast, March 6th 2013 Neil Defty Business & Development Manager CEM [email protected]

Which Baseline to use?

Student 1

Student 2

Student 3

Student 3 - IPR

Key Questions for Target Setting • What type of valid and reliable predictive data should be used to set the targets? • Should students be involved as part of the process (ownership, empowerment etc.)? • Should parents be informed of the process and outcome?

Key points to consider might include: • Where has the data come from? • What (reliable and relevant) data should we use? • Enabling colleagues to trust the data: Training (staff) • Communication with parents and students • Challenging, NOT demoralising, students…. • Storage and retrieval of data • Consistency of understanding what the data means and does not mean

Value Added: The theory and Stats bits…

. Measuring Value Added – Terminology Subject X

Exam grade Outcome

120 100 80

-ve VA +ve VA

60 40

Residuals

20 BASELINE SCORE

VA

Trend Line/Regression Line

Measuring Value Added – An Example National Trend ‘Average’ Student

A* B

Result

C D

-ve (- 2 grades) Aldwulf

Beowulf

Subject A

+ve (+ 2 grades)

Cuthbert

E

Subject B

F G

U Low Ability

Average Ability

High Ability

Baseline Score

The position of the national trend line is of critical importance

Some Subjects are More Equal than Others…. A-Level A*

140

A

120

B

Grade

100

>1 grade

Photography Sociology

80

C

English Lit Psychology Maths

D

60

Physics

Latin

E

40

5

C

6

7

B

A Average GCSE

8

A*

Burning Question : What is my Value Added Score ? Better Question : Is it Important ?

Value Added Charts Pre 16

VA Score

Performance above expectation Good Practice to Share?

Performance in line with expectation Performance below expectation Problem with Teaching & Learning?

Danger of Relying on Raw Residuals Without Confidence Limits

Which Subjects Cause Most Concern?

4

3

0.0

-0.3 -0.3

-2.9 -3

Short Course Religious Studies Spanish Science Religious Studies Physics Physical Education Music Mathematics History German Geography French English Literature English Drama Design & Technology Chemistry Business Studies Biology Art & Design Additional Science Additional Applied Science

Which subjects now cause most concern ?

0.0 0.1

0.1 0.1

0.7 0.4 0.2

0.0 0.0 -0.4 -0.3 -1

0.2 0.2

0.5 1.0 0.5 1

0.8

1.1 2

0

-2

-4

Average Standardised Residual

Average Standardised Residuals by Subject

Business Studies

Religious Studies

Value Added Charts Post 16

SPC Chart VA Score

Performance above expectation Good Practice to Share? Performance in line with expectation Performance below expectation Problem with Teaching & Learning? 2000

2001

2002

2003

2004

2005 Year

2006

2007

2008

2009

2010

Subject Summary -

Subject Summary -

Current Year

3 Year Average

A2-English Literature Statistical Process Control (SPC) Chart

2008

2009 Year

2010

Student Level Residuals (SLR) Report Scatter Plot A2 – English Literature

General Underachievement?

Student Level Residuals (SLR) Report Scatter Plot A2 – English Literature

Too many U’s?

Other things to look for…

Why did these students do so badly? Why did this student do so well? How did they do in their other subjects?

Summary of Process • Examine Subject Summary • Determine ‘interesting’ (i.e. statistically significant) subjects • Look at 3 year average as well as single year if available • Look at trends in ‘Interesting Subjects’ • Examine student data – Scatter graphs • Identify students over / under achieving • Any known issues? • Don’t forget to look at over achieving subjects as well as under achieving

Baseline Choice

GCSE or Baseline Test? •

• • • •

Do students with the same GCSE score from feeder schools with differing value-added have the same ability? How can you tell if a student has underachieved at GCSE and thus can you maximise their potential? Has a student got very good GCSE scores through the school effort rather than their ability alone? Does school GCSE Value-Added limit the ability to add value at KS5? Can you add value at every Key Stage?

The Effect of Prior Value Added

Beyond Expectation

In line with Expectation

Below Expectation

+ve Value-Added

0 Value-Added

-ve Value-Added

Average GCSE = 6

Average GCSE = 6

Average GCSE = 6

Do these 3 students all have the same ability?

GCSE as Baseline

Same School - Spot the Difference ?

Test as Baseline

National or School Type Specific?

Comparison to all schools

Comparison to Independent Schools Only

Comparison to FE Colleges Only

Comparison to all schools

Questions:

→ →

How does the unit of comparison used affect the Value Added data and what implications does this have on your understanding of performance? Does this have implications for Self Evaluation?

Definitions: •

Residual – difference between the points the student attains and points attained on average by students from the CEM cohort with a similar ability



Standardised Residual – the residual adjusted to remove differences between qualification points scales and for statistical purposes



Average Standardised Residual – this is the ‘Value Added Score’ for any group of results



Subject VA – average of standardised residuals for all students’ results in the particular subject



School VA – average of standardised residuals for all students’ results in all subjects for a school / college



Confidence Limit – area of statistical uncertainty within which any variation from 0 is deemed ‘acceptable’ and outside of which could be deemed ‘important’

View more...

Comments

Copyright � 2017 NANOPDF Inc.
SUPPORT NANOPDF