`Predictive` data

January 14, 2018 | Author: Anonymous | Category: Math, Statistics And Probability, Statistics
Share Embed Donate


Short Description

Download `Predictive` data...

Description

Peter Hendry: CEM Consultant

Using CEM Data in Practice: Feedback from CEM Secondary Assessments Glasgow Conference 13th Feb 2013

[email protected] [email protected]

Assessment for Excellence: 2 systems S1/S2 Baseline Assessment (MidYIS): A Test of Developed Ability at start of S1 or S2 •

4 sections: Vocabulary, Maths, Non-Verbal and Skills (Proof Reading, and Perceptual speed and accuracy



Computer adaptive* or paper-based



Any time Term 1 + catch ups



Value Added measures to SOSCA and to Scottish Qualifications

S2 Curriculum-based Assessments (SOSCA): A curriculum assessment of knowledge, skills and understanding at end of S2



Assesses what has been taught in the classroom



Covers, Maths, Science, Reading (English)



Computer adaptive*



Four week assessment window mid April to mid May + catch ups



Value Added measures from PIPs, InCAS and MidYIS and to Scottish Qualifications

AfE: key elements • School and students’ test performances are compared to a nationally representative sample • Assessment data from the tests provides school and pupil data to inform teaching and learning • ‘Predictive’ data indicates future potential performance to SQ, and will to National 4 and 5, to inform target setting and subsequent monitoring

• ‘Value-added data’ provides measures of relative performance at SQ (and will for National 4 and 5) and SOSCA from earlier baselines to inform self-evaluation

AfE Feedback: MidYIS and SOSCA Assessment Feedback: • Standardised score system: 50-150, mean 100, SD 15

• School data including component averages and band profiles (historical and current) • Pupil data in the form of spreadsheets and Individual Pupil Records (IPRs)

AfE Feedback: MidYIS and SOSCA ‘Predictions’ Feedback: • Currently to S2 SOSCA from S1/S2 baseline (MidYIS) and to SQ from MidYIS and SOSCA

‘Value-Added’ Feedback: • To SQ from curriculum baseline (SOSCA), and from S1/S2 baseline (MidYIS) • To S2 curriculum assessment (SOSCA) from P7 baseline (PIPS or InCAS) and/or S1/S2 baseline (MidYIS)

www.cemcentre.org

Detailed Information about the assessment. Well worth a look. Data for your school.

Bands, percentiles, standardised scores…

25% 25%

25%

C B

25%

A

D 0

50

60

70

80

90

100

110

120

130

90

95 98

140

Standardised scores Percentiles:

1

5

10

20

30 40 50 60 70

80

Mean 100, Standard Deviation of 15

99

150

SCHOOL A: S1 baseline data Year 2010/2011 YIS Year 8/S1 Test Academic

MidYIS Year 8/S1 Test Academic Year 2010/2011

es Standardised On A Nationally Representative Sample of Schools Scores Standardised On A Nationally Representative Sample of Schools

e Surname Forename Forename CEM Id JAMES JAMES COREY COREY JOHN JOHN SAMUEL SAMUEL DANIELLE DANIELLEFRAZER FRAZER

Standardised Scores

Sort Data By...

Standardised Scores

Mean Score: 100

Sort Data By...

Sort Data By...

Sort Data By...

Mean Score: 100

Standard Deviation: 15

Standard Deviation: 15

CEMForm Id

AverageAverage93.3

93.3

St. error St. error 1.2

1.2

1.2

81 D 76 D 93 C 88 D 117 98A

D 69 D 61 C 87 D 91 A C98

69 D 61 D 87 D 91 C 98 89 C

C

89

D

Form Sex

M M M M F M

Sex Cust Cust M 81 M 76 M 93 M 88 F 117 M 98

89.9 89.9

90.9 90.9

97.397.3

90.7 90.7

1.2

1.2 1.2

1.2 1.2

1.21.2

D 75 D 64 D 101 C 82 C D 107

98

75 64 101 82 107 98

D D B D B C

D 68 68 D 57 57 B 92 92 D 91 91 B 105 105 C 117

117

D D C C B A

D D C C B A

7272 65 65 89 89 88 88 109 10993 93

Band

Band

St. Score

St. Score

MidYIS Score MidYIS Score Band

Band

St. Score

St. Score

Band

Band

St. Score

St. Score

Non Verbal Non Verbal Skills Skills Band

Band

St. Score

St. Score

Band

Band

St. Score

St. Score

Vocabulary MathsMaths Vocabulary

DD D D D D D D B BC C

SCHOOL A: S1 baseline data Intake profiles: overall and test component bands

A school with a ‘completely average’ intake would have 25% of pupils within each band.

SCHOOL A: S1 Baseline MidYIS Year 8/S1 Historical Band Profiles Scores Standardised on a Nationally Representative Sample of Schools Year

Band A

Band B

Band C

Band D

2013 2012 2011 2010 2009

5 5 6 11 8

12 18 18 20 18

17 28 30 29 38

65 49 46 41 36

Historical Band Profiles: Overall Score Percentage of pupils falling into each MidYIS Band over time. (Nationally Representative Standardisation) (Band A = Top 25% of the sample, Band D = Bottom 25% of sample.) 100% 90%

80%

Percentage

70%

60%

Band D

50%

Band C

40%

Band B

30%

Band A

20% 10% 0%

2009

2010

2011

2012

2013

MidYIS Baseline Testing Year Test Section

Overall Show Band Percentage Values

Start Year

2009

End Year

2013

based Standardised Feedback 2012 Curriculum 2AfE S2 Curriculum based Standardised Feedback 2012

tandardised On AfE Curriculum based Sample 2012

Scores Standardised On AfE Curriculum based Sample 2012

SCHOOL A

Scores Standardised Standardised Scores Sort Data SortBy... Data By...

100 Mean Score: Mean Score: 100 15 Standard Deviation: Standard Deviation: 15

Surname

Forename AfE Id James Form Id Forename AfE Corey James John Corey Samuel John Danielle Samuel Frazer

Danielle Frazer

St. error Form

Sex M M M M F M

A D

119 106

D D B C A B

77 77 87 101 90 94 11987 10678

110 90

87 D 90 D 84 D 87 D 75 D 78 D D110 86 B D 90 81 D

B D

117 92

92.5

92.5

1.3

1.3

1.3

84 75 D 86 D 81 D117 D92

D D 88 D 66 D 50A 50C

88 D 66 D 78 D 50 D 78 D 50 D 50 D 105 B D 96 C

A C

105 96

B C

100 91

1.3

1.3

1.3

D D 89 D

91

89 63 D 53 D 63 66 53 B 90D D 66 C 100

D D 84 D 66 D C50 B

B C

90 100

C B

98 95

D D 100

1.3

96.9

Energy & Forces

97.6

96.9

96.1

97.6

96.9

1.3

1.3

1.3

1.3

1.3

1.3

137 107

A B

1.3

D 104 B 91 D 57 D 74 91 104 B D 111 74 D 57 D 73 D 90 A 139 A 111 121 B 7396 D C 9095

139 96

A C

121 95

Band Band

St. St. Score Score

Band

Overall OverallScience Science

Energy & Forces

Band

St. Score

St. Score

Earth & Space Band

Earth & Space St. Score

Band

Life Band

96.1

96.9

84 D 84 66 D 50 D 84 D 50 D 78 D 50 D 84 98D C 78 137 D 95 C 84 107 D

C C

St. Score

St. Score

Band

Mathematics

Living Things & theLiving Processes Thingsof & the Processes of Life

Band

Band

Overall Mathematics Overall 92.1

92.1

1.3 78 78 D 50

St. Score

Band

St. Score

Band

92.6

92.6

St. Score

Shape, Position & Movement

Science Science

Shape, Position & Movement

92.6

92.6

St. Score

Band St. Score

Information Handling

St. Score Band

Information Handling

St. Score Band

Number, Money & Measurement

Band St. Score

Number, Money & Measurement Band

Overall Reading

1.3

1.3 D D B C A B

St. Score

St. Score

Overall Reading

Band

Band

Passage Comprehension St. Score

St. Score

Comprehension

Band

Band

St. Score

Text Passage Comprehension

1.3

1.3 C D D D A D

95.9

95.9

1.3

1.3

1.3

Sex Cust 94 CustM M 71 77 C 94 M 76 77 D 71 M 81 101 D 115 F 76 D 81 M 8594

Mathematics Mathematics

97.2

97.2

1.3

St. error

St. Score

Band

98.1

98.1

94.2

115 85

Band

St. Score

94.2

Average

Average

St. Score

Band

St. Score

Speed Reading

Speed Reading Text Comprehension

ReadingReading

1.3

C D C A D D AA DC

A C

92 50

C D

92 50

C D

138 99

A C

79 138 79 99

D A D C

SCHOOL A: S2 curriculum (SOSCA) data

School no.1

School no.2

STANINES Stanine 1

2

4%

50

60

3

4

5

6

7

8

9

7% 12% 17% 20% 17% 12% 7%

70

80

90 100 110 120 Standardised Test Score

4%

130

140

150

Individual Pupil Record Date Of Birth: Sex:

Vocabulary Maths Non-Verbal Skills MidYIS Score

Student Number: Form:

22/02/99 F

Band

Stanine

Percentile

Standard Score

A

7

87

117

C

5

45

98

B

6

67

107

B

6

63

105

B

6

73

109

Nationally Standardised Scores with 95% Confidence Band 150

Nationally Standardised Scores

S1 baseline IPR

DANIELLE

140 130 120

110 100

90 80 70

60 50

Vocabulary

Maths

Non-Verbal

Skills

MidYIS Score

S2 curriculum (SOSCA) IPR Individual Pupil Record Date Of Birth: 22/2/1999 Sex: F

Danielle Student Number: Form:

AfE Band

Stanine

Percentile

AfE Standardised Score

Speed Reading Text Comprehension Passage Comprehension Overall Reading Number, Money & Measurement Information Handling Shape, Position & Movement Overall Mathematics Living Things & the Processes of Life Earth & Space

A A B A

7 8 6 7

84 90 74 87

115 119 110 117

B

6

62

105

B

5

51

100

C

4

26

90

C

5

46

98

A

9

99.3

137

A

9

99.5

139

Energy & Forces

A

8

92

121

Overall Science

A

9

99.4

138

S2 curriculum (SOSCA) IPR Individual Pupil Record Date Of Birth: 22/2/1999 Sex: F

Danielle Student Number: Form:

Predictions Feedback • Currently available to Standard Grade from AfE baselines • In the future to National 4 & 5 Qualifications from AfE baselines • Also to AfE S2 curriculum (SOSCA) from S1 and/or S2 baseline (MidYIS) • Includes Chances Graphs

How is a ‘prediction’ generated?

******************************* 2

******************************************* ‘PREDICTION’ (expected grade) GRADE

************************************* ************************************** ******************************** ***************************** 50% on or below the trend line ***************** BASELINE SCORE

3 4

GRADE

Trend line (regression line)

50% on or above the trend line *********************** 1

Grade

Data for 2012 examinations

AfE S2 Curriculum based Assessments 2012 Predictions & Chances Graphs to Standard Grade subjects based on 2012 AfE scores Predictions & Chances Graphs to Standard Grade subjects based on 2012 AfEInformation scores Please refer to the Sheet for details of

the method used to make these predictions Please refer to the Information Sheet for details of the method used toPoints make these predictions To Grades

Adjust

Standard Grade Point Predictions

M

Kieran Roland M Megan

F

Danielle

F

Frazer

M

84 75 68 100 117 92

84 66 60 89 98 95

92 85 3.8 50 54 5.9 72 63 5.3 93 93 3.3 138 120 1.5 99 95 3.2

3.1 4.2 3.9 2.8 1.8 2.7

4.3 6.6 5.9 3.6 1.5 3.5

3.3 5.4 4.8 2.7 1.0 2.6

3.8 5.6 5.1 3.3 1.7 3.2

3.8 5.6 5.1 3.3 1.7 3.2

3.3 4.7 4.3 2.9 1.7 2.8

3.0 4.3 3.9 2.6 1.4 2.5

3.4 4.8 4.4 3.0 1.8 2.9

4.0 5.9 5.4 3.5 1.9 3.4

4.2 6.4 5.8 3.6 1.7 3.5

4.2 6.0 5.5 3.7 2.1 3.6

3.7 5.2 4.8 3.3 2.0 3.2

84 75 68 100 117 92

84 66 60 89 98 95

92 85 4 3 50 54 6 4 72 63 5/6 4 93 93 3/4 3 138 120 1/2 2 99 95 3 2/3

4/5 6/7 6 3/4 1/2 3/4

3/4 4 4 3/4 3 3/4 4 4 4 3/4 5/6 5/6 5/6 4/5 4/5 5 6 6/7 6 5 5 5 5 4/5 4 4/5 5/6 6 5/6 5 2/3 3/4 3/4 3 2/3 3 3/4 3/4 3/4 3/4 1 1/2 1/2 1/2 1/2 2 2 1/2 2 2 2/3 3 3 3 2/3 3 3/4 3/4 3/4 3

Overall AfE Score

Graphic Communication

M

Corey

German

James

Geography

M

French

Frazer

English

F

Drama

Danielle

Craft & Design

F

Computing Studies

Megan

Chemistry

Kieran Roland M

Business Management

M

Biology

Corey

Cust

Art & Design

M

AfE Id

Administration

James

Form

AfE Science Score

Sex

AfE Maths Score

Forename

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

AfE Reading Score

Teacher's Adjustment (Levels)

S

Predictions: Chances Graphs

Point Prediction =1.8 Grade Prediction = 2

Chances Graphs based on Pupil’s SOSCA Test Score

50% chance of a grade 2 – the most likely single grade. 50% chance of a different grade

Not a label for life ...just another piece of information • The Chances graphs show that, from almost any baseline score, students come up with almost any grade - - -there are just different probabilities for each grade depending on the baseline score. • In working with students these graphs are more useful than a single predicted or target grade • Chances graphs show what can be achieved: – By students of similar ability – By students with lower baseline scores

Latest information as at 13 Feb 2102 • Replacing SG with Nat 4 & 5 predictions: CEM statisticians have put together a proposal. • This has been looked at by colleagues in Scottish Authorities. • The CEM Technical team have now published the new predictive data. • This data is now available on the CEM secondary+ website alongside the existing SG predictive data

The equivalence we have assumed between CfE outcomes and Standard Grade outcomes is outlined below: National Level National Level 5 A,B National Level 5 C,D National Level 4 National Level 3 National Level 2

Standard Grade Credit Level 1 Credit Level 2 General Level (3 and 4) Foundation Level (5 and 6) Standard grade level 7

For each student in each CfE subject, a predicted grade is given. The range of predicted grades is as follows Grade National Level 5 A National Level 5 B National Level 5 C National Level 5 D National Level 4 National Level 3 National Level 2

CEM Predicted Grade A B C D N4 N3 N2

Where a subject is only available at National Level 4 (e.g. Business), only grades N4, N3, N2 will be reported. For subjects only available at National Level 5 (e.g. Accounting), the full range of grades will be reported. Whilst we recognise that grades N4, N3, N2 are not available we have included them in the chances graphs to aid the teacher in supporting the student in making realistic subject choices.

AfE S2 Curriculum based Assessments 2012

New slide!!

Predictions & Chances Graphs for CfE National Qualifications based on 2012 AfE scores The grade 'predictions' and chances data presented here are simulated based on the relationship between AfE S2 Curriculum based, Standard Grade and Curriculum for Excellence levels, in the absence of any actual CfE examination data being available. They should therefore be treated with caution. The relationships will be updated once the first cohort are awarded CfE qualifications in 2014.

C

N4

B

C

C

C

C

D

C

C

B

N4

A

B

B

B

B

C

B

103 116

108

C

D

D

115

112 123

119

B

C

125

135 117

129

A

B

Art & Design

102

N4/D N4/D

Drama

D

N4

Design & Manufacture

D

D

Dance (N5)

D

99

Computing Science

D

99

Chemistry

D

104

Care

Business Management (N5)

D

95

Biology

D

Administration & It

C

Accounting (N5)

N4

Overall AfE Score

D

AfE Science Score

D

AfE Maths Score

N4

AfE Reading Score

Business (N4)

CfE Grade Predictions

N4/D N4/D N4/D N4/D N4/D N4/D

. Measuring Value-Added – Terminology Subject X

Exam grade Outcome

120 100 80

-ve VA

+ve VA

60 40 20 04

25

46

BASELINE SCORE

67

8 Raw Residual

Trend Line/Regression Line

SCHOOL A: S2 curriculum (SOSCA) data Year 8 MidYIS Score 2010/2011 Against AfE Overall Mathematics 2012 170

160 150 140

AfE Score

130 120 110

100 90 80 70 60

50 50

60

70

80

90

100

110

120

130

140

MidYIS Test Score (Standardised on a Nationally Representative Sample)

150

SCHOOL C: AfE data

SCHOOL C: AfE data

Standardised Residual Bar Chart

99.7% confidence limit 95% confidence limit

ANY VALUE IN THE INNER SHADED AREA SHOWS THE EXPECTED RESULTS i.e. AVERAGE VALUE ADDED

SCHOOL C: SQ data

View more...

Comments

Copyright � 2017 NANOPDF Inc.
SUPPORT NANOPDF