Document

Curriculum Based Measurement Writing Progress Monitoring

Presenters Kathleen Begeny-Johnson & Jerry Purles June 12, 2013

What is your Professional Role?  General Education Teacher  Special Education Teacher  School/District Administrators  Coaches/Consultants/Coordinators

Objectives  Learn how to monitor a student’s writing

performance to indicate what the teacher needs to specifically instruct each student  Administer, score, and graph data from Curriculum Based Measurement – Writing (CBM-W)  Use CBM-W for data-based instructional decision making

What Is the Difference Between Traditional Assessments and Progress Monitoring (PM)? Curriculum-Based Measurement (CBM) is one

type of PM:  CBM

provides an easy and quick method to gathering student progress  Teachers can analyze student scores and adjust student goals and instructional programs  Student data can be compared to teacher’s classroom or school district data

Curriculum-Based Measurement (CBM) Overview  CBM was designed to be a set of simple, efficient

standard procedures that     

Are objective Allow for comparison of students to peers and to grade-level benchmarks Allow for repeated measurement Show student growth Serve as indicators of student performance and progress in academic areas

Scoring Categories Category

Notation Definition

Correct Letter Sequence

CLS

Total Words Written

TWW

A count of the number of words written. A word is defined as any letter or group of letters separated by a space, even if the word is misspelled or is a nonsense word.

Words Spelled Correctly

WSC

A count of the number of words that are spelled correctly. A word is spelled correctly if it can stand alone as a word in the English Language.

Correct Writing Sequence

CWS

A count of correct writing sequences found in the sample. A correct writing sequence is defined as two adjacent writing units (i.e., word-word or word-punctuation) that are acceptable within the context of what is written. Correct writing sequence takes into account correct spelling, grammar, punctuation, capitalization, syntax, and semantics.

Correct Minus Incorrect Word Sequence

CIWS

Number of incorrect word sequences subtracted from number of correct sequence

CBM –W

University of Minnesota Research Institute for Problem Solving Finding Grade 1 (2009) Measures

Scoring

Sentence Copying (3-5 min)

TWW, WSC, CWS, CIWS, CLS

Story Prompt (5 min)

TWW, WSC, CWS, CLS

Picture Word (3-5 min)

TWW, WSC, CWS, CLS

Photo Prompt (5 min)

CWS, CLS

Sample Narrative Prompt

TWW = 36

Sample Expository Prompts  Describe a game you like to play and tell why

you like it.  Describe your favorite day of the week and tell why you like it.  Think of the most valuable think you own that was not bought in a store. Why is it important to you?  Think of something you made with your own hands. How did you do it? What would you differently next time?

Sources for Prompts  Research Institute on Progress Monitoring  www.progressmonitoring.org

 Write Source 

http://wwwthewritesource.com

 NWREL 

http://educationnorthwest.org/resource/514-

How to Administer CBM - W

Select Materials  Select type of measures based on   

Grade level Purpose of progress monitoring What will provide you with the most meaningful information

 Determine the number of prompts needed  

How many weeks do you plan to monitor progress How often do you plan to monitor progress

 Create That tap students background knowledge  The end of a mid-sentence (for narrative/story starters)  A sentence that includes a “tell why” component (expository) TIP Keep materials consistent throughout the monitoring period 

Materials Needed for Written Expression CBM CBM Writing Manual with story starters and

scoring criteria Story starter appropriate for the students grade level Lined paper for student responses Stop watch Data recording sheet (optional) Graph paper or computer graphing program

General Finding within and Across Grades Grade Level

Type of Prompt

Duration

Scoring Procedures

1-2

Sentence Copying

3-5 min

WW, WSC, CWS

Picture-word

WW, WSC, CWS

Narrative

WW, WSC, CWS

3-4

Narrative

3-5 min

CWS, CIWS

5-6

Narrative

3-5 min

CWS, CIWS,

Expository

5 min

CWS, CIWS

Narrative

5-10 min

CWS, CIWS, %CWS

7-9

Expository CWS, CIWS 10-11

Narrative

7-10 min

CWS, CIWS

Selection of Prompts •

Narrative Prompts (story starters)  

Primary Narrative Prompts Grade 1-3 Intermediate Narrative Prompts Grade 4-6 (Included in the scoring manual)

The story starters should represent the

experiences and background knowledge of the students

Administering CBM Writing  Say these specific directions to the students:

“You are going to write a story. First, I will read a sentence, and then you will write a story about what happens next. You will have 1 minute to think about what you will write, and 3 minutes to write your story. Remember to do your best work. If you don't know how to spell a word, you should guess. “Are there any questions?” (Pause). “Put your pencils down and listen.” “For the next minute, think about ... (insert story

starter).”

Administering CBM Writing  After reading the story starter, begin your stop watch and allow 1

minute for students to “think.” (Monitor students so that they do not begin writing).  After 30 more seconds say: “You should be thinking about . . . .

(insert story starter)”.

Let 30 more seconds pass . . .

 At the end of the 1 minute say: “Now begin writing.” Restart your

stopwatch (set for 3 minutes).  Monitor students' participation. If individual students pause for about

10 seconds or say they are done before the test is finished, move close to them and say “Keep writing the best story you can.” This prompt can be repeated to students should they pause again.

Administering CBM Writing  After 90 seconds say: “You should be writing

about… (insert story starter).” At the end of 3 minutes say: “Stop. Put your

pencils down.” If students want to finish their story, they may do so on a separate piece of paper or continue on the same sheet after a slash mark has been inserted at the end of 3 minutes.

Administering CBM Writing Testing Considerations  Testing not teaching  Best vs. fastest  Monitor student performance  Do not answer questions

Scoring Total Words Written (TWW)  What Is A Word? 

Any letter or group of letters separated by a space is defined as a word, even if the word is misspelled or is a nonsense word.

 The total number of words written are counted

regardless of spelling or context.

 Correct spelling, word usage, capitalization,

and punctuation are ignored when calculating the number of words written.

 Underline each word written when scoring  Students can score their own TWW

Example Scoring Total Words Written (TWW)

Sentence The sky was blue The sky was blew I tuk a baf I tuka baf Iv graqz zznip

Total Words Written TWW = 4 TWW = 4 TWW = 4 TWW = 3 TWW = 3

In your manual there are specific scoring procedures to follow

______ ___ ____ ___

____ __________ ___ ______ ___ _____ ___ ____ ____ __

________ ______ ____ __ __________ ____ ___

___

______ ____ _____

______ ___

____ ___ ______

____

___

_______

___ _____ __ __

________ ______

___ _ ______ TWW = 42

Scoring Words Spelled Correctly (WSC)  Number of correctly spelled words regardless of

context  A word is counted correct if it can be found in the English language  Incorrectly spelled words are circled  WSC is calculated by subtracting the total number of circled words from the TWW  Teachers score the WSC

Scoring Words Spelled Correctly (WSC)  What is a correctly spelled word? 

A word is spelled correctly if it can stand alone as a common word in the English Language regardless of context

Sentence Bill will reed the book. And can not pake. Daz ran down the road.

WSC WSC = 5 WSC = 3 WSC = 4

Scoring WSC Found in English Language, regardless of context

______ ___ ____ ___

____ __________ ___ ______ ___ _____ ___ ____ ____ __

________ ______ ____ __ __________ ____ ___

___

______ ____ _____

______ ___

____ ___ ______

____

___

_______

___ _____ __ __

________ ______

___ _ ______ TWW = 42 WSC = 39

How to Score CWS  Correct Word Sequences (CWS): 



CWS is any two adjacent, correctly spelled words acceptable within the context of the sample to a native English speaker. The teacher considers the units of writing and their relations to one another

EX: Nobody^ could^see^ the^ trees ^ of ^ the ^ forest ^ . 

 

The two words must be syntactically and semantically correct Acceptable word sequence: “the car” Unacceptable word sequence: “car eyebrow”

How to Score CWS  Correct Word Sequences (CWS): 

Carat is placed ABOVE two words if it represents a CWS  “the ^ car”



Correct carats placed between:  Between any two correct word sequence  Between a word and the line at the beginning of a sentence  Between a word and the correct punctuation at the end of a sentence

How to Score CWS  Correct Word Sequences (CWS):  Carat is placed BELOW two words if it represents

an incorrect word sequence  “car V eyebrow” 

Incorrect carats placed between:  Any two incorrect word sequence  Between a misspelled circled word and noncircled word  Between an un-capitalized word and line at beginning of a sentence  Between an incorrect word and punctuation at end of a sentence

How to Score CWS  Correct Word Sequences (CWS):  

Teacher must read entire sample before scoring Vertical line placed where a sentence should end

^ Mary ^ asked ^ if ^ I ^ would ^ come ^ over ^. ^ I ^ said ^ no ^. CWS = 12 Judgment calls may have to be made about where sentences end: Make decision rules and stick to them when scoring. Misspelled words are circled 



How to Score CWS Since the first word is correct, it is marked as a correct writing sequence

Because the end mark is considered essential punctuation, a carat is placed between the word and the end mark to make a correct writing sequences.

^ It ^ was ^ dark ^ . ^ Nobody ^ could  seen  the ^ trees ^ of ^ the  forrast  . Misspelled words are not counted.

CWS = 10

Grammatical or syntactical errors are not counted. A down carat is place between a incorrect writing sequences. (Note: incorrect writing sequence within the same sentence).

CWS Scoring Practice Steps to Score CBM 1. Read entire sample before scoring, place vertical lines where sentence ends and circle misspelled words 2. CWS- Two adjoined words are: Spelled correctly Semantically correct Syntactically correct Correctly punctuation

One day, we were playing outside the school and … I Shrunk a Person olmost Steped on me But I Ran to fast ten David nodest me. I seid Can You help me with Everthing he sied Yes! Ov course I sead Ya!

How to Administer and Score Written Expression CBM One day, we were playing outside the school and … ˆ I Shrunk 



a v Person  olmost Steped onˆme But

 I Ran to fast  



ten David nodest  meˆ. ˆI seid

Can You  help ˆme ˆwith Everthing   he  sied

Yes ˆ! Ov course ˆIsead Ya ! CWS = 7

CIWS Scoring Procedure  Correct Minus Incorrect Word Sequence

(CIWS) The number of incorrect word sequences subtracted from the number of correct sequences. ^A ^cat  chasd  the ^mouse^ into ^ the  hoose . CWS = 5

IWS = 4

CIWS = 1

ESTIMATED TARGETS BASED ON SELECTED SCORES FROM MULTIPLE RESOURCES (Best Practices in School Psychology V; The ABCs of CBM) These scores represent averages of the selected scores. Use with caution. As we collect data, we will develop local norms Grade

Time of year

TWW

WSC

1

Fall

7-8

5

Spring

14-20

10

Fall

12-24

20

Spring

25-30

27

Fall

23-36

32

14

Spring

34-36

33

26

Fall

33-41

38

23

Spring

41-46

44

37

Fall

37-51

48

34

Spring

42-57

55

41

Fall

41-47

42

41

Spring

53-58

56

51

Fall

48-51

49

53

Spring

58

56

64

Fall

59-74

70

48

Spring

58-67

66

67

2

3

4

5

6

7

8

CWS

Example of a Baseline CBM Writing Fall 2011

100 90 80

60

Baseline (Median Data-point)

50

• Administer 3 separate CBM writing probes • Plot 3 baseline data points on the graph • Identifying the median score

40 30 20 10

The values are 15, 22, 10

0

8/ 30 /2 9/ 011 9/ 9/ 20 15 11 / 9/ 20 21 11 / 9/ 201 30 1 / 10 20 /5 11 10 /2 /1 011 4/ 20 11

TWW & WSC

70

Baseline

Date Baseline

TWW

How to Set and Graph Goals  Once baseline data have been collected (best

practice is to administer three probes and use the median score), the teacher decides on an end-ofyear performance goal for each student.  Three options for making performance goals: Published norms (AIMSweb www.aimsweb.com) http://fehb.org/CSE/CCSEConference2011/Wright/wright_P re_Conference_AIMSweb%20Norms_resource.pdf  Intra-individual framework *Tip* 

Set reasonable/attainable yet ambitious goals

Setting Goal  Student baseline performance 

Example: 24 CWS (median of 25, 24, 23)

 Desired rate of progress 

Example: 1.5 CWS per week

 Amount of the time the student will be monitored 

Example: 35 week

 Take the growth per week (1.5) X the number of

weeks (35) = 1.5 X 35 = 52.5  Add total to baseline (24 CWS) 24 CWS + 52.5 = 76.5 goal in 35 weeks

Example of a Graphed Goal CBM Writing Fall 2011

100 90 80

60

Performance Goal

Baseline (Median Data-point)

50 40 30

Goal Iine is something to compare data against

20 10 0

8/ 30 /2 9/ 011 9/ 9/ 20 15 11 / 9/ 20 21 11 / 9/ 201 30 1 / 10 20 /5 11 10 /2 /1 011 4/ 20 11

TWW & WSC

70

Baseline

Date Baseline

TWW

Using an Intra-Individual Framework to Set Goals  Intra-individual framework:  

  



Weekly rate of improvement is calculated using at least eight data points. Subtract the lowest from the highest score  EX. 12, 16, 15, 19, 16, 21, 26, 24 (26-12=14) Divide the difference by the number of weeks the data was collected 14 ÷ 8 = 1.75 This baseline rate of growth is multiplied by 1.5 (1.75 x 1.5= 2.625) This number is multiplied by the number of weeks left until the end of the year 2.625x16 weeks = 42 This number is then added to the median score of the first eight data points used to calculate the baseline growth rate  



12, 15, 16, 16, 19, 21, 24, 26 = 17.5 median score 42 + 17.5 = 59.5

This is our end of year performance goal ( 59.5 round up to 60)

Example: Using an Intra-Individual Framework to Set Goals  First eight scores:

3, 2, 5, 6, 5, 5, 7, 4.

 Difference between the lowest and highest

score 7 - 2 = 5  Divide difference by number of weeks of data points 5 ÷ 8 = .625  Multiply by 1.5 (1.5 x 0.625 = 0.9375)  Multiply by weeks left: 0.9375 × 14 = 13.125  Product is added to the median: 13.125 + 5 = 18.125.  The end-of-year performance goal is 18.

Summary  CBM-W is designed to be a simple, efficient

approach to monitoring student progress in writing  CBM-W provides useful information about whether instruction is benefiting student or whether a change is necessary

References  Assessing Writing Using Curriculum Based

Measurement Webinar presented by Dr. Erica Lembke & Dr. Kristen McMaster 2013.  Using CBM for Progress Monitoring in Written

Expression and Spelling by Todd Busch, Tracey Hall, & Erica Lembke. 2007