PPT Template Presentation: This PTT includes all slides to FACILITATE a data analysis session of PVAAS data for multiple elementary teams. This was used with 7 teams from 7 elementary schools in ONE school district.
The Power of
PROGRESS & ACHIEVEMENT Pennsylvania Value-Added Assessment System (PVAAS)
School District
PDE Data Tools PSSA Data Interactive by eMetric
Interactive tool for analyzing PSSA data by district, school, grade, subgroup and student – Reporting category information is also available
PVAAS
Value-added tool that reports growth of cohorts of students and projections for individual students for performance on future PSSAs
PA AYP
Public site that reports all AYP related data on district, school, grade and subgroup - Reporting category information is also available
4Sight Member Center
Reporting site for benchmark data by grade, class, subgroup and student – intended to be formative and used often throughout the school year
Stair-Step Expectations
Grade Three Grade 3 Test
Grade Four
Grade Five
Grade 4 Test
Grade 5 Test
In a perfect world students would start school at about the same academic level. Proficiency levels could simply be achieved by students gaining age appropriate knowledge and skills each year.
Differentiated Reality
Grade Three
Grade Four
Grade 3 Test
Grade Five
Grade 4 Test
Grade 5 Test
Students begin school at different places, and they often progress at different rates. Yet all students are expected to achieve the same level of academic proficiency each year.
The Need for Progress Measures
Grade Three
Grade Four Grade 3 Test
Grade Five Grade 4 Test
Grade 5 Test
To measure school effectiveness in this “differentiated” world, we need to pay attention not only to proficiency levels but also to how much progress students make in a given year.
PVAAS IS: • Analysis of existing assessment data – Longitudinal PSSA scores in available content areas • Math, Reading, Science, Writing
– Provides reliable estimates
• A measure of a school’s effectiveness • Critical Relationship:
Growth Data vs. Achievement Data
PVAAS: •
Is not another test
•
Is not a comparison of single test scores
•
Critical concept: Does not provide a measure of growth for an individual student
The TWO Types of PVAAS Information Looking Forward/Planning… Looking Back/Evaluation…
PVAAS Projection Reports For Individual Students and Cohorts of Students
Value-added Growth Reports For Cohorts of Students
Today NO individual measure of student growth.
PVAAS Reporting (Math, Reading, Writing, Science)
Value-added growth reports for grades 4-8 & 11 Projections to Basic, Proficient, Advanced Grade last tested - 3 4 or 5 (future PSSAs) Grade last tested - 4 5 or 6 (future PSSAs) Grade last tested - 5 6 or 7 (future PSSAs) Grade last tested - 6 7 or 8 (future PSSAs) Grade last tested - 7 8 (future PSSAs) Grade last tested - 8 11 (future PSSAs)
Student Projections Wouldn’t it be great to know the likelihood that a student will be proficient on a future PSSA?
Student Projection Methodology
A separate longitudinal modeling process focused on the performance scores of individual students. The model uses all of the available data (both math and reading) to project future PSSA performances in Math and Reading. Projections can be calculated for an individual student or groups of students.
How do I know that the projection is accurate?
The projection is precise, and is created using a model that has been reviewed and approved by four different peer review panels and the GAO (US Government Accountability Office).
The most recent 2008 growth model proposal to USDOE includes information regarding the statistical model and projection reliability study. (http://www.ed.gov/admins/lead/account/growthmodel/pa/index.html)
Recent studies have confirmed that the PVAAS projections (even as far as 3 years into the future) are more reliable at looking at the future performance of a student than the most recent PSSA score.
PVAAS Student Projection Report
PVAAS Projections for Groups of Students Student Search
Identified Students Meeting Search Requirements
Data Packets: Student Level Timing
Source
Report
Initial Meeting – Looking Back
PSSA: eMetric/GROW
Individual Student Report
Initial Meeting – Today
4Sight
Baseline
Initial Meeting – Looking Forward
PVAAS
PVAAS Projection
Periodic Meetings (based on subject area and grade level)
Examples: DIBELS 4Sight GRADE GMADE Core Phonics Local Formative Assessment
PVAAS Projections with Other Data: Elementary Scenario/Reading Student NOW in 5th Grade PSSA
Grade 4 Proficiency Range: 1255-1468
1210
4Sight Baseline
Grade 5 Proficiency Range: 1275-1496
1200
PVAAS Projections
DIBELS ORF
To Basic: To Proficient: To Advanced: Grade 5 – Low Risk 104+ at beginning of year 124+ at end of year
93.6% 64.8% 5.5% 89 Some Risk
PVAAS Projections with Other Data: High School Scenario/Math Student NOW in 9th Grade PSSA
Grade 8 Proficiency Range 1284-1445
Grade 9 Proficiency Range 4Sight 1304-1508 Baseline To Basic: PVAAS To Proficient: Projections (11th Grade)
Grade 8 Final Grades
To Advanced: Final Exam: 91% Course Grade: 87%
1450
1507 98.7% 89.9% 45.7%
What Does This Mean to You?
The district could provide you each with the probability of students in your classrooms (current grades 4-9) Grade level/course specific teams can collaborate to make sure adequate supports are in place for all students to reach success.
How could we use this information?
For students with each level of probability of proficiency, what do we need to keep doing to keep at this level? What do we need to do to increase the level of probability? • 70-100% likelihood of proficiency
– what is our action plan?
• 40-70% likelihood of proficiency
– what is our action plan? Goal date for proficiency?
• 0-40% likelihood of proficiency
– what is our action plan? Goal date for proficiency?
Value-Added Wouldn’t it be great to know if we made one year’s growth with last year’s 4th, 5th, 6th, 7th, and 8th graders? 11th graders?
Value-Added Measure of Cohort Growth
Comparison to a State Growth Standard
The Growth Standard specifies the minimal designated academic gain from grade to grade for a cohort of students. The use of a Growth Standard creates the possibility that ALL schools can demonstrate appropriate growth.
PVAAS Value-Added Growth Descriptors – Grade 4 – 8 Math and Reading The PA Growth Standard specifies the minimal acceptable academic gain from grade to grade for a cohort of students. The Growth Standard is met when the students in a school or district on average maintain their relative position when compared to the 2006 statewide distribution of student scores (2006 is the base year). Estimated mean NCE gain equal or greater than zero. There is evidence that the school has met or exceeded the Growth Standard. All schools can achieve this rating. Estimated mean NCE gain less than zero by one standard error or less. There is slight evidence that the school has not met the Growth Standard. Estimated mean NCE gain less than zero by between one and two standard errors. There is moderate evidence that the school has not met the Growth Standard. Estimated mean NCE gain more than two standard errors less than zero. There is significant evidence that the school has not met the Growth Standard.
PVAAS School Report Grades 4 through 8
Has every grade in the school met or exceeded a year’s worth of growth?
Writing, Science & Grade 11 Reading/Math Growth Descriptors Progress is defined by the district or school effect, which is a measure of the difference between observed performance and predicted performance. Progress is significantly above the progress of the average district in the state. Progress is not detectably different from the progress of the average district in the state. Progress is significantly below the progress of the average district in the state.
PVAAS School Report Grades 11- Math and Reading Grades 4, 8 & 11 – Science Grades 5, 8 & 11 – Writing
Performance Diagnostic Report
Has every predicted PSSA performance level (Below Basic, Basic, Proficient, Advanced) in each grade met or exceeded a year’s worth of growth?
Growth Standard Methodology What the whiskers tell us… Exceeded Growth Standard; More than One Year’s Growth Green
Met the Growth Standard; Made One Year’s Growth Yellow
Growth Standard (One Year’s Growth)
Below Growth Standard; Less than One Year’s Growth Rose
Predictive Methodology What the whiskers tell us… ABOVE Expectations -Observed Score Was Greater Than Predicted Score NDD – No Detectable Difference Between Observed Score and Predicted Score
Observed – Predicted (One Year’s Growth)
BELOW Expectations -Observed Score Was Less Than Predicted Score
Patterns on Performance Diagnostic Reports
A
Patterns of Growth C
B
9/13/10
What Does This Mean to You?
The district could provide each grade level/course specific team with their growth reports • Grades 4-8, and 11 • Subgroup reporting
How could we use this information?
As a grade level/course specific team, discuss why and how you may have gotten the growth results on the report? Which students made a year’s worth of growth? Which did not? Why? Why not? What is our team’s plan of action as a grade level/course specific team?
Where do we go from here? How do we plan to use our PVAAS data this next month? This school year? • Value-added reports • Diagnostic reports • Student projections How do we plan to share our PVAAS data? • With other Administrators? • With Teachers? • With Parents? • With Others?
District Goal: PVAAS will be used by teachers and administrators for annual school evaluation and planning, grade/subject level evaluation and planning, and for student level planning.
Data Packet: Annual School-Level Analysis Question
Tool
Report
1) School Met AYP?
PAAYP
Performance Chart
2) Every Grade Met AYP?
eMetric
3-Year Portrait
3) Every Grade Met Growth Standard?
PVAAS
School Value-added Report
4) Every Grade 3 Year Trend Change in Proficiency?
eMetric
3-Year Portrait
5) Every Predicted Performance Level Met Growth Standard?
PVAAS
Performance Diagnostic Report
6) Every Subgroup Met AYP?
eMetric
3-Year Portrait
7) Achievement Gap Narrower?
eMetric
3-Year Portrait
8) Every Grade Projected To Meet AYP Target?
PVAAS
Grade Projection Summary
9) School and Subgroups Met Participation Targets?
PAAYP
Data Table
10) Every Subgroup Met Growth Standard?
PVAAS
Performance Diagnostic Report
11) Subgroups Performed Similarly in Reporting Categories?
eMetric
3-Year Portrait
Data Packets: Grade Level Timing
Source
Report
Initial Meeting – Looking Back
eMetric eMetric PVAAS PVAAS eMetric/PVAAS
Performance Levels Reporting Categories Value Added Report Performance Diagnostic Report PSSA/ PVAAS Scatterplots
Initial Meeting– Looking Forward
PVAAS 4Sight Baseline 4Sight Baseline
Projection Summary Report Performance Levels Subscale Reports
Initial Meeting– Subgroup Comparisons
Same as above
Same as above
Periodic Meetings
4Sight 4Sight Local Formative Assessment
Performance Levels Subscale Reports
Data Packets: Student Level Timing
Source
Report
Initial Meeting – Looking Back
PSSA: eMetric/GROW
Individual Student Report
Initial Meeting – Today
4Sight
Baseline
Initial Meeting – Looking Forward
PVAAS
PVAAS Projection
Periodic Meetings (based on subject area and grade level)
Examples: DIBELS 4Sight GRADE GMADE Core Phonics Local Formative Assessment
Questions and Feedback
Questions: PVAAS Materials or Statewide Implementation
[email protected] 717-606-1911
PVAAS Report Web Site https://pvaas.sas.com
www.pde.state.pa.us
333 Market Street Harrisburg, PA 17126
