We are a sharing community. So please help us by uploading **1** new document or like us to download:

OR LIKE TO DOWNLOAD IMMEDIATELY

EVALUATING MATHEMATICS KNOWLEDGE FOR TEACHING AS AN OUTCOME OF PRE-SERVICE TEACHER EDUCATION: LESSONS FROM THE HIGH PERFORMING COUNTRIES IN THE IEA-TEDS-M STUDY Maria Teresa Tatto Michigan State University Michael Rodriguez University of Minnesota Improving Education through Accountability and Evaluation: Lessons from Around the World October 3-5, 2012 Center for American Studies Via Michelangelo Caetani 32 00186 Rome, Italy http://www.invalsi.it/invalsi/ri/improving_education/

IEA Teacher Education Study in Mathematics

Teacher Education and Development Study in Mathematics Funding provided by: – A grant from the National Science Foundation to Michigan State University REC-0514431; – The IEA – The participating countries and their institutions – Support on item development and expert judgment came form the international mathematics education community.

IEA Teacher Education Study in Mathematics

2

The Teacher Education and Development Study in Mathematics or TEDS-M • First cross-national data-based study about initial mathematics teacher education (e.g. Tatto, Senk, Rowley, & Peck, 2011) • 23.000 future teachers from 17 countries: nationally representative samples • Launched by IEA, directed by MSU, in collaboration with ACER and National RCs

Session Outline • Introduction to TEDS-M • Methods: Challenges & successes • Results

IEA Teacher Education Study in Mathematics

5

Introduction

IEA Teacher Education Study in Mathematics

6

TEDS-M Main Research Questions What are the policies that support prospective primary and secondary teachers’ achieved level of mathematics and related teaching knowledge? What learning opportunities are available to them via their teacher education programs and requirements?

level and depth of the mathematics and related teaching knowledge attained by

What is the

prospective primary and secondary teachers at the end of their preservice teacher education?

IEA Teacher Education Study in Mathematics

Representative Samples in 17 Participating Countries Botswana Canada Chile Georgia Germany Malaysia

Norway Oman Philippines Poland Russia Singapore

Spain Switzerland Chinese Taipei Thailand USA

TEDS-M Collected Data 2008-09: • 15,163 Primary Future Teachers • 9, 389 Secondary Future Teachers • In 500 institutions which included – 451 units preparing future primary teachers, and – 339 units preparing future secondary teachers

• 4837 Teacher Educators

IEA Teacher Education Study in Mathematics

Four Surveys • • • •

Teacher Education Programs Teacher Educators Primary Future Teachers Secondary Future Teachers – – – –

Background Opportunities to learn Beliefs Assessments of knowledge • Mathematics Content – University level – School level

• Mathematics Pedagogical Content IEA Teacher Education Study in Mathematics

Challenges and Successes of Building Measures Relevant to Research on Teacher Education in International Contexts

IEA Teacher Education Study in Mathematics

Validity

Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Standards for Educational & Psychological Testing (AERA, APA, NCME, 1999, p. 9).

Intended Inferences • Includes core content of OTL for teacher preparation in Mathematics • Captures core beliefs about teaching and learning mathematics • Measures are appropriate, meaningful, and useful across international contexts

TEDS-M Conceptual Framework

Sources of Validity Evidence for Opportunities to Learn in TE

• Each of the OTL indices must be analyzed for psychometric quality –Content related evidence –Internal structure of measures •Score consistency (reliability) •Measurement invariance

–Relations to other variables

Using of Rasch Scaling for Program Measures • Rasch scaling provides an efficient way to estimate trait values for individuals who have not responded to every item. • IRT places items and persons on the same scale – providing a tool for score interpretation. • Concurrent calibration across Primary and Secondary Levels; then applied to Educators.

Reporting & Interpretation • OTL Topics Studied – Original Metric: Number of Topics Studied

• OTL & Beliefs Scales – Scales are centered at 10.0 – Indicating the “midpoint” of the rating scale • Perceptions: Agree to Disagree • Frequencies: Never to Often

Trends Across Measures • All OTL and Beliefs measures with PFT vary more at the FT level, much less so at program level • For most OTL measures, less than 10% of variance occurs between programs. • Mathematics content and mathematics education pedagogy tend to be more consistent within program • Proportion of variance due to programs for beliefs measures was very small (all less than 5%)

OTL Interpretation • Consistency within programs and significant variation across programs & countries –Evidence of coherence in enacted curriculum • Significant variation within programs –Evidence of variation in a enacted curriculum

Design of the Assessment A critical part of an international study is deciding what to include in the assessment component. The assessment needs to be appropriate to the target population. For this study, the content was selected based on research on the mathematics knowledge needs of teachers. Mathematics content knowledge – MCK Mathematics pedagogical content knowledge – MPCK The mathematics content knowledge was based on the content assessed by TIMSS When trying to get comprehensive information about MCK and MPCK, the content is greater than the time available.

Design of the Assessment Content Coverage

Number and Operations Algebra and Functions Geometry and Measurement Data and Chance

Cognitive Levels Knowing Applying Reasoning

Mathematics Pedagogical Content Knowledge Mathematics Curricular Knowledge Knowledge of Planning for Mathematics Teaching and Learning Enacting Mathematics for Teaching and Learning

Design of the Assessment Three Item Types Multiple-choice Complex Multiple-choice Constructed Response

Sixty minutes were allocated for the mathematics knowledge survey. Two-thirds of items on MCK and one-third on MPCK

The Primary Assessment There was not enough time for all prospective teachers to take all items. Items were distributed over five blocks that would take 30 minutes Booklets were made up of two blocks. There were five booklets total with a spiraled design Blocks 1 and 2, 2 and 3, 3 and 4, 4 and 5, 5 and 1. Each booklet overlapped with two other booklets. This allowed all of the results from the booklets to be linked together to get results on the full set of items.

The Secondary Assessment The number of prospective teachers in the sample was much less than the number of prospective primary teachers. Because of the sample size needed to do the data analysis, only three blocks of items could be used. The knowledge survey had three booklets with two blocks each. The booklets were composed of blocks 1 and 2, 2and 3, and 3 and 1. The overlap between booklets allows the results from the booklets to be combined.

Computing the Reported Scores The item responses from the booklets were analyzed using item response theory (IRT). This allowed the results of all the booklets for each of primary and secondary to be reported on the same scale. There were four scales.

Primary MCK Primary MPCK Secondary MCK Secondary MPCK

The results are reported on a scale that has an international mean of 500 and standard deviation of 100.

Interpretation of MCK and MPCK The relative performance of groups can be determined by comparing the results to the international mean, but this does not tell much about the meaning of score points. To help give meaning to the score scale, anchor points were selected and performance at the anchor points was described.

Content experts were given items that persons at those points could answer correctly with confidence. They were also given items that persons at those points would find challenging. They created descriptions from those items. The descriptions were then cross-validated on second sets of items.

Anchor Point Description -Primary Persons at Point 1 on the Primary MCK scale were likely to correctly answer items involving basic computations with whole numbers identification of properties of operations with whole numbers reasoning about odd or even numbers.

They were generally able to

solve straightforward problems using simple fractions to achieve success at visualizing and interpreting standard two-dimensional and three-dimensional geometric figures, and solving routine problems about perimeter. understand straightforward uses of variables and equivalence of expressions solve problems involving simple equations.

Anchor Point Description Primary Persons at Point 1 tended to over-generalize have difficulty solving abstract problems and problems requiring multiple steps. They had limited knowledge of proportionality multiplicative reasoning least common multiples They had difficulty solving problems that involved coordinates problems about relations between geometric figures reasoning about multiple statements relationships among several mathematical concepts (such as understanding that there is an infinite number of rational numbers between two given numbers) Finding the area of a triangle drawn on a grid identifying an algebraic representation of three consecutive even numbers.

Example Item for Primary Anchor Point 1

Options A, B, and C answered with confidence at Anchor Point 1 Option D was challenging at Anchor Point 1

Final Report on MCK and MPCK Both Primary and Secondary had anchor point descriptions for MCK (2) and MPCK (1). Samples of items that could be done with confidence and those that were challenging are available for each anchor point. The scales based on the international average, the anchor points, the anchor point descriptions, and the sample items give meaning to the results reported on the scale.

Results

IEA Teacher Education Study in Mathematics

43

Teacher Education and Mathematics Knowledge for Teaching The results of our study are presented in detail in the TEDS-M international report (IEA – see www.iea.nl or TEDS-M website): Tatto, M. T., Schwille, J., Senk, S. L., Ingvarson, L., Rowley, G., Peck, R., Bankov, K., Rodriguez, M. & Reckase, M. (2012). Policy, Practice, and Readiness to Teach Primary and Secondary Mathematics in 17 Countries. Findings from the IEA Teacher Education and Development Study in Mathematics (TEDS-M). Amsterdam, The Netherlands. IEA Teacher Education Study in Mathematics

Future Primary Teachers MCK

IEA Teacher Education Study in Mathematics

Future Secondary Teachers MCK

IEA Teacher Education Study in Mathematics

What May Help Explain These Results?

IEA Teacher Education Study in Mathematics

What may explain these findings? Quality Assurance Mechanisms in Teacher Education in the TEDS-M Countries Entry into Teacher Education Control over Promotion Selection supply of of teaching standards teacher as attractive for entry to education career teacher students education

Accreditation of teacher education programs

Botswana Canada Chile Chinese Taipei Georgia Germany Malaysia Norway Oman (Sec) Philippines Poland Russian Fed. Singapore Spain (Prim) Switzerland Thailand USA

Key

Strong QA procedures

Entry to the teaching profession

Relative strength of QA system Moderate Moderate/High Low High Low Moderate/High Moderate Moderate/Low Low Low Moderate Moderate High Moderate/Low Moderate Low Moderate/Low

Moderately strong QA procedures

Limited QA procedures

A closer look at the countries where future teachers had the highest scores in our assessments

IEA Teacher Education Study in Mathematics

IEA Teacher Education Study in Mathematics

TIMSS, PISA & TEDS-M

IEA Teacher Education Study in Mathematics

Regressions Primary Programs

IEA Teacher Education Study in Mathematics

Regressions Secondary Programs

IEA Teacher Education Study in Mathematics

HLM Model of Future Teacher MCK and PCK, Given Future Primary Teacher Level and Program Level Characteristics

IEA Teacher Education Study in Mathematics

What may explain these findings? Individual differences among future teachers within programs The following characteristics of Future teachers are positively correlated with higher levels of knowledge: • socioeconomic status, age, gender and previous school performance: that is, wealthier, younger future teachers who reported that they did well in high school tended to perform better in the TEDS-M knowledge tests, and males do better than females (except for Russia).

What may explain these findings? Contextual and programmatic influences The following characteristics of education programs are positively correlated with higher levels of knowledge. • Opportunity to learn Primary level - Advanced level school mathematics, specifically function, probability and calculus Secondary level - University level mathematics, specifically geometry, and the opportunity to read research in teaching and learning ….. continued

What may explain these findings? Contextual and Programmatic Influences • A conceptual, problem solving and active learning orientation view of mathematics • Coherent curricular content and orientation • Rigorous selection standards • Demanding and sequential (versus repetitive) university and school mathematics curriculum • Formative evaluations at critical points in the program (written and oral) • Stringent graduation requirements

In sum… • To date research on the effects of teacher education has been underfunded – NSF and IEA support changed this, more high quality research by teacher educators is needed. • TEDS-M is a first step in the direction to better understand what makes a difference in preparing effective and knowledgeable teachers. • Some critics of teacher education believe it is possible to bypass colleges of education and prepare teachers in an easier, faster way but TEDS-M does not support that: the countries that prepare the most knowledgeable teachers rely on university-based teacher education programs.

IEA Teacher Education Study in Mathematics

Future Research FIRSTMATH, a study of novice teachers’ development of mathematical knowledge for teaching FIRSTMATH will explore: • the connections between what teachers bring with them when they begin to teach, and what is learned on the job as it concerns knowledge, skills and curricular content; and • the degree to which standards, accountability and other similar mechanisms operate to regulate the support that beginning teachers of mathematics receive during their first years of teaching. For more information on TEDS-M and FIRSTMATH consult the websites http://teds.educ.msu.edu/ and http://firstmath.educ.msu.edu/ • FIRSTMATH next meeting Cork, Ireland November 2530, 2012 IEA Teacher Education Study in Mathematics

View more...
IEA Teacher Education Study in Mathematics

Teacher Education and Development Study in Mathematics Funding provided by: – A grant from the National Science Foundation to Michigan State University REC-0514431; – The IEA – The participating countries and their institutions – Support on item development and expert judgment came form the international mathematics education community.

IEA Teacher Education Study in Mathematics

2

The Teacher Education and Development Study in Mathematics or TEDS-M • First cross-national data-based study about initial mathematics teacher education (e.g. Tatto, Senk, Rowley, & Peck, 2011) • 23.000 future teachers from 17 countries: nationally representative samples • Launched by IEA, directed by MSU, in collaboration with ACER and National RCs

Session Outline • Introduction to TEDS-M • Methods: Challenges & successes • Results

IEA Teacher Education Study in Mathematics

5

Introduction

IEA Teacher Education Study in Mathematics

6

TEDS-M Main Research Questions What are the policies that support prospective primary and secondary teachers’ achieved level of mathematics and related teaching knowledge? What learning opportunities are available to them via their teacher education programs and requirements?

level and depth of the mathematics and related teaching knowledge attained by

What is the

prospective primary and secondary teachers at the end of their preservice teacher education?

IEA Teacher Education Study in Mathematics

Representative Samples in 17 Participating Countries Botswana Canada Chile Georgia Germany Malaysia

Norway Oman Philippines Poland Russia Singapore

Spain Switzerland Chinese Taipei Thailand USA

TEDS-M Collected Data 2008-09: • 15,163 Primary Future Teachers • 9, 389 Secondary Future Teachers • In 500 institutions which included – 451 units preparing future primary teachers, and – 339 units preparing future secondary teachers

• 4837 Teacher Educators

IEA Teacher Education Study in Mathematics

Four Surveys • • • •

Teacher Education Programs Teacher Educators Primary Future Teachers Secondary Future Teachers – – – –

Background Opportunities to learn Beliefs Assessments of knowledge • Mathematics Content – University level – School level

• Mathematics Pedagogical Content IEA Teacher Education Study in Mathematics

Challenges and Successes of Building Measures Relevant to Research on Teacher Education in International Contexts

IEA Teacher Education Study in Mathematics

Validity

Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Standards for Educational & Psychological Testing (AERA, APA, NCME, 1999, p. 9).

Intended Inferences • Includes core content of OTL for teacher preparation in Mathematics • Captures core beliefs about teaching and learning mathematics • Measures are appropriate, meaningful, and useful across international contexts

TEDS-M Conceptual Framework

Sources of Validity Evidence for Opportunities to Learn in TE

• Each of the OTL indices must be analyzed for psychometric quality –Content related evidence –Internal structure of measures •Score consistency (reliability) •Measurement invariance

–Relations to other variables

Using of Rasch Scaling for Program Measures • Rasch scaling provides an efficient way to estimate trait values for individuals who have not responded to every item. • IRT places items and persons on the same scale – providing a tool for score interpretation. • Concurrent calibration across Primary and Secondary Levels; then applied to Educators.

Reporting & Interpretation • OTL Topics Studied – Original Metric: Number of Topics Studied

• OTL & Beliefs Scales – Scales are centered at 10.0 – Indicating the “midpoint” of the rating scale • Perceptions: Agree to Disagree • Frequencies: Never to Often

Trends Across Measures • All OTL and Beliefs measures with PFT vary more at the FT level, much less so at program level • For most OTL measures, less than 10% of variance occurs between programs. • Mathematics content and mathematics education pedagogy tend to be more consistent within program • Proportion of variance due to programs for beliefs measures was very small (all less than 5%)

OTL Interpretation • Consistency within programs and significant variation across programs & countries –Evidence of coherence in enacted curriculum • Significant variation within programs –Evidence of variation in a enacted curriculum

Design of the Assessment A critical part of an international study is deciding what to include in the assessment component. The assessment needs to be appropriate to the target population. For this study, the content was selected based on research on the mathematics knowledge needs of teachers. Mathematics content knowledge – MCK Mathematics pedagogical content knowledge – MPCK The mathematics content knowledge was based on the content assessed by TIMSS When trying to get comprehensive information about MCK and MPCK, the content is greater than the time available.

Design of the Assessment Content Coverage

Number and Operations Algebra and Functions Geometry and Measurement Data and Chance

Cognitive Levels Knowing Applying Reasoning

Mathematics Pedagogical Content Knowledge Mathematics Curricular Knowledge Knowledge of Planning for Mathematics Teaching and Learning Enacting Mathematics for Teaching and Learning

Design of the Assessment Three Item Types Multiple-choice Complex Multiple-choice Constructed Response

Sixty minutes were allocated for the mathematics knowledge survey. Two-thirds of items on MCK and one-third on MPCK

The Primary Assessment There was not enough time for all prospective teachers to take all items. Items were distributed over five blocks that would take 30 minutes Booklets were made up of two blocks. There were five booklets total with a spiraled design Blocks 1 and 2, 2 and 3, 3 and 4, 4 and 5, 5 and 1. Each booklet overlapped with two other booklets. This allowed all of the results from the booklets to be linked together to get results on the full set of items.

The Secondary Assessment The number of prospective teachers in the sample was much less than the number of prospective primary teachers. Because of the sample size needed to do the data analysis, only three blocks of items could be used. The knowledge survey had three booklets with two blocks each. The booklets were composed of blocks 1 and 2, 2and 3, and 3 and 1. The overlap between booklets allows the results from the booklets to be combined.

Computing the Reported Scores The item responses from the booklets were analyzed using item response theory (IRT). This allowed the results of all the booklets for each of primary and secondary to be reported on the same scale. There were four scales.

Primary MCK Primary MPCK Secondary MCK Secondary MPCK

The results are reported on a scale that has an international mean of 500 and standard deviation of 100.

Interpretation of MCK and MPCK The relative performance of groups can be determined by comparing the results to the international mean, but this does not tell much about the meaning of score points. To help give meaning to the score scale, anchor points were selected and performance at the anchor points was described.

Content experts were given items that persons at those points could answer correctly with confidence. They were also given items that persons at those points would find challenging. They created descriptions from those items. The descriptions were then cross-validated on second sets of items.

Anchor Point Description -Primary Persons at Point 1 on the Primary MCK scale were likely to correctly answer items involving basic computations with whole numbers identification of properties of operations with whole numbers reasoning about odd or even numbers.

They were generally able to

solve straightforward problems using simple fractions to achieve success at visualizing and interpreting standard two-dimensional and three-dimensional geometric figures, and solving routine problems about perimeter. understand straightforward uses of variables and equivalence of expressions solve problems involving simple equations.

Anchor Point Description Primary Persons at Point 1 tended to over-generalize have difficulty solving abstract problems and problems requiring multiple steps. They had limited knowledge of proportionality multiplicative reasoning least common multiples They had difficulty solving problems that involved coordinates problems about relations between geometric figures reasoning about multiple statements relationships among several mathematical concepts (such as understanding that there is an infinite number of rational numbers between two given numbers) Finding the area of a triangle drawn on a grid identifying an algebraic representation of three consecutive even numbers.

Example Item for Primary Anchor Point 1

Options A, B, and C answered with confidence at Anchor Point 1 Option D was challenging at Anchor Point 1

Final Report on MCK and MPCK Both Primary and Secondary had anchor point descriptions for MCK (2) and MPCK (1). Samples of items that could be done with confidence and those that were challenging are available for each anchor point. The scales based on the international average, the anchor points, the anchor point descriptions, and the sample items give meaning to the results reported on the scale.

Results

IEA Teacher Education Study in Mathematics

43

Teacher Education and Mathematics Knowledge for Teaching The results of our study are presented in detail in the TEDS-M international report (IEA – see www.iea.nl or TEDS-M website): Tatto, M. T., Schwille, J., Senk, S. L., Ingvarson, L., Rowley, G., Peck, R., Bankov, K., Rodriguez, M. & Reckase, M. (2012). Policy, Practice, and Readiness to Teach Primary and Secondary Mathematics in 17 Countries. Findings from the IEA Teacher Education and Development Study in Mathematics (TEDS-M). Amsterdam, The Netherlands. IEA Teacher Education Study in Mathematics

Future Primary Teachers MCK

IEA Teacher Education Study in Mathematics

Future Secondary Teachers MCK

IEA Teacher Education Study in Mathematics

What May Help Explain These Results?

IEA Teacher Education Study in Mathematics

What may explain these findings? Quality Assurance Mechanisms in Teacher Education in the TEDS-M Countries Entry into Teacher Education Control over Promotion Selection supply of of teaching standards teacher as attractive for entry to education career teacher students education

Accreditation of teacher education programs

Botswana Canada Chile Chinese Taipei Georgia Germany Malaysia Norway Oman (Sec) Philippines Poland Russian Fed. Singapore Spain (Prim) Switzerland Thailand USA

Key

Strong QA procedures

Entry to the teaching profession

Relative strength of QA system Moderate Moderate/High Low High Low Moderate/High Moderate Moderate/Low Low Low Moderate Moderate High Moderate/Low Moderate Low Moderate/Low

Moderately strong QA procedures

Limited QA procedures

A closer look at the countries where future teachers had the highest scores in our assessments

IEA Teacher Education Study in Mathematics

IEA Teacher Education Study in Mathematics

TIMSS, PISA & TEDS-M

IEA Teacher Education Study in Mathematics

Regressions Primary Programs

IEA Teacher Education Study in Mathematics

Regressions Secondary Programs

IEA Teacher Education Study in Mathematics

HLM Model of Future Teacher MCK and PCK, Given Future Primary Teacher Level and Program Level Characteristics

IEA Teacher Education Study in Mathematics

What may explain these findings? Individual differences among future teachers within programs The following characteristics of Future teachers are positively correlated with higher levels of knowledge: • socioeconomic status, age, gender and previous school performance: that is, wealthier, younger future teachers who reported that they did well in high school tended to perform better in the TEDS-M knowledge tests, and males do better than females (except for Russia).

What may explain these findings? Contextual and programmatic influences The following characteristics of education programs are positively correlated with higher levels of knowledge. • Opportunity to learn Primary level - Advanced level school mathematics, specifically function, probability and calculus Secondary level - University level mathematics, specifically geometry, and the opportunity to read research in teaching and learning ….. continued

What may explain these findings? Contextual and Programmatic Influences • A conceptual, problem solving and active learning orientation view of mathematics • Coherent curricular content and orientation • Rigorous selection standards • Demanding and sequential (versus repetitive) university and school mathematics curriculum • Formative evaluations at critical points in the program (written and oral) • Stringent graduation requirements

In sum… • To date research on the effects of teacher education has been underfunded – NSF and IEA support changed this, more high quality research by teacher educators is needed. • TEDS-M is a first step in the direction to better understand what makes a difference in preparing effective and knowledgeable teachers. • Some critics of teacher education believe it is possible to bypass colleges of education and prepare teachers in an easier, faster way but TEDS-M does not support that: the countries that prepare the most knowledgeable teachers rely on university-based teacher education programs.

IEA Teacher Education Study in Mathematics

Future Research FIRSTMATH, a study of novice teachers’ development of mathematical knowledge for teaching FIRSTMATH will explore: • the connections between what teachers bring with them when they begin to teach, and what is learned on the job as it concerns knowledge, skills and curricular content; and • the degree to which standards, accountability and other similar mechanisms operate to regulate the support that beginning teachers of mathematics receive during their first years of teaching. For more information on TEDS-M and FIRSTMATH consult the websites http://teds.educ.msu.edu/ and http://firstmath.educ.msu.edu/ • FIRSTMATH next meeting Cork, Ireland November 2530, 2012 IEA Teacher Education Study in Mathematics

We are a sharing community. So please help us by uploading **1** new document or like us to download:

OR LIKE TO DOWNLOAD IMMEDIATELY

Thank you for interesting in our services. We are a non-profit group that run this website to share documents. We need your help to maintenance this website.

To keep our site running, we need your help to cover our server cost (about $400/m), a small donation will help us a lot.