Chapter 8

Measuring Achievement and Aptitude: Applications for Counseling Session 7

Definitions 

Achievement tests – Provides information about what an individual has learned or acquired.  Provide

information to help individuals understand their academic strengths and limitations.



Aptitude test – Predict future performance or ability to learn  Often

come to counseling because they are trying to make a decision about their future.

Six Areas of Assessment Using Achievement  Survey

achievement batteries  Individual achievement tests – Typically cover the areas of reading, math, and spelling 

Diagnostic tests – Diagnosis learning disabilities – Assess achievement strengths and limitations

Six Areas--2 

Criterion-referenced tests – Measure knowledge or comprehension to determine if certain criterion or standard has been met



Minimum-level skills – Measure skills for promotion, entrance, or graduation – Level is established before test is administered



Subject area tests – Example: test that covered knowledge of assessment strategies in counseling

Achievement Battery TerraNova 



 



TerraNova – – – – – –

Administered to children K – 12. Combination of norm and criterion referenced Schools can select from basic battery Schools can select format Spanish version Construction involved Item Response Theory

– – – –

Norm-referenced Criterion-referenced Objective master Performance level

– – – –

National percentiles National percentile ranges Stanines Grade equivalents

– – –

Reliability r= mid .80 to mid .90 Recently published not much on construct validity yet Content validity

Information Provided

Provides a profile Scores provided

Psychometric properties



Item analysis

Discussion Would you use this test? Provide rationale? What do others think ?

Aptitude Assessments Hood & Johnson (1997) argued that counselors in a variety of fields need to be knowledgeable about the predominant scholastic aptitudes tests.  If you do not have a working knowledge of these instruments may not be viewed as credible. 

SAT 

Scholastic Assessment Test

– First adminstered in 1926 as the Scholastic Aptitude Tests – 1994 was revised and renamed SAT – 2004 added an essay—writing sample



Consists of two tests – SAT I: Reasoning test 

3 hour test that measures verbal and mathematical reasoning abilities

– SAT II: Subject tests 

Consists of single subject tests – – – –

Writing Math level 1 Biology French

Interpretation  Score

ranges between 200 and 800 on both the Verbal and Mathematical sections of SAT I – Mean is 500 – SD is 100

 ETS

(Education Testing Services) uses complex formula to equate scores on different versions.

Psychometric Properties of Aptitude Tests These tests are used to make decisions that often have a significant influence on people’s lives and, therefore, the validity of these instruments deserves analysis.  GRE 

– Combined scores have validity coefficients of .31-.37 – Undergraduate grade point averages have r = .35-.39 – Adding the GRE and Grade point results in r = .49 -.63

Discussion  Would

you use this test?

 Provide  What

rationale?

do others think?

Work Samples Assessment Another way of assessing vocational/career aptitudes  Philosophical base 

– Work performance can best be assessed by using a sample of the actual work the individual would perform

Work Sample Assessments 

Valpar Component Work Sample (VCWS) – 23 individual work samples on computer – Criterion-referenced scoring – Speed test (completed in certain amount of time_ – Norms on nondisabled and disabled workers – Comes with built in computer scoring system



SAGE system – Non computer and computer versions

Test Preparation and Performance 

Do workshops really increase scores?

– Mixed review – Depends on individuals test taking sophistication—can they learn to learn to meet exams requirement or test format (logical problem solving) – Individual who have experience in taking standardized test have a distinct advantage (Anastasi, 1981) – Manuals and – sample tests have been constructed to level the playing field and provide persons with such experiences.



Coaching

– Coaching programs do make a significant positive difference in scores (contested but not yet disproven) – Less expensive ways (travel experiences, tutoring, trips to museums – The closer the coaching material to the actual test content the greater the improvement in test scores – The more time individuals spend reviewing the material, the more likely it is that they will cover the material on the test.

Exam Results 2

1

0 92.25

106.5

108

116.25

Mean = 122.3 s.d. = 17.08 Mode = {106,5; 108; 150} Median = 118.5

117

118.5

126

129

130.5

133.5

142.5

Z= data point – mean standard deviation

150

Raw and Z-scores Raw scores 92.25 106.5 106.5 108 108 116.25 117 118.5 126 129 130.5 133.5 142.5 150 150

Mean = 122.3 s.d. = 17.08

Z-scores -1.75934498 -0.925046612 -0.925046612 -0.837225731 -0.837225731 -0.354210886 -0.310300446 -0.222479565 0.216624839 0.392266601 0.480087482 0.655729244 1.182654529 1.621758933 1.621758933