Page 1 of 12

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 1 of 12

Presentation to the Conference 'Effective Assessment at University', University of Queensland, 4-5 November 1998. © David Boud

Assessment and learning – unlearning bad habits of assessment by David Boud University of Technology, Sydney

There is probably more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education. This would not be so bad if it were not for the fact that the effects of bad practice are far more potent than they are for any aspect of teaching. Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment. Assessment acts as a mechanism to control students that is far more pervasive and insidious than most staff would be prepared to acknowledge. It appears to conceal the deficiencies of teaching as much as it does to promote learning. If, as teachers, we want to exert maximum leverage over change in higher education we must confront the ways in which assessment tends to undermine learning. (Boud 1995a) I wrote those words in 1995. Are they still true today? There has been a lot of effort put into improving university assessment practices in the years since, but I fear we still have a long way to go. If it is true that there is a lot of poor practice in assessment, then why is that so? None of us consciously want to perpetuate poor practice. We would all want to do what is best. I think perhaps we have only to look to how we learned, and how by-and-large we still learn about assessment. Most of us discovered assessment through many years of being assessed, often inappropriately, and learned through the principle of inheritance. We found ourselves implementing someone else's approach as a tutor or as a new staff member. We were so overwhelmed with all the other demands on us at the time, that we tended to accept what we found and only modified it within very limited boundaries-changed bits of subject content, not the assessment activity itself. And how did the academic whose assessment practices we modelled learn about assessment. You guessed it-from someone else in a similar situation. Even when we were faced with designing an assessment scheme for a new subject we looked around for what was acceptable practice within our context. We found lots of rules, some explicit, but many implicit. These made it clear that there were distinct limits to what we could do. We were cautious about challenging them, as we had no knowledge of

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 2 of 12

what constituted good assessment. It was very unusual for us to read about assessment, and when we did we tended to limit ourselves to practical guides about methods and techniques. Many of us have not been exposed to ideas which challenge our assessment thinking. Times have changed. There are now courses for new staff, there are workshops on assessment, and there is a growing literature on assessment specifically in the university context. Many Australian universities like my own and the University of Queensland, have new policies and guidelines for assessment. The wonderful array of innovations being presented at this conference is testimony to the changes taking place. There are clear signs that assessment is being thought through afresh. I join in celebrating these achievements, but I am not sure how wide the impact has been. I have engaged in many forms of innovation in assessment myself. In particular I have taken a great interest in the role of students taking responsibility for their own assessment and in the use of self-assessment and peer feedback. However, I don't want to focus on that today. There are many examples of new practice on display here. What I want to do is rather more risky. I want to revisit what we might call conventional assessment practice from the point of view of someone who has spent some time away from it, in other territory. I want to look at some of the habits of assessment which are commonplace and raise questions about them. My view is that a large part of what occurs in assessment in higher education is based on bad habits copied from the past and a lack of critical thinking about what we do. We are locked in to patterns of assessment which cannot be justified on any educational grounds whatsoever. Assessment is a vital part of teaching and learning. We should not undermine its positive influence through unthinking adherence to existing conventions. My aim is to get back to basics in assessment; to ask, what is assessment really about, and how can we ensure that we devote our energies to achieving this. I am not under the illusion that there ever was a golden age of assessment. There definitely was not! So, I am not arguing that we return to the past. What we need to focus on now are the questions: why are we assessing? And how can we do it effectively? Good assessment means that we must focus unerringly on our educational goals and not be distracted by apparent short-term convenience. Good assessment means thinking and rethinking very hard and clearly, it doesn't mean spending lots of time on repetitive activities as we often do now. Unless we can find assessment activities and procedures satisfying for our students and ourselves we will not be able to sustain them. Unfortunately, finding a new position on assessment involves a certain amount of disruption-in many ways existing conventions are comfortable and we have adapted our lives to them-are we prepared to undertake the journey of questioning our assumptions and working through the implications? Why should we bother? The reason for me is in the observation I made right at the start. Students can't escape the effects of bad assessment. We owe it to ourselves and our students to devote at least as much energy to ensuring that our assessment practices are worthwhile as we do to ensuring that we teach well.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 3 of 12

I'd like to start by drawing attention to some common assumptions made about assessment. You might think these assumptions state the obvious, but that is part of the problem: the obvious is not necessarily true. I'd like to suggest that they might not be valid. Following this, I'd like to outline points indicating directions for thinking about assessment. They do not provide a blueprint for what to do, but they suggest some fruitful directions for exploration.

Some common (erroneous) assumptions Assessment merely measures what a student knows; it doesn't influence or change it, except perhaps to provide motivation to study. Understanding of assessment has gone through a number of phases, and this assumption represents the earliest and the most naive. Research on assessment shows that assessment has a direct backwash effect on learning. If assessment tasks reward recall, then they will prompt students to rote learning and memorisation of facts. Similarly, if assessment tasks emphasise understanding of principles, then deeper approaches to learning can be prompted. We realise now that changes to assessment practice often have a greater influence on students' study patterns than teaching and the curriculum. If we want to influence what and how students learn, then assessment is the starting point. Assessment is about creating league tables of the brightest and the best. I am certainly not opposed to celebrating the brightest and the best, but there is a price to be paid for this. That price is the undermining of university standards and the distracting of our attention and the attention of students away from what constitutes excellent work. Through a focus on comparing students with each other, rather than with explicit standards we lose sight of what is most important in a university education. It is easy to discriminate between students, much harder to be honest about the standards to be achieved by them. We are seeing currently a widespread move in Australian higher education away from norm-referenced assessment (ie. based upon discriminating between students) towards criterionreferenced assessment (ie. based on judging whether students have met established standards). Norm-referenced assessment is now prohibited by university policy at the University of Queensland and at an increasing number of other Australian universities. The reasons for these moves are many, but one which comes to mind is that as universities become larger and larger we cannot justify a system in which more and more graduates are seen by themselves and others to have failed. We need graduates able to tackle the pressing problems of society. We need many graduates with higher level skills, the ability to

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 4 of 12

judge ones own work and who are able to operate autonomously as professionals, not just a few. Aspirations should be in terms of doing the best and most appropriate work in a pluralistic society. Criterion or standards-referenced assessment means that we can address the diverse needs of a university population and not worry about whether one person deserves 1 or 2 percentage points more than another does. This is an absurd thing to do anyway, as assessment has not and will never be that accurate. All students must be treated identically What I am referring to is the notion that all students must be exposed to exactly the same assessment tasks which are identical in all respects, for it is only through this that they can be judged equitably. After the phase of assessment thinking in which it was assumed assessment merely measured learning, technical interests dominated assessment theory and practice. There was an emphasis on finding statistical solutions to assessment problems. Reliability became celebrated above all, and the simplest way to achieve high levels of reliability was to use standardised approaches which treated everyone the same. This certainly met some limited criteria of equity, but it did not acknowledge legitimate diversity of outcome. While we may wish students in our courses to have a good grounding in basic ideas and concepts, it is not necessary for them all to possess exactly the same specific knowledge. This has been acknowledged from the earliest days in which there was a choice of question in formal examinations, but we have been through a phase in which even that was not acceptable. Today, we expect our students to learn how to learn as and when they need to, not be beholden to a standardised curriculum. This means that validity of assessment must come to take on greater importance. That is, assessment needs to reflect what is most important in educational outcomes and this may be at the expense of simple reliability. When we have multiple sources of assessment, the risk of unreliability in any one of them can be countenanced more readily than can the risk of invalidity. Equity of treatment means that all students must have equal opportunity to demonstrate the desired learning outcomes for the course. This may mean that they engage in dissimilar assessment tasks. Students are congenital cheats and will do everything in their power to do better than their peers. In this view, collaboration must be avoided at all costs. This is a hangover from the days of norm-referenced testing that blindly rewarded competition between students. That it is still an issue today betrays the extent to which we have failed to move towards a criterion-referenced view. Students will, of course, seek to maximise their advantage in doing well in their courses. But we need to recognise that in order to maximise opportunities for learning, students need to learn from their

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 5 of 12

peers. Assessment systems are intrinsically individualistic, even when they are criterion or competency based; it is the individual student's work that is normally judged. We therefore need to be careful that we are not inhibiting opportunities for peer learning when we assume that students will always cheat. We need to be judicious in not encouraging students to pass off the work of others as their own, but at the same time encourage collaboration in learning. Linked to this concern is another assumption. The unseen examination is the yardstick against which all assessment practice must be judged. This assumption has acted as the dead hand on assessment for many years. Thankfully, there are clear signs that it is in rapid decline. Many universities have policies that require multiple forms and multiple occasions of assessment in any given subject. This has been on the grounds that








a variety of forms are needed to assess the variety of learning outcomes, it is unwise to place all ones eggs in a single assessment basket and students need to be given early warning of problems they might have in meeting the requirements of the subject.

Nevertheless, examinations are too often seen as the only way of avoiding plagiarism and impersonation. This assumption has locked in an emphasis on memorisation that ill befits the information age in which we operate. The move to open book tests is a healthy one as it creates a discipline for us to ensure that we are assessing understanding rather than privileging memory work. Of course, students will cheat. But they tend to do so (a) when there is an incentive to do so, (b) when they are not likely to be discovered, (c) when assessment tasks are not tailored to student interests and (d) and when tasks are such that answers can be recycled or copied. Anonymity breeds cheating. Laziness in the construction of assessment tasks breeds cheating. When staff members know students, and students know staff members, especially when they know who is likely to be marking their work, cheating is discouraged. When assessment activities appear fresh, then cheating becomes too hard. Ironically, the moves of student organisations towards anonymity of assessment on the grounds of possible bias have created more problems than it has solved. Cheating breeds in a context of anonymity, and good quality feedback is compromised when it cannot be tailored to the recipient.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 6 of 12

Purposes of assessment Before proceeding further we should reflect on what it is that we are referring to as assessment. There are two key purposes: to certify and to prompt learning (often labelled summative and formative assessment). It used to be argued that we must always keep these two purposes separate as the characteristics of good assessment for formative and summative purposes differ fundamentally. It is true that they have fundamentally different features. Formative assessment requires the feedback of good quality rich information sufficiently detailed and focused to enable the learner to benefit from it. Summative assessment is judged in terms of the extent to which it accurately portrays what a student knows or can do or is equipped for (for example, further advanced study in the same area). However, our time and energy is limited and students often do not make fine discriminations between assessment for different purposes. For them, assessment is assessment! A given set of assessment tasks has to perform double duty. This is one of the greatest challenges we face. How do we design assessment to both provide sufficient feedback to enable a student to learn from it while at the same time accurately portraying achievement for (normally) an external audience? If we look at the time we spend on assessment as staff members, I fear it is skewed dramatically in favour of the summative. An emphasis on formative aspects is undermined by the fact that students receive information about their work at the times at which they are least likely to be able to benefit from helpful comments: once the semester has ended or once there teaching has ceased on the topic.

An agenda for assessment Having examined some common assumptions about assessment that are no longer tenable, where does that leave us? While it is not possible here to produce an authoritative list of assessment principles, the following are steps in the direction of an agenda for assessment based upon my interpretation of current research and thinking in the area. I believe that the greatest barrier to improved assessment and therefore improved learning is our own experience of being assessed and our distress about this. No matter how academically successful we have become almost all of us have had unpleasant experiences of being assessed. We have known fear, humiliation and embarrassment. And because of this assessment has become charged with a degree of emotion that does not act in our own interests or those of our students. For some academics this leads to a degree of machismo in the setting of tasks: If it hurts, it must be doing good. If it was good enough for me, then it must be good enough for my students. For other staff the emotionality of assessment leads to the opposite. Needless to say, neither of these is appropriate. If students are to show what they are capable of, we must be specially mindful of the emotions of assessment and do all we can to remove from assessment acts as many unnecessary vestiges of tension as is possible. Their importance to the careers of students makes them charged enough without us doing

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 7 of 12

anything to inflame the situation. This means that we must be mindful of







briefing students clearly and in writing about assessment tasks being explicit about criteria and how work will be judged being very clear that we are judging particular work from students not students themselves and being careful that we do not inadvertently re-create situations which students have previously found traumatic.

The latter provides a particular challenge to our imagination. We must become aware of our own experiences of assessment so that these do not intrude inappropriately. Ensuring assessment enhances learning. Too much of what we do in assessment acts directly to inhibit our best teaching intentions. Unless the overall pattern of assessment in a subject and a course as a whole acts directly to pursue the kind of learning outcomes desired, then there is a risk that these will be undermined. Assessment acts as a signal to point students to what is most important to learn. If the signal points to outcomes that do not represent what we hope students will achieve in our course, then we are misleading them. This is much more common than you might assume. No single assessment method can deliver across of broad range of educational goals. Unless there is a real diversity of strategies and unless these strategies involve interesting and challenging tasks of the kind that we expect students will perform, then we are necessarily distorting learning. Show a friend a complete set of your assessment tasks and ask him or her what they would infer about what you would regard as important for your students. It is a sobering activity. One can apply this test for oneself in anticipation of a colleague doing it for us. If we find that the most important learning outcomes do not shine through a reading of the assessment tasks, then the assessment practices in that subject are likely to be inadequate. That is the easy part. We need to do the same for the course as a whole if we are to ensure assessment contributes to learning. When we do this we are likely to find many additional problems with assessment. If we are all biased in favour of memorisation (and much research suggests this to be the case) then the cumulative effect on students is great indeed. Very often general goals such as the ones I mentioned earlier about learning how to learn and becoming an autonomous learner, fall through the gaps between subjects and never get pursued actively. Assessment should always be judged in terms of its consequences for learning. Not in terms of some naïve theory about testing teaching, or about 'knowing the basic facts'! Good assessment prompts good learning. If students are not engaging in the course in the way we are hoping for, we should look first to assessment and deal with that. I remember well when I was first appointed to the University of New South Wales many years ago. Colleagues in the Centre where I was working were undertaking a study of why students were

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 8 of 12

leaving first year lectures in a particular discipline. I will not name it for reasons that will shortly be obvious. Not only did the number of students in lectures decline rapidly throughout the semester, but students were also leaving en masse during lectures. The lecturers were hurt and dismayed. Could their lecturing be so bad? Thankfully for them, it was not. They had however managed to contrive an assessment scheme that directly resulted in the observed behaviour. How did we find out what the problem was? The sophisticated research design involved standing outside the lecture room and asking students why they were leaving early, to which they gave a direct answer! This was one very obvious consequence of assessment-most students were not exposed to most of the teaching in the course. However, this could not be detected simply through reading the list of assessment tasks. The interaction between the assessment tasks and the teaching activities needed to be examined also. A less obvious, but potentially more important issue of consequences is that unless we enable students to take responsibility for making judgements about their learning we are failing to equip them for learning in the future. At the end of the day, all that matters educationally is self-assessment. If our students become effective assessors of their own learning we have assisted them to become lifelong learners. In terms of a specific agenda for assessment and learning, nothing has a higher place than ensuring the development of students' abilities to self assess. I don't want to get into more detail on this matter here, but I'd be pleased to take this up in discussion. Assessment should not be distracted by the technicalities of grading The more we use numbers and make fine-grain numerical distinctions, the more we lose sight of our educational purposes and obscure what learning is about. We have known since classic studies in the 1930s that the error in the marking of essay-type tests is such that it is meaningless to report percentage marks, or in a manner which has such fine distinctions. Four passing grade bands are as accurate as one can get with most of the assessments procedures available. Even that may be too fine-grain for some purposes. Let us not pretend we can be more precise. We should devote the energy we expend on making finer distinctions to making the specific discriminations more transparent. This would involve grounding them in the details of the standards underpinning the academic work we expect of students. My view is that unless we can substitute for a mark a descriptive and meaningful statement about the qualities that students have exhibited in their work we are not engaged in a meaningful communication with them. There are lots of ways to do this other than time consuming extended written comments (assignment attachment forms, item banks, etc.) and, of course, students have a lot to offer each other. Unless assessment tasks are worthwhile learning exercises in their own right we have wasted valuable teaching time. We should avoid spending time on tasks that do not contribute to student learning. The marking of terminal assessments, unlike the setting of such assignments, rarely contributes to students' future learning. We should find ways of minimising how much time we spend on it.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 9 of 12

Avoid communicating to students in code. Every act of assessment gives a message to students about what they should be learning and how they should go about it. The message is coded, is not easily understood and often it is read differently and with different emphases by staff and by students. The message is always interpreted in context and the cues which the context provides offer as much or more clues to students than the intentions of staff, which are rarely explicit. (Boud 1995a) Criterion-referenced assessment implies that what is important is the criterion, not a symbol detached from it. The further feedback to students gets from the specifics of the assessment tasks and the criteria used to judge them, the more obscure and unhelpful it becomes. What would you find most useful to your learning a 'C' or a statement explaining why your conclusion was not justified by your argument? The language we use in making assessment judgements betrays our attitude and positions us in a discourse of power. Statements by us that say a piece of work is 'poor' or even that it is 'very good' betray our unthinking use of authority and use of code words as a substitute for clarity of communication. (This also applies to the ticking of five point scales.) Richard Rorty has used the term 'final vocabulary' to describe words such as these which terminate discussion and Karen Moni this morning referred to these as 'weasel words'. Words like these say nothing, they block understanding and are an artefact of our power. They are a sign of disrespect. Disrespect begets disrespect, not just to us, but to what is being learned. There is much to be said about power and authority and how it is exercised in assessment and I hope that this will be considered further during the conference. Time on task is important. We learn far more from teaching a course than students ever do from taking it. How can we shift the balance of learning from us to them? Time on important tasks is essential. Students learn from their own activity, not ours. Learning involves active processing. The more opportunities you provide for it, the greater the benefit for students. Our aim must always be to encourage students to engage in activities that allow them to learn and practice those skills and abilities that are most important to their learning in the knowledge domain of the course. The more we fill up their time with other kinds of activities, the more they will be directed away from what is essential. Assessment provides a signal that points to what we believe to be important. We need to be sure our signals are not misleading. Assessment must be viewed through the eyes of students Students prepare themselves in terms of their prior experience of assessment activities of the kind they are expecting us to provide. No matter how good our assessment tasks, if students believe they will do well by memorisation, they will prepare themselves to memorise no matter what the intrinsic merits of your particular assessment.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Page 10 of 12

We must not forget that students are not homogeneous, we need many lenses through which to view their experiences. There is no substitute for direct engagement with students to discover their perceptions. Most of the questionnaires we use, while they provide much of value, are no substitutes for an understanding of how students interpret what we do. Unless we can imagine how students will respond to our assessment (and of course, our teaching) then we will never be effective in promoting good outcomes. We should remember the story of the exodus of students from the first year lectures. Finding a way of asking the right questions can save us a lot of time and energy. We shouldn't use large classes as an excuse to treat students as objects to be processed rather than individuals to be treated with respect. We may need to enrol the insights of tutors and demonstrators to tune in to the range of students' experience. It is good practice to model our concern for students' perspectives with all colleagues with whom we work. Inducting new staff into this way of looking at teaching, learning and assessment may be the most effective mentoring we can do.

Key strategies There are a number of key strategies we need to consider. These include







Maximising the use of assessment tasks that involve students in meaningful learning and adopting deep approaches to their study. learning from each other. rich, detailed, descriptive feedback.

However, the design of assessment tasks is the greatest challenge. Much can be done by fine-tuning existing assessment activities, but we must be clear about the principles we should consider. Not all of the following can be achieved in any given assessment task, but the totally of assessment tasks across subjects must adequately portray what is most important for learning in the course. Let me end on a practical note. If we are talking about criteria, what criteria should we use in designing our own assessment tasks? Criteria for the design of assessment tasks Well-designed assessment tasks: 1. are authentic and set in a realistic context (ie. oriented towards the world external to the course itself) 2. are worthwhile learning activities in their own right. (ie. each separate act of assessment can be credibly regarded as a worthwhile contribution to learning) 3. permit a holistic rather than a fragmented approach, (eg. engage students in the whole of a process rather than a particular puzzle) 4. are not repetitive for either student or assessor. Assessment-related

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

5.

6. 7. 8.

Page 11 of 12

work is a productive use of time for all those involved. (There are some limited situations in which practice, which might appear to be repetitive, can be justified.) prompt student self-assessment. (ie. the range of assessment tasks leaves students better equipped to engage in their own selfassessment now and in the future. They shift the emphasis from looking to teaching staff for judgements to students looking to themselves and the nature of the task.) are sufficiently flexible for students to tailor them to their own needs and interests are not likely to be interpreted by students in a way fundamentally different to those of the designer do not make assumptions about the subject matter or the learner which are irrelevant to the task and which are differentially perceived by different groups of students (eg. use of unnecessarily genderspecific examples, assumptions about characteristics, etc.)

Conclusion Students judge us by what we do, not what we say. If we assess badly, then students will learn badly, no matter what good we are doing in your teaching. Learning from mediocre teaching can be transformed by the choice of imaginative and engaging assessment tasks. Learning from inspirational teaching can fail to be consolidated by mediocre assessment. I said at the start that I wanted to revisit conventional assessment practice from the point of view of someone who has been away from it for a while. I've done that and while I am encouraged by some of the changes I have seen and see here today, there is still a very long way to go before we can have assessment practices which are educationally defensible and seriously contribute to the kinds of learning we most desire. I am encouraged to see that the University is taking these issues seriously.

Further reading on assessment and learning Boud, D. (1995) Enhancing Learning through Self Assessment. London: Kogan Page. Boud, D. (1995a). Assessment and learning: contradictory or complementary? In P. Knight (Ed.) Assessment for Learning in Higher Education. London: Kogan Page, 35-48. Eisner, E. W. (1993). Reshaping assessment in education: some criteria in search of practice. Journal of Curriculum Studies, 25, 3, 219-233. Marton, F., Hounsell, D. & Entwistle, N. (1996). The Experience of Learning: Implications for Teaching and Studying in Higher Education. Second Edition. Edinburgh: Scottish Academic Press. Nightingale, P., Te Wiata, I., Toohey, S., Ryan, G., Hughes, C. and Magin, D. (1996). Assessing Learning in Universities. Sydney: University of New South Wales Press. Rowntree, D. (1987). Assessing Students: How Shall We Know Them? Second Edition. London: Kogan Page.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011

TEDI conferences: Assessment and learning – unlearning bad habits of assessment

Conference Entry Page

About the Conference

Page 12 of 12

Abstracts Conference Program

TEDI Home About Assessment UQ Home

This site built on a Macintosh using Dreamweaver.. Last modified 20/1/99; 10:14:13 AM.

http://www.tedi.uq.edu.au/conferences/A_conf/papers/Boud.html

9/04/2011