Critical evaluation of research information

Critical evaluation of research information Laura Jeffrey Researcher Training

Course outline • Importance of evaluation • Forms of value – Subjective • Cognitive bias

– Objective – Intersubjective

• Relationship between the forms of value

The need to evaluate information • Much training is about directing you to the right information = searching and retrieval • One of the distinctive elements of postgraduate research is that you have to be critical and reflect on what you find • What defines your evaluative criteria?

Ecology of resources • Term developed by Rosemary Luckin • Resources are interconnected and they evolve e.g. natural resources • Information resources are transformed into knowledge • Knowledge becomes a resource • Therefore prior knowledge shapes what we go on to create

Role of the researcher • In theory we can select almost any information to complete a task • In practice we filter it by selecting resources we think most appropriate • Motivation - affected by the learning we have already done

Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP

Value • Filtering process = value judgement – By researcher – Made on their behalf

• What forms of value are there and how do they work together to create information literate researcher?

Subjective form of value • Decisions we make – Is this what I want, do I need this, is it relevant?

• Privileges you as the researcher in the decision making process • If we omit it we get groupthink (Janis, 1972) or battery cognition (Blaug, 2007) • Importance therefore of asserting individual criticality

Cognitive bias • SO… subjectivity is vulnerable to bias and hunches • Concept of cognitive bias was developed in 1970s by Tversky and Kahneman • Four main groups – Social – Memory – Probability/belief – Decision making

Social biases • Ascribe positive or negative traits to self, individuals or groups • Loading values or anticipating action based on prior experience or a bias against self, individuals or groups • Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects

Memory biases • How you perceive past events • False memory, positive memory, imbalanced memory • E.g. A Photo, a Suggestion, a False Memory • Academic impact: importance of accurate record keeping and note taking

Probability and belief • To disregard or to pay too much attention to probability • Academic impact: need to treat each research finding as distinct and to judge it in its own right

Decision making biases • Influences on your decisions by own biases or those of a group • Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex

Cognitive biases • On your table, group the forty cards into four piles of ten – Social – Memory – Probability – Decision

Objective form of value • Scientific measures of validity or reliability • Exists so that subjective values don’t unduly influence work • Omit this scheme of value and we risk information (and knowledge formed from it) becoming counterknowledge (Thompson, 2008).

Intersubjective form of value • Based on the shared values of a community e.g. morals, ethics, laws, economics • Allows for discussion of scientific method as it can’t explain everything • Acceptance in a community • If we omit this then values are relativist

Three forms of value Subjective

Objective

Intersubjective

Summary of relationship of values • Awareness of benefits of all three values • Awareness of drawbacks of each individual value • Experience variation in your engagement with information • Understand when different forms of value should be prioritised or used in conjunction • Application to the research process