4. lecture_Collecting primary data

Collecting primary (qualitative?) data

Questionnaires -.. techniques of data collection in which each person is asked to respond to the same set of questions in a predetermined + large samples + quantitative analysis - hard to produce a good questionnaire - only one possibility to ask the sample you are interested in

Questionnaire design …affects reliability, validity, response rates • Careful design of individual questions; • Clear layout of the questionnaire form; • Lucid explanation of the purpose of the questionnaire; • Pilot testing; • Carefully planned and executed administration

When to use • …for descriptive research – to identify / describe the variability of different phenomena (e.g., attitudes, opinions, practices) • …explanatory/analytical research - examine / explain relationships between variables; cause-and-effect relationships Suitable in a multiple-methods research design

Types of questionnaires • Self-administered questionnaires (internet, email, post, delivered by hand) • Interviewer-administered questionnaires (telephone; face-to face or structured interviews)

The choice of questionnaire …depends on various factors: • Characeristics of the respondents from whom you wish to collect data • Importance of reaching a particular person as respondent • Possible contamination of respondent’s answers - insufficient knowledge/experience - uninformed response - socially desirable answers - discussing answers with others - the poor recording • Size of sample required for analysis (likely response rate) • Types and number of question you need to ask

Questionnaire design requirements You should think of… - representativeness/ accurateness of sample –> generalising; comparing/relating results to earlier research - are your measurement tools compatible - good overview of the relevant literature define the theory you wish to test - understanding of research context (e.g., organisation)

Designing the questionnaire • Validity and reliability - does the questions and answers of the questionnaire make sense The question must be understood by the respondent in the way intended by the researcher and the anser given by the respondent must be understood by the researcher in the way intended by the respondent 

Assessing validity • Internal validity - the ability of your questionnaire to measure what you intend to measure • Content validity – refers to the extent to which the measurement tool provides adequate coverage of the questions you study. How to check: 1) definition of the research through the literature 2) discussion with some experts-supervisor 3) panel of individuals to assess whether each measurement question is essential or not essential • Criterion-related validity/ predictive validity – concerned with the ability of the measurement tool to make accurate predictions. How to check: - compare the data you gathered with that specified in the criterion in some way (e.g., correlation) • Construct validity – refers to the extent to which your measurement questions actually measure the presence of those constructs you intended them to measure – (attitude scales, ability/ personality tests). How to check: - how well can you generalise from your measurement questions to your construct?“

Testing for reliability …whether or not your tool will produce consistent findings at different times and under different conditions (different samples; different interviewers etc) 1) Test re-test – correlating data with those from the same questionnaire collected under as near equivalent conditions as possible 2) Internal consistency – correlating the responses to each question with those to other questions in the questionnaire (e.g., Cronbach’s alpha – we, psychologists, all love it) 3) Alternative form – comparing responses to alternative forms of the same question / groups of questions - check questions (However, respondents may get mad)

Designing individual questions • Adopt, adapt, or develop • Open questions (open-ended questions) - allow respondents to give answers in their own way • Closed questions (forced-choice questions) provide a number of alternative answers from which the respondent is instructed to choose - quicker/ easier to answer - easier to compare the responses - easier to analyse - must be clearly interpretable

Types of closed questions • List – list of items, any of which may be selected Useful: when you need to be sure that the respondent has considered all possible responses. For example: „What kind of music styles to you listen to?“ indie/ rock/ folk/ indie/ jazz/ easy listening/ indie/ world / classical/ indie

• Category – only one response can be selected from a given set of categories Useful: if you need to collect data about behaviour or attributes; categories should not overlap and should cover all possible responses! For example: „How often do you go to the cinema?“ 2 or more times a week/ once a week/ couple times a month/ never

• Ranking – the respondent is asked to place something in order. Useful: when you want to discover the relative importance of the things to the respondent. Seven / eight items maximum -> respondents rank accurately when they can remember all items For example: „Rank the factors listed in order of importance to you in your choice of field of study“ other people’s advice/ personal interest/ potential future income/ career possibilities / possibility to travel etc. • Rating – a rating device to record responses Useful: when you need to collect opinion data; Likert-type rating scales; 4, 5, 6, 7-point rating scale. For example: „I feel that my opinion is important in decision processes in the organisation“ agree/tend to agree/ tend to disagree/ disagree

Interviews • …purposeful discussion between two or more people  • Structured interviews – questionnaires based on a predetermined/ standardised / identical set of questions -> intervieweradministered questionnaires (already discussed). The social interaction should not mediate the responses, the questions should be read exactly as written and in the same tone of voice – be as computer-like as possible. Also known as quantitative research interview • Semi-structured interviews – non-standardized; list of themes/questions to be covered, partly varying from interview to interview; additional questions may be added; the order of questions may vary. Also known as qualititive research interview • In-depth interviews – informal; to explore in depth a general area of interest; no predetermined list of questions; only clear idea about the aspect(s) you want to explore. The interviewee is given the opportunity to talk freely. Also known as non-directive interview

Links to the purpose and research strategy • Semi-structured and in-depth interviews: - „what“, „how“ and „why“ questions - exploratory studies - to find out what is happening and to seek new insights - explanatory studies - semi-structured interviews -> to understand the relationships between variables • Useful: - to help identify the questions for your questionnaire - explore / explain themes that have emerged from your questionnaire - to validate findings from your questionnaire

Situations favouring non-standardised interviews • The purpose of the research – to understand the reasons for the decisions of participants; to understand the reasons for the decisions for their attitudes/ opinions. Understanding if interviewees use words/ ideas in a particular way; to lead discussion into areas you hadn’t previously considered; to formulate some new question • The significance of establishing personal contact – managers likely to agree to be interviewed rather than complete a questionnaire. Receiving the questionnaire through the post ->problematic to provide sensitive information; trust the way in which the information is used; understanding the meaning of the questions • The nature of questions – large number of questions; complex or openended questions; the order and logic of questioning may need to be varied. • Length of time required and completeness of the process – issues that are complex, unclear, or large in number; empty answers in questionnaire give no hint, why they are unanswered; by personal interview, you will form some indication of why a response could not provided -> incentive to modify the question

Data quality issues • Forms of bias – the interviewer bias – comments, tone or non-verbal behaviour of the interviewer creates bias in the way that interviewees respond to the questions being asked. Imposing interviewer’s own beliefs/ frame of reference  bias in interpreting responses

Data quality issues • Response/ interviewee bias – respondent’s sensitiveness to the unstructured exploration to certain themes –> not reveal and discuss a topic in question; providing a partial „picture“ of the situation casting him/herself in a socially desirable role • Validity and generalisability – getting the right sample - some parts of the sample remove themselves from taking part because of the lack of time -> biased sample - no generalisations can be made about the population small and unrepresentative samples

Successful interview preparation • Level of knowledge about the organisational / situational context - your prior knowledges raise your credibility • Level of information supplied to the interviewee – credibility through the supply of relevant information to participants before the interview; possibility for them to consider the information being requested and assemble supporting organisational documentation if needed • Appropriate location – comfortable; unlikely to be disturbed; quiet • The beginning of interview – shaping the credibility - explaining your research - gaining the consent/ confidence of the interviewee - supplying the information how the data is used/ how the anonymity is guaranteed

Successful interview preparation 2 • Approach to questioning – - questions be phrased clearly, understandably, with neutral tone of voice; - avoid too long questions; - avoid questions made up of two or more questions; - avoid too many theoretical concepts; - ground your questions in the real-life experiences of your participant rather than abstract concepts; - use critical incident technique - respondent is asked to describe in detail a critical incident that is key to the research question; - leave sensitive questions until near the end - greater time for the building up the trust; if the question upset the respondent, the irritation will not affect further questions :P

Successful interview preparation 3 • Interviewer’s behaviour during the interview – comments / nonverbal behaviour indicating any bias in your thinking should be avoided; - neutral, but not uninterested response; - do not look bored! - listen attentively – hold back your own thoughts, give enough time to develop the response, avoid projecting your own views. Careful listening allows yo to identify comments that may be significant to the research topic. • Test your understanding – e.g., summarize an explanation provided by the respondent – so he/she can evaluate the adequacy of your interpretation

Observation • If your research question/objective is concerned with what people do, an obvious way in which to discover this is to watch them do it. • Definition - the systematic observation, recording, description, analysis and interpretation of people’s behaviour

Participant observation Researcher… - participate fully in the lives and activities of subjects and thus becomes a member of their group, organisation or community - not only observes, but also feels, what is happening The meanings and context of the phenomena are discovered (hopefully).

Researcher roles • Complete participant – you as a researcher attempt to become a member of a group in which you conduct research. Example: studying luchtime drinking in particular work setting. The question of ethics. Threat: in gaining the trust of your „colleagues“, you might value that trust so much that you lose the sight of your research purpose • Complete observer – you would not reveal the purpose of your activity to those you are observing. You do not take part in the activities of the group. Example: studying the consumer behaviour in the supermarkets - how people tend to react to the „rush hours“

Structured observation • • • •

systematic has high level of predetermined structure quantify behaviour how often things happen rather than why they happen • …moves more and more into the Internet – „indirect observation“, traces of behaviour • personal digital assistants

Threats to validity and reliability • Subject error – observing a situation that is not representative – but you do not know it –> choose subjects who in as many respects as possible are „normal“ examples of the population under study • Time error – at different times a day and a week there may be very different processes occuring • Observer effect – the process of the observer’s observation of behaviour changes the nature of that behaviour owing to the fact that the subject is conscious of being observed. It is experimentally demonstrated that also cockroacher run faster when other cockroaches are watching.. To overcome this effect: 1) to observe in secret 2) minimal interaction – observer tries to melt into the background, has no or little interaction, avoids eye contact, sits in an unobtrusive position 3) habituation – subjects become familiar with the process of observation so they take it for granted. Several observation sessions are necessary in the same setting with the same subjects

Homework 3 • You have for now chosen your research strategy. Now, specify your data collection method (write just a short passage to extend and make more clear, how your measurement tool should look like. For example – if you wanted to use questionnaire, design some questions and explain, why you choose to ask these. If you want to interview someone, then design some topic/question and specify, would it be structured or semi/structured interview, etc)

• Email it.. Before our next lecture 