1. Research Methods for Applied Linguistics

11 Research Methods Prof. TIAN Bing Shaanxi Normal University

I. overvie w & history 

An Introduction

IV. Methods and Testing



Traditional Thoughts of Education



Research M ethods



Foreign Language Education



Language Testing.



(Pedagogical) Lexicography

V. Learning II. Lg Description 

Language Descriptions



Language Corpora.



Stylistics.



Discourse Analysis. vs CA



Second Language Learning.



Individual Differences in Second Language Learning.



Social Influences on Language Learning.

VI. Teaching III. Cognitive & Social



Fashions in Language Teaching



Language Acquisition: L1 vs L2



Language, Thought, and Culture.



Language and Gender.



Language and Politics.



Language Teacher Education.



World Englishes.



The Practice of LSP



Bilingual Education.

M ethodology. 

Computer Assisted Language Learning

Fig. 0 A Bird’s-Eye-Vie w of Applied Linguistic Studies

1. Research Methods for Applied Linguistics: Scope, Characteristics, and Standards

  



1 The Scope of Applied Linguistics Research 2 Characteristics of Applied Linguistics Research 3 Standards for Sound Applied Linguistics Research 4 Ethical considerations

1.1 Defining applied linguistics research 

1 Definitions that listed the types of research (e.g., “An investigation of



a particular topic, or problem, through a document search and/or empirical study (the conducting of experiments) and analysis” and “Investigation through the reading of literature, experimentation and/or any other type of data gathering . . .”) 2 Definitions that listed the topics of research (e.g., “In its widest sense, to seek new ways to improve language education and intercultural communication training” and “Searching for information on how students process information, internalize data and retain it for communicative purposes.”)





3 Definitions that covered the purpose of research (e.g., “The search for information that will help practitioners (in this case, teachers) better carry out their jobs . . .” and “Systematic study of language issues and use in order to improve delivery of services to our students.”) 4 Definitions that enumerated the steps in the process of research (e.g., “Working toward truth, proving theories, trying out new approaches – and then compiling results, analyzing results and sharing with colleagues” and “Stating a hypothesis; gathering data; testing the hypothesis; relating the conclusions to issues at hand.”)



any systematic and principled inquiry in applied linguistics.

2 Characteristics of Applied Linguistics Research 

  

 

Applied linguistics research can be described from many different perspectives including at least (1) the contextual factors involved in applied linguistics research, (2) van Lier’s parameters of educational research design, (3) Grotjahn’s data collection methods, data types, and data analysis procedures, (4) other sets of research characteristics, and (5) the qual–quant continuum.

2.1 Contextual factors in applied linguistics research 



1 International and national contexts. International and national organizations and governmental bodies support a fair amount of applied linguistics research. Since they control the money, their political priorities tend to influence who will do such research and how. 2 Professional contexts. Within applied linguistics, the types of research that are popular at any given time vary; for a few years, interest in one type of research may increase at the expense of another, then interest may be rekindled for that latter type. In other words, even research can have its trends and fads.





3 Institutional contexts. Institutional contexts in applied linguistics research can refer to everything from entire school districts, to individual language programs, or even to very specific individual tutoring situations. Within these institutional contexts a number of factors can influence the type and quality of research: the size of the institution, availability of resources to support research, institutional policies and priorities, the institution’s past experiences with researchers, and even the personalities of the various administrators and teachers involved. 4 Local contexts. Local contexts refer to the specific circumstances in which the research will take place. The context may be a classroom, laboratory, private home, Internet bulletin board, or even a coffee shop. A number of factors in the local context may turn out to be important to the success or failure of a research study:

  





(a) physical context (e.g., class size, layout of the school, etc.), (b) time context (e.g., minutes per class, classes per day, etc.), (c) social context (e.g., language backgrounds, ethnic mix of the students, etc.), (d) pedagogical context (e.g., teaching methods used, preferred learning styles of the students and teachers, etc.), (e) psychological context (e.g., comfort level of participants with regard to research studies, etc.).



5 Personal contexts. Individual researchers have certain preconceptions about the aspects of applied linguistics that ought to be researched, the form that research should take, and their role in the research process. Such preconceptions and preferences arise from individual differences in abilities, personalities, motivations, priorities, training, etc. and may influence the types of research a particular individual or group will be interested in doing.

2.2 Van Lier’s parameters of educational research design 



As shown in Figure 19.4 (from van Lier, 1988) research can also be described in terms of an intervention axis (i.e., degree of intervention, from intervention to non-intervention) and a selectivity axis (i.e., selectivity of focus, from highly selective to non-selective). For example, on the intervention axis, research can take the form of a formal experimental design with a randomly assigned treatment and control groups, which would be an intervention study, or it can take the form of a series of informal classroom observations, which would be more in the direction of a non-intervention study. On the selectivity axis, research can be highly selective in focus (e.g., obligatory use of the definite article by immigrant Chinese adolescent men from Guangdong province), or non-selective (e.g., all language related behaviors observed of all participants in a population of students).











The selectivity and intervention axes also create what van Lier calls four territories: 1 controlling, in which the researcher conducts a carefully planned experiment restricted both in participants and content focus; 2 measuring, in which intervention is minimal but the data focus is highly restricted; 3 asking/doing, in which the researcher might intervene to ask participants to talk about what they are thinking; 4 watching, in which the researcher just observes with both intervention and selectivity kept to a minimum.

2.3 Grotjahn’s data collection methods, data types, and data analysis procedures 

According to Grotjahn (1987), research can be classified in terms of data collection methods (i.e., experimental vs. nonexperimental), data types (i.e., qualitative vs. quantitative), and data analysis procedures (statistical vs. interpretive).

2.4 Other sets of research characteristics 

2.4.1 Time orientation



Another way to classify research is according to time orientation, or the amount of time invested in gathering data. Along those lines, studies are sometimes classified as either cross-sectional or longitudinal (e.g., see Larsen-Freeman & Long, 1991). Cross-sectional studies are those conducted over a short period of time, often with a relatively large number of participants. For instance, a study might gather language proficiency data (using a test), motivation data (using a questionnaire), and personal information data (on the same questionnaire) from 300 students in a one-shot cross-sectional study.





In contrast, longitudinal studies are typically carried out over a relatively long period of time, often with a small number of participants. For example, a study might dedicate five years to following five students of varying backgrounds, making careful observations of their language proficiency growth, their motivation to learn languages, their personal characteristics, etc.



Other ways of describing the time orientation have recently surfaced like prolonged engagement (observations and involvement with a group of people over a long period of time), persistent observations (frequent observations over that long period of time), and the cyclical nature (data collection, analysis, interpretation, followed by further data collection, analysis, interpretation, etc.) of longitudinal studies in the exploratoryinterpretative tradition (see especially, Davis, 1995, pp. 444–5).

2.4.2 Theory generation 





Studies can also differ in terms of theory generation, which can take two forms: hypothesis forming and hypothesis testing. Hypothesis forming research may begin with some very general framing questions, but will typically have no hypotheses to start with. In such a study, the researcher will make every effort to keep an open mind and form hypotheses about what is going on only after a great many observations of various sorts. The resulting hypotheses are typically considered part of the interpretation, and the researcher often goes back to the participants to ask them if the hypotheses are reasonable (in a process called member checking). One of the great strengths often cited for qualitative research is its potential for forming new hypotheses.





In contrast, hypothesis testing research begins with a set of research questions and hypotheses. Sometimes the hypotheses are stated; more often they are implicit in the research questions. The statistical analyses in such studies are designed to formally test the probability that the hypotheses are true and typically include some form of probability statement, like p < 0.01, which indicates that there is less than a one percent probability that the observed difference (or relationship) is a chance fluctuation. One of the strengths often cited for quantitative research is its potential for hypothesis testing.

2.4.3 Variable description 





Similarly, variable description can take two forms in applied linguistics research: variable definition and variable operationalization. In variable definition research, the researcher attempts to begin with no preconceived notions of what the important variables in the study will be, or how they will be defined. As the study progresses, the process of discovering and describing variables serves to gradually define them.





In variable operationalization research, the researcher clearly outlines the variables of interest from the outset (particularly the dependent, independent, and moderator variables) and explains how each one was operationalized, that is, how each one was observed or measured and quantified. For instance the variable Japanese language proficiency might be operationalized a scores on a particular Japanese proficiency test, or the variable nationality might be operationalized as 1 for Chinese, 2 for Japanese, and 3 for Korean based on asking each of the participants what passport they hold, etc.

2.4.4 Researcher perspective 



The researcher’s perspective is sometimes described as either emic or etic. Researchers adopting the emic perspective make every effort to understand the point of view of the participants and to examine how the interpretations drawn from the research relate to those views through practices such as member checking (getting the participants’ reactions to the interpretations drawn from the research). Researchers assuming the etic perspective take an outsider’s view during the data gathering process, often attempting to be as objective as possible (for more on the emic/etic distinction, see Davis, 1995, p. 433).

2.5 The qual–quant continuum  



2.5.1 The qualitative versus quantitative dichotomy Reichardt and Cook (1979, p. 10) summarize the differences between qualitative and quantitative research as shown in Table 19.2. Notice that the column on the left in Table 19.2 is labeled “Qualitative paradigm” and that the column to the on the right is labeled “Quantitative paradigm,” with each column containing adjectives and adjectival phrases describing each “paradigm.” A number of these distinctions are uncontroversial and make eminent sense; for example, the fact that the qualitative paradigm advocates qualitative methods and the quantitative paradigm advocates quantitative methods and the fact that the qualitative paradigm is typically naturalistic where the quantitative paradigm would more accurately be characterized as controlled.







However, I disagree with other distinctions, especially those that begin with un-. “Uncontrolled observation” and “ungeneralizable” both seem to me to be unfair characterizations of qualitative research. The observations in qualitative research are often well planned and structured in their own ways, as in a well-designed interview schedule, a classroom observation checklist, or a carefully planned discourse coding scheme. “Ungrounded” appears to me to be an equally unfair characterization of quantitative research because such research is sometimes quite exploratory. I would also argue that the use of the terms subjective and objective has become outdated partly because the two terms have become highly loaded over the years (Porter, 1998) and partly because those loaded meanings do not accurately characterize the two types of research.

2.5.2 General problems with the qualitative versus quantitative dichotomy 







Such a qualitative versus quantitative approach also has a number of general problems: 1 Dichotomizing qualitative versus quantitative research leaves out altogether secondary research types like literature reviews. 2 It treats as monolithic at least seven very distinct qualitative research techniques (case study research; introspection research; discourse analysis research; interactional analysis research; classroom observation research; interviews; and questionnaires). 3 It represents as monolithic at least ten qualitative research traditions that come from a variety of other fields like anthropology and theology (see Table 19.3, adapted slightly from Lazaraton, 1995, p. 460).









4 It presents as monolithic at least six very different quantitative research techniques (interviews; questionnaires; descriptive; exploratory; quasiexperimental; and experimental). 5 It ignores the way survey research, including interviews and questionnaires, is both qualitative and quantitative. 6 It ignores the ways researchers often combine qualitative and quantitative research techniques (as shown above in the Grotjahn’s analysis in Table 19.1). 7 It confuses research methods (interpretive, survey, and statistical) and research techniques (like those listed in the second and fourth points above).

2.5.3 The qual–quant interactive continuum 

Perhaps a more constructive and accurate approach would be to view qualitative and quantitative research as a matter of degrees, a continuum, rather than a clear-cut dichotomy like the one shown in Table 19.2. As Newman and Benz (1998) put it, “All behavioral research is made up of a combination of qualitative and quantitative constructs” (p. 9). They advocate the notion of a qual–quant research continuum, as opposed to a dichotomy, and insist that the continuum be considered “interactive.”



Figure 19.5 shows how 12 research characteristics can be combined to create an almost infinite number of possible interrelationships and thus to describe a wide variety of different research types.