An Empirical Question: How Students Learn Economics? Some Evidence from The JEE (1969 to 2011) Dr. Nürşen Albayrak Zanca Associate Professor of Economics University of the Incarnate Word Department of Economics San Antonio, Texas, 78209
[email protected] The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
OVERVIEW PROBLEM STATEMENT: Motivation & Objective EMPIRICAL EVIDENCE ON CLASSROOM
EXPERIMENTS: Do They Improve Learning? Literature from 1970s; 1980s; 1990s; and 2000s A META-ANALYSIS OF LECTURE-LESS TEACHING APPROACH (The Journal of Economic Education, 1969-2011) CONCLUDING REMARKS & COMMENTS The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation & Objective The higher education in the U.S. has become a
mass process. The U.S. Population # of Enrollment 152 million (1950) 2.3 million (1950) > 300 million (2010) >16.9 million (2005) Financially, the reliance on large classes as the major teaching device is very important to the universities. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation & Objective As a result of this mass process,
lecture classes have become larger to a point where there can be no interaction between student and teacher. In other words, students become an entirely passive subject in the learning process. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Lecture-less vs. Lecture Education specialists agree that instructors
should use active learning. An important issue in economics education is
that the predominant method of teaching style is traditional lecture method which provides little opportunity for active learning. Lecture = “One-Way Talk” = “Chalk & Talk” The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation & Objective An increasingly popular alternative to the
lecture-oriented ‘Chalk-and-Talk’ approach is ‘Classroom Experiments’ Classroom experiments engages students in active learning (away from passive learning). Some economists questioned whether the choices that instructors make about how to teach introductory economics make a difference in learning of economics. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A Wide Range of Other EXPERIENTIAL Teaching Tools duration/length of the course classroom size student characteristics (cognitive skills, gender, seating, math skills, note taking skills) absenteeism use of algebra and graph use of newspaper articles use of sports; use of case studies Use of CAI (computer assisted instruction) use of computer technology and integrating various Internet tools use of music use of literature and drama, motion picture use of humor, using the Simpsons /animated cartoons use of acceptance speeches by the Nobel Laureates in economics and, and so on.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
METHODOLOGY Most frequently, researchers have conducted controlled tests, where
the relative effectiveness of any of the above listed experiential learning style against traditional lecture was analyzed within a multiple linear regression model. The dependent variable (a change measured as the difference in post- and pre course TUCE scores) was controlled with a variety of student- and instructor-level characteristics. Often, the independent variables included experimental group vs. control group (captured with a dummy variable), cumulative GPA (representing students aptitude), academic experience (credit hours earned), business major, other student demographic characteristics, such as sex, ethnicity, age, and so on. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
METHODOLOGY
The most common technique (Multiple Linear Regression) Y = f ( S, PRE, X, ) Y S
adjusted post-TUCE examination score teaching style 1 for the Experimental section 0 for the Control (lecture) section PRE pre-TUCE examination score X a vector of student characteristics (Cumulative GPA, Academic Experience, Major, Sex, etc) error term A test of differences between means (t-test) The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Scope & Limitations of the Paper The primary aim of this paper is to provide a
literature review on classroom experiments. Articles focused on describing the actual experiments are out of the scope of this paper. Articles in which instructors describe their experiences using classroom experiments are considered. The scope of the META-ANALYSIS is restricted to literature review from the JEE (1969-2011) The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Objective of the Paper 1st : The primary purpose is to determine to what degree
classroom experiments contribute to the learning of economics. Is there any systematic evidence for classroom experiments in learning of economics? Can students learn economics with classroom experiments better than in the traditional lecture? 2nd :To provide a literature review and a meta-analysis on
classroom experiments from the JEE since its inception (Vol. 1, Fall 1969) up to date. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Classroom Experiments: Background Simply, teaching economics through the use of
experiments A pedagogy that is derived from the field of experimental economics. Chamberlin used experimental economics to discover Monopolistic Competition (1948) Vernon L. Smith won Nobel Prize in Economics for developing Experimental Economics (2002) Vernon Smith is considered by many to be the father of experimental economics. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Trends in the number of publications in experimental economics field (1948-1998)
Books
Textbooks WebPages Internet
Newsletter Classroom Experiments
Academic
journals Conference
Papers Reports Unpublished
papers
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Empirical Evidence on Classroom Experiments: Do They Improve Learning? While there are many advocates of classroom experiments [Post
(1985); Millerd & Alastair (1987); Spencer & Van Eynde (1986); Walker (1987); Friedman & Sunder (1994); Frank (1997); Gremmen & Potters (1997); Holt (1999); Carlson & Skaggs (2000); Dickie (2000); Truscott et al. (2000); Wilson & Nelson (2000); Barnett & Kriesel (2003); Jensen & Owen (2003); McGuire & Zanca (2003); Emerson & Taylor (2004), Zanca & McGuire (2005), and Dickie (2006)], there are also those who claim classroom experiments have not been empirically any superior to the chalk-and-talk method [McConnell & Lamphear (1969); Paden & Moyer (1969); Danielsen & Stauffer (1972); Luker et al. (1975); Sharp, et al. (1975); Wentford & Lewis (1975); Seigfried & Fels (1979); Frass (1982); De Young (1993); Fels (1993); Williams & Walker (1993); Cardell et al. (1996), Laury (1999); Yandell (1999); and Maxwell et al. (2005)] The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
NEGATIVE Empirical Evidence No systematic evidence exists to support the
hypothesis that students participating in experiments learn economics better than students getting lecture treatment. Undergraduate instruction in economics is largely a passive experience. Any opportunity that replaces lectures with active participation would benefit students. However, efficacy of classroom market experiments are drawn from anecdotal evidence (positive remarks made by students) and subjective analysis in supporting the effectiveness of the experimental approach. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
POSITIVE Empirical Evidence Students exposed to the classroom experiments have
higher gains in the TUCE than their traditional lectureoriented (non-experimental) cohorts. The results were statistically significant. In conclusion, indeed, using classroom experiment
does increase student learning.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Back to Question #1: Is there any systematic evidence for classroom experiments in learning of economics? Anecdotal Evidence
“Anything is better than a lecture!” “If it is good for me, it is good!” “This is what I do, and I like it! So do my Students!” Empirical Evidence (i.e, statistical significance) Some say YES, and some say NO!
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
What is a META-ANALYSIS? It is well acknowledged that it is not proper to
recommend an intervention or a treatment based on a single study. Instead, it is advisable to examine the cumulative effect of numerous studies before application. How to we learn about the cumulative effects? The goal of the meta-analysis is accumulating knowledge. Meta-analysis is the rubric used to describe quantitative methods for combining evidence across studies. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
HOW TO CONDUCT A META-ANALYSIS? STEP#1: Literature Search
The scope of this study is restricted to literature review from the JEE. The JEE frequently offer original articles on innovative teaching techniques in economics. (The JEE, Fall 1969, Volume 1, Number 1).
STEP #2: Inclusion Criteria
Lecture-less vs. Traditional Lecture
STEP #3: Describing, Classifying and Coding Research Studies STEP #4: Computation and Interpreting of Effect Sizes () The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A META-ANALYSIS OF CLASSROOM EXPERIMENTS (The JEE, 1969-2007) An effect size is defined as the strength of the relationship between
the independent variable and the dependent variable. For example, if one is comparing an intervention group to a control group, the most informative and straight forward measure of experimental effect size is the mean difference between experimental and control groups divided by pooled within-group standard deviation.
ˆ (X E X C ) / S X
P t 1 nE 1 nC
1/ 2
S nE 1S nC 1S /nE nC 2 2 P
2 E
2 C
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A META-ANALYSIS OF CLASSROOM EXPERIMENTS (The JEE, 1969-2007) How should (i.e., effect size) be
interpreted? small effect medium (moderate) effect large effect
= 0.2 = 0.5 = 0.8
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS OF THE LITERATURE REVIEW & THE META-ANALYSIS OF CLASSROOM EXPERIMENTS (The JEE, 1969-2007) The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS Economists have been active in the improvement of
their discipline. A great deal of work has been done in the improvement of learning of economics. The effect of classroom experiments on student learning has been widely investigated, but the results have been inconclusive, and often contradictory. While the empirical evidence shows some support for the classroom experiments over classical lecturing methods, there is no consensus on this matter.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS The current paper provides some cumulative evidence
on the impact of lecture-less teaching on student learning in economics. The results of the meta-analysis indicate that the average effect size for comparisons of lecture-less (experiment group) vs. traditional lecture (control group) instruction is less than 0.2. In other words, there is NO significant difference between lecture-less vs. lecture style teaching on students’ learning. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS Evidence of this paper indicated that lecture-less
vs. lecture format of teaching does not have any statistically significant difference in student learning. In other words, students do not learn economics better in experimental group vs. control group (i.e, old-traditional-boring lecture!) As an advocate of classroom experiments, the results of the meta-analysis was surprising for the author! The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS The current paper provides some support
for classroom experiments being an effective teaching tool, however, not necessarily an effective learning method. Despite the inconclusive result on learning of economics with lecture-less class, there is a pattern for student as well as instructor satisfaction with classroom experiments. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS A byproduct of classroom experiments is that it
renews instructors commitment to teaching and students participation in active learning. The vast majority of students enrolled in principles of microeconomics are not economics majors, they generally have a limited aptitude for microeconomic analysis. Such innovative, interesting techniques can enhance the recruitment of economics majors both to sustain our discipline, and to increase the analytical skills of college students. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS The reason to adopt of experiential teaching
techniques may vary from Making students active learners Attracting students into ECON classes/discipline Making class less boring, less abstract and more applicable Improve student evaluations Improving students’ learning is NOT the case! The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Lecture = “One-Way Talk” = “Chalk & Talk” Some researchers argue may be students feel more comfortable
with the traditional lecture style, where the instructor feeds them the information that they needed to know for the test. Some argue that the lecture method’s comparative advantages lie in transmitting information. Lectures are often less effective than other teaching methods in promoting independent thought, developing critical thinking skills in students, and changing student attitudes. If one wants to achieve these latter objectives, one should be aware of the limitations of one-way talk (and chalk), and modify the lectures accordingly and/or supplement them with other teaching devices. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH Extend the Meta-Analysis to
Other Journals Conference papers Other form of discussion/academic papers Submit the findings of this paper to the JEE TITLE: “Lecture vs. What? A Comparison of Outcomes in Economics Learning?”
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH Many articles on this topic appear regularly in most
mainstream economics journals. Among those journals are American Economic Review, Journal of Economic Education, Journal of Economic Inquiry, Journal of Economic Perspective, Southern Economics Journal, and American Economist, and many others. Others, including conferences sponsored by the AEA, and others (just like this one, the GCEA!) have sessions on teaching of economics where innovative papers are presented in this field. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH Overall, the level of student interest and active learning
involvement component of classroom experiments cannot be denied. There has not been enough evidence that these games are improving students’ economics understanding. Economics educators should continue to search the effect of experiential pedagogy on student learning. The author of this paper hopes that the paper has created an appetite to the reader to learn more about classroom experiments. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PERSONAL ANECDOTAL EVIDENCE (Positive remarks made by students) n= 47 students, ECON 2302 MICRO Expernomics made economics much easier. It was very efficient getting the main principle across the
class. It helped me to better understand and master the economic concepts. It helps learn better. My only suggestion is to HAVE MORE games/experiments. Other classes should adopt it, too. Everyone participated b/c it involved food and money! We tend to remember easier. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence…. On the first day of school, a teacher was glancing over the roll when she noticed a number after each student’s name, such as 154, 136, or 142. “Wow! Look at these IQs,” she said to herself. “What a terrific class!”
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence…. The teacher promptly determined to work harder with this class than with any other she ever had. Throughout the year, she came up with innovative lessons that she thought would challenge the students, because she did not want them to get bored with work that was too easy. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence…. Her plan worked! The class outperformed all the other classes that she taught in the usual way. Then, during the last quarter of the year, she discovered what those numbers after the students’ name really were: their locker numbers. The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
COMMENTS & QUESTIONS Thank You! The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
