helps determine a cause and effect relationship

4.2 - Experiments

Observational Studies measures variables of interest without attempting to influence the responses. sample surveys

watching animals in nature you provide no influence on responses

just notices relationship, doesn’t imply causation

Experiment

deliberately imposing some treatment(s) on individuals to measure their responses. does the treatment cause a change in the responses? helps determine a cause and effect relationship

Lurking Variable a variable that is not the explanatory or the response variable in a study but may influence the response variable.

example: you provide data for the relationship between the number of dinners you eat with your family and your GPA. The number of dinners you eat with your family might not be the only reason for a higher GPA.

•

A lurking variable could be the interest your parents have in your education therefore your parents involvement is creating a higher GPA, not necessarily the number of dinners.

Confounding occurs when 2 variables are associated in such a way that their effects on a response variable cannot be distinguished from each other. By designing effectively, you can prevent lurking variable from becoming confounding variables. Observational studies often fail due to confounding of explanatory variables and lurking variables.

Observational or Experiment? 1.

Does reducing screen brightness increase battery life in laptop computers? To find out, researchers obtained 30 new laptops of the same brand. They chose 15 at random and adjusted their screens to the brightest setting. The other 15 laptops were left at the default setting - moderate brightness. Researchers then measured how long each machine’s battery lasted.

2.

A study of child care enrolled 1364 infants and followed them through their sixth year in school. Later, the researchers published an article in which they stated that “the more time children spent in child care from birth to age four-and-a-half, the more adults tended to rate them, both at age four-and-a-half and at kindergarten, as less likely to get along with others, as more assertive ,as disobedient, and as aggressive.”

Let’s look at the child care study some more:

• • •

What are the explanatory and response variables? Does this study show that child care causes children to be more aggressive? Explain Are their any lurking variables? Are they confounded?

Effects of binge drinking •

A common definition of “binge drinking” is 5 or more drinks at one sitting for men and 4 or more for women. An observational study finds that students who binge drink have lower average GPA than those who don’t. Identify a lurking variable that may be confounded with the effects of binge drinking. Explain how confounding might occur.

•

Could a lurking variable be “controlled” so that it is not confounded?

Parts of an Experiment 1. Explanatory Variable (aka Factors) - we can have more than one 2. Response Variable - what we are measuring as a result of the experiment 3. Treatment - the specific condition applied to individuals in an experiment. Could be 1 or many in one experiment. 4. Experimental Units - the smallest collection of individuals to which treatments are applied. (when they are humans they are called subjects) 5. Levels - when there is more than one treatment option due to multiple factors in an experiment

Identify the experimental units, the explanatory and response variables, and treatments in the following experiments.

•

A study published in the New England Journal of Medicine in March 2010 compared 2 medicines to treat head lice: an oral medication (ivermectin) and a topical lotion (malathion). Researchers studied 812 people in 376 households in 7 areas around the world. Of the 185 households randomly assigned to ivermectin, 171 were free from head lice after 2 weeks compared with only 151 of the 191 households randomly assigned to malathion.

•

Identify the experimental units, the explanatory and response variables, and treatments in the following experiments. Does adding fertilizer affect the productivity of tomato plants? How about the amount of water? To answer these questions, a gardener plants 24 similar tomato plants in identical pots in his greenhouse. He will add fertilizer to the soil in half the pots. Also, he will water 8 of the plants with 0.5 gallons of water per day, 8 of the plants with 1 gallon of water per day, and the remaining 8 plants with 1.5 gallons of water per day. At the end of three months, he will record the total weight of tomatoes produced by each plant.

A Template to Design an Experiment Explanatory Variables/Factors

Response Variable

Subjects/Experim ental units

Treatment(s)

more will be added later

This will be written out and later a design can be drawn

Ch 4.2 - Experiments Day 5

Randomized Design •

•

To help control lurking variables, an experiment needs comparisons This helps confounding from occurring

•

Examples of a Completely Randomized Design 30 students volunteer to be subjects in a caffeine experiment. On 30 identical slips of paper there are 15 A’s and 15 B’s. They are mixed in a hat and each student selects one slip of paper. Students who receive A drink the cola with caffeine and students who receive B drink the cola without caffeine. At the end of an hour we will ask the students if they still feel energized and compare the results.

A health organization wants to know if a lowcarb or a low-fat diet is more effective for longterm weight loss. The organization decides to conduct an experiment to compare these two diet plans with a control group that is only provided with a brochure about healthy eating. Ninety volunteers agree to participate in the study. Assign a number 00 to 90 to all subjects in alphabetical order by last name. Go to a line in Table D and read two-digit groups from left to right (throwing out any repeated digits or digits larger than 90). The first 30 digits will have the brochure, the next 30 digits will do the lowcarb diet, and the remaining 30 will have the low-fat diet. At the end of the year the total weight loss of each group will be compared.

Control Groups •

•

These are important to provide a baseline for comparing the effects of the treatments

aka placebo groups (more will be discussed about placebos tomorrow)

Three Principles of Experimental Design 1. Control

Control Groups Control lurking variables by creating groups with the only difference being the treatments

2. Random Assignment 3. Replication

impersonal chance to assign experimental units to treatments create roughly equal groups by balancing the effects of lurking variables that can’t be controlled

use enough experimental units in each group so differences from the treatments can be distinguished from just chance differences between the groups Don’t get lost in the vocabulary - remember the main goal is to create “large” treatment groups with no systematic differences between them other than the treatment!

A Template to Design an Experiment Explanatory Variables/Factors

Response Variable

Keep in mind 3 Principles: 1. Control 2. Random Assignment 3. Replication

Sample

Random Assignment of experimental units/subjects

Assign Treatment(s)

State what results will be measured and compared

more will be added later

What conclusions can be drawn?

What went wrong? •

Will cash bonuses speed the return to work of unemployed people? A state department of labor notes that last year 68% of people who filed claims for unemployment insurance found a new job within 15 weeks. As an experiment, this year the state offers $500 to people filing unemployment claims if they find a job within 15 weeks. The percent who do so increases to 77%. What flaw does this design have? Is it impossible to say whether the bonus really caused the increase?

Design a completely randomized experiment. • 150 students are willing to serve as subjects to study the effects of repeated exposure to an advertising message. The answer depends on both the length of the ad and on how often it is repeated. The different lengths that will be used are 30 second ads and 90 second ads. The commercial will either be shown 1, 3, or 5 times during the program. After viewing, all the subjects answered questions about their recall of the ad, their attitude toward the camera ad, and their intention to purchase it.

Ch 4.2 - Experiments Day 6

Placebo Effect •

a response to a dummy treatment • subjects do not know they are receiving a placebo because the effects are so strong

How to Control the Placebo Effect • •

•

Double Blind Experiments - neither the subject nor those who interact with them know who has what treatment Single-Blind Experiments - when the subject knows what treatment they receive, but those measuring the responses do not.

Double Blind is the best... WHY?

A Template to Design an Experiment Explanatory Variables/Factors

Response Variable

Keep in mind 3 Principles: 1. Control 2. Random Assignment 3. Replication

Sample

Random Assignment of experimental units/subjects

Assign Treatment(s)

State if there is any blinding, if so why

more will be added later

State what results will be measured and compared

What conclusions can be drawn?

and now, for two more infamous stat words:

• Statistically Significant •

“an observed effect so large that it would rarely occur by chance”

•

How can we measure if an experiment’s results were statistically significant?

Can random just be “unlucky?” • •

•

NO! If you think there is going to be a difference between the way the experimental units will react to the treatment, then you need to create a design to control those lurking variables.

i.e. With the distracted driving activity, let’s say that you feel women will forget more anyways because they are worse drivers. Then you will need to design your experiment to have women and men in both groups, but still randomly assigned!

Blocking •

Used mainly when there are strata already built into the population where a stratified random sample will be appropriate. Blocking is a form of control in the experimental design.

1. Block - the group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments.

2. Randomized Block Design - random assignment of experimental units to treatments that is carried out separately within each block

Stratified Random Sample or Randomized Block Design? Stratified Random Randomized Block Sample Design Controls lurking variables Forms similar groups

Done when taking the sample from the population Done when assigning units to the treatments

Examples of a Randomized Block Design •

Anne is an avid baker who would like to compare two different chocolate chip cookie recipes (A and B). So she recruits 10 volunteer taste testers to rate each type of cookie on a scale of 1 (very bad) to 10 (very good). She will make 10 of each type of cookie, for a total of 20. Each cookie tray will hold only 10 cookies, so she will use two trays to bake them at the same time in the same oven, one sheet on the lower rack and one of the upper rack. She thinks that the cookies will bake differently depending on which rack they are on, we will use the 10 locations on the lower-rack cookie sheet as 1 block and the 10 locations on the upper-rack cookie sheet as the 2nd block. On each sheet Anne will randomly place 5 of each type of cookie. This way each type of cookie will have 5 on the lower rack and 5 on the upper rack, balancing out the effect of rack location.

•

What are the experimental units?

•

What are the treatments?

•

Draw a diagram to represent this situation.

Matched Pairs • • •

•

a type of blocking block size is always 2 because only two treatments every experimental unit will receive both treatments, in a random order or the experimental units are paired as closely as possible then each treatment is assigned randomly to each unit results are compared from each individual and then as a whole group

A Template to Design an Experiment Explanatory Variables/Factors

Response Variable

Keep in mind 3 Principles: 1. Control 2. Random Assignment 3. Replication

Sample

Random Assignment of experimental units/subjects

Assign Treatment(s) State if there is any blinding, if so why

State if there is any blocking, if so why

State what results will be measured and compared

What conclusions can be drawn?