Hypothesis Testing

Hypothesis Testing

A Research Question • Everybody knows men are better drivers than women. • Hypothesis: A tentative explanation that accounts for a set of facts and can be tested by further investigation – Hypothesis: Men are better drivers than women

Erin Crocket NASCAR

Proof • How would you prove that men are better drivers? • Gather information on the dependent variable (driving) and compare based on the independent variable (gender).

• What if most men were better but not all? • What about all the people you never asked?

What happens if this is reversed? • How could you disprove that there was a difference in driving ability by gender? • Null hypothesis: stating that no differences would be found when you really are interested in finding them. • If the hypothesis says that something is true then the null hypothesis says that it is not true. • A short diversion to Popper

Karl Popper: Method of Falsification • Form a hypothesis • Try to prove it wrong • From the results make a new hypothesis

Writing Null Hypotheses • What is your dependent variable? • What groups will you compare around the dependent variable (independent variable)? • Rewrite your problem statement as an expected outcome. • Rewrite the expected outcome statement as expecting no differences between groups to appear.

Hypothesis Testing • Hypothesis: Men are better drivers than women. • Null Hypothesis: There is no difference between men and women as drivers. • You gather evidence and decide if the evidence says that men are better drivers. • You show that the hypothesis is not not true. • You reject the null hypothesis.

Using Null Hypotheses • Use a sample (otherwise we would use descriptive statistics) • Stop talking about individuals and only compare groups (mean and standard deviation)

• Measure a bunch of people and if there is no difference you say, “I have been unable to find evidence that differences exist.” • You have been unable to “reject the null hypothesis.” • If you wanted to show that men drive differently than women, you failed. • The problem with understanding this is the double negative. You are trying to show that a hypothesis is not “not true.” • But, of course there will be some difference.

Example • The purpose of this study is to examine the impact of reading circles on comprehension scores. • Null hypothesis: There is no impact of reading circles on comprehension scores.

• If there is a difference between group means you have a problem. • How big do the differences have to be before we believe they didn’t just happen by chance? • The differences need to be significant: that means they are so big they are unlikely to happen by chance.

Rejecting the Null • If the group differences are small then you could make the case that the differences in comprehension scores could happen naturally (by chance) and not because of reading circles. (the null—there is no difference— is not rejected) • If the group differences are big enough then they are unlikely to have happened by chance: there is a significant difference in comprehension scores when students are in reading circles. (the null—there is no difference—is rejected)

Why do this? • This is Popper’s fault: Falsification Theory • Inferential statistics uses samples of populations to determine if differences in group means could occur by chance. • Inferential statistics is not used to prove hypotheses. It is used to demonstrate that null hypotheses are not true.

Think about this as steps: 1.

You want to know that something is true but you can’t absolutely know because you can’t test all cases.

2.

Instead you focus on the opposite of what you want to know is true (the null hypothesis).

3.

Start gathering evidence.

4.

If enough cases show that something is true then saying it was not true is false.

5.

You have rejected the null hypothesis.

6.

For the time being, the thing you wanted to show is true (your hypothesis) is the best explanation.

Ok, one more semantic step and we are done: • What we are interested in knowing is whether something made two groups different. • The evidence that you gather to see if that is true are means and standard deviations of groups. • The null hypothesis always says there will be no difference between the groups. • If there is a very low probability that the difference in the groups could occur by chance, that is the evidence that the null hypothesis is wrong. • Low probabilities (p values) show that the thing you wanted to show (your hypothesis) is more likely to be true than not true. You have rejected the null hypothesis.

Hypothesis Testing • Write your problem statement as a quantitative problem: group, characteristic to be measured (dependent), intervention (independent). • Remember we write problem statements like we don’t know what the answer is: What is the impact of reading circles on comprehension scores? (Non-Directional)

Hypothesis Testing • Now, write your problem statement as a hypothesis: What you think will be the impact of the independent variable on the dependent. (Directional) • Don’t put this in your paper but the hypothesis is what you are thinking is true.

Hypothesis Testing • Now, write the null hypothesis. No observable impact of the independent variable will occur on the dependent variable. (No Direction) • There will always be group mean differences but you want to show that group mean differences are large enough that you can reject the null hypothesis.