STAT1008 Lecture Notes - Lecture 19: Dependent And Independent Variables, Breakfast Cereal, Multiple Comparisons Problem
STAT1008 Week 6 Lecture B
● Intervals and tests
○ If a 95% CI misses the parameter in H0, then a two-tailed test should reject H0 at
a 5% significance level
○ If a 95% CI contains the parameter in H0, then a two-tailed test should not reject
H0 at a 5% significance level
○ A CI represents the range of plausible values for the population parameter
○ If the null hypothesized value is not within the CI, it is not a plausible value and
should be rejected
○ If the null hypothesized value is within the CI, it is a plausible value and should
not be rejected
○ CI are most useful when you want to estimate population parameters
○ Hypothesis tests and p-values are most useful when you want to test hypotheses
values about population parameters
○ CI give you a range of plausible values; p-values quantify the strength of
evidence against the null hypothesis
● Interval, Test or neither?
○ Are the following questions best assessed using a CI, hypothesis test or is
statistical inference not relevant?
■ Do majority of adults riding a bicycle wear a helmet? Hypothesis test
■ On average, how much more do adults who played sports in high school
exercise than adults who did not play sports in high school? Confidence
interval
■ On average, were the 23 players on the 2010 Canadian Olympic hockey
team older than the 23 players on the 2010 US Olympic hockey team?
Hypothesis testing
● Statistical vs practical significance
○ With small sample sizes, even large differences or effects may not be significant
○ With large sample sizes, even a very small difference or effect can be significant
○ E.g. If Labor was 50.1% and Liberal was 49.9% then there is no practical
significance
● Multiple Testing
○ When multiple hypothesis tests are conducted, the chance that at least one test
incorrectly rejects a true null hypothesis increases with the number of tests.
○ If the null hypotheses are all true, alpha of the tests will yield statistically
significant results just by random chance.
○ Consider a topic that i being investigated by research teams all over the world
■ Using alpha = 0.05, 5% of teams are going to find something significant,
even if the null hypothesis is true
○ Consider a research team/company doing many hypothesis tests
find more resources at oneclass.com
find more resources at oneclass.com
Document Summary
If a 95% ci misses the parameter in h0, then a two-tailed test should reject h0 at a 5% significance level. If a 95% ci contains the parameter in h0, then a two-tailed test should not reject. A ci represents the range of plausible values for the population parameter. If the null hypothesized value is not within the ci, it is not a plausible value and should be rejected. If the null hypothesized value is within the ci, it is a plausible value and should not be rejected. Ci are most useful when you want to estimate population parameters. Hypothesis tests and p-values are most useful when you want to test hypotheses values about population parameters. Ci give you a range of plausible values; p-values quantify the strength of evidence against the null hypothesis. With small sample sizes, even large differences or effects may not be significant.
