Review of Statistical Inference

Introduction

  • This lecture reviews the concepts of statistical estimation and inference:

    • point estimation
    • interval estimation
    • hypothesis testing

  • There is an interactive widget in this lecture’s slides – please see class website :)

Point Estimation

  • point estimate a sample statistic that estimates the value of a population parameter; this estimate is based on the sample data.

  • Sample statistics estimate population parameters.

    • \bar{x} estimates \mu
    • s estimates \sigma
    • \hat{p}_1 - \hat{p}_2 estimates \pi_1 - \pi_2

Interval Estimation

  • confidence interval a range of plausible values for the parameter under investigation; this estimate is based on the sample data.

    • Lower bound: minimum plausible value for the parameter according to the data
    • Upper bound: maximum plausible value for the parameter according to the data

  • Confidence intervals are found for population parameters.

    • CI for \pi
    • CI for \mu_1-\mu_2
    • CI for \sigma^2

Interval Estimation

  • level of estimation the expected proportion of intervals that will contain the true value of the population parameter if the population is repeatedly sampled.

  • A common level of estimation is 95%.

    • If we were to draw 100 samples from the same population, we expect 95 of the estimated confidence intervals to contain the true value of the population parameter.
Wider intervals correspond to higher confidence

Confidence Intervals

  • Often, confidence intervals will take the form

\text{point estimate} \ \pm \ \text{margin of error}

  • Lower bound = point estimate - margin of error

  • Upper bound = point estimate + margin of error

  • The margin of error is the critical value \times standard error.

    • The critical value will depend on the level of confidence and the sampling distribution of the point estimate.

    • The standard error corresponds to the point estimate.

Example 1

  • The Grand Floridian claims average wait time at the Beach Pool bar is 8 minutes, but guests believe it is longer.

    • Analyst determines that the 95% CI for \mu is (8.09, 11.91)

    • We are 95% confident the true average wait time at the Beach Pool bar during summer break is between 8.09 and 11.91 minutes.

  • Takeaway: because the interval is larger than 8, the average wait time is likely longer than 8 minutes.

Example 2

  • Disney wonders if the nightly campfires are less popular at Port Orleans French Quarter than at Port Orleans Riverside.

    • Analyst determines that the 95% CI for \pi_{\text{POFQ}} - \pi_{\text{POR}} is ($-$0.321, 0.021).

    • We are 95% confident the true difference in campfire attendance proportions (POFQ − POR) is between $-$0.321 and 0.021.

  • Takeaway: because the interval contains 0, we cannot conclude that one resort’s campfire is more popular than the other.

Hypothesis Testing

  • hypothesis testing a statistical procedure that allows us to make inferences about population parameters based on sample data.

  • General steps:

    1. Make a statement about the nature of the population.

    2. Collect evidence to test the statement.

    3. Analyze the data to determine the plausibility of the statement.

Hypotheses

  • hypothesis a statement regarding the population parameter.

  • We will be stating two hypotheses: the null and the alternative.

    • H_0: null hypothesis a mathematical statement about the population parameter; this hypothesis is assumed true until evidence indicates otherwise.

    • H_1: alternative hypothesis a statement about the population parameter that contradicts the null hypothesis; this is also called the research hypothesis.

  • Note! H_0 + H_1 should cover the real line, \mathbb{R}.

    • Some textbooks teach that “=” is always used in H_0.

Hypotheses

  • One sample tests

    • Two-tailed test
      • H_0: parameter = hypothesized value
      • H_1: parameter \ne hypothesized value
    • Left-tailed test
      • H_0: parameter \ge hypothesized value
      • H_1: parameter < hypothesized value
    • Right-tailed test
      • H_0: parameter \le hypothesized value
      • H_1: parameter > hypothesized value

Hypotheses

  • Examples:

    • “Guests spend more than $50/day at a theme park” \to H_1: \mu > 50.

    • “80% of guests are satisfied with their visit to the park” \to H_1: \pi = 0.80.

    • “The variability in daily guest spending is less than $100” \to H_1: \sigma^2 < 100.

    • “The median wait time for an attraction at Hollywood Studios is greater than 30 minutes” \to H_1: M > 30.

    • “Fewer than 25% of guests purchase popcorn buckets” \to H_1: \pi < 0.25.

Hypotheses

  • Two sample tests

    • Two-tailed test
      • H_0: parameter1 - parameter2 = hypothesized difference
      • H_1: parameter1 - parameter2 \ne hypothesized difference
    • Left-tailed test
      • H_0: parameter1 - parameter2 \ge hypothesized difference
      • H_1: parameter1 - parameter2 < hypothesized difference
    • Right-tailed test
      • H_0: parameter1 - parameter2 \le hypothesized difference
      • H_1: parameter1 - parameter2 > hypothesized difference

Hypotheses

  • Examples:

    • “Guests spend more at Disney’s Hollywood Studios than at Disney’s Animal Kingdom” \to H_1: \mu_{\text{DHS}} > \mu_\text{DAK}.

    • “The proportion of guests who purchase Lightning Lane passes differs between annual passholders and single-day ticket holders” \to H_1: \pi_\text{AP} \ne \pi_\text{SD}.

    • “The difference in Lightning Lane purchase rates between passholders and single-day guests less than 10 percentage points” \to H_1: \pi_\text{AP} - \pi_\text{SD} < 0.10.

    • “The wait time for a ride is different in the morning vs. the afternoon on the same day” \to H_1: \mu_d \ne 0, where \mu_d = \mu_\text{am} - \mu_\text{pm}.

Hypotheses

  • Once we have the alternative hypotheses, we know the null hypothesis is in the opposite direction.

    • “Guests spend more than $50/day at a theme park”
      • H_0: \mu \le 50
      • H_1: \mu > 50
    • “The difference in Lightning Lane purchase rates between passholders and single-day guests less than 10 percentage points”
      • H_0: \pi_\text{AP} - \pi_\text{SD} \ge 0.10
      • H_1: \pi_\text{AP} - \pi_\text{SD} < 0.10

Hypotheses

  • Once we have the alternative hypotheses, we know the null hypothesis is in the opposite direction.

    • “80% of guests are satisfied with their visit to the park”
      • H_0: \pi = 0.80
      • H_1: \pi \ne 0.80
    • “Guests spend more at Disney’s Hollywood Studios than at Disney’s Animal Kingdom”
      • H_0: \mu_{\text{DHS}} \le \mu_\text{DAK} \to H_0: \mu_{\text{DHS}} - \mu_\text{DAK} \le 0
      • H_1: \mu_{\text{DHS}} > \mu_\text{DAK} \to H_1: \mu_{\text{DHS}} - \mu_\text{DAK} > 0

Test Statistics

  • After stating our hypotheses, we will construct an appropriate test statistic.

    • A test statisic is a measure of the distance from what you observed to what you are assuming to he true.

  • The choice of test statistic depends on:

    1. The hypotheses being tested.

    2. Assumptions made about the data.

  • The value of the test statistic depends on the sample data.

    • If we were to draw a different sample, our calculations would result in a different value for the test statistic.

  • We will use the test statistic on our way to drawing conclusions about the hypotheses.

p-Value

  • p-value the probability of observing our test statistic or one more extreme.

    • More extreme?

      • If the alternative hypothesis is right-tailed, we will consider the right tail of the distribution.

      • If the alternative hypothesis is left-tailed, we will consider the left tail of the distribution.

      • If the alternative hypothesis is two-tailed, we will consider both tails of the distribution.

  • We use the p-value to determine if we reject the null (believe the alternative) or fail to reject the null (do not believe the alternative).

p-Value

  • p-value the probability of observing our test statistic or one more extreme.
Alternative hypothesis:
1.50
p-value = 0.1336

p-Values

  • Mathematically, we are asking

    • Right-tailed test: P[\text{distribution} \ge \text{test statistic}]

    • Left-tailed test: P[\text{distribution} \le \text{test statistic}]

    • Two-tailed test: P[\text{distribution} \ne \text{test statistic}]

Rejection Regions

  • We use rejection regions to determine if we reject the null or fail to reject the null.

  • Classical approach: critical values and statistical tables.

  • Modern approach: p-values.

    • p-value the probability of observing our test statistic or one more extreme.

  • All rejection regions will take the same form:

Reject H_0 if p < \alpha.

Errors in Hypothesis Testing

  • Wait, what is \alpha?

    • \alpha is the significance level of the test.

  • A Type I error (\alpha) means that we reject the null when we should not.

    • \alpha = P[\text{reject } H_0 \text{ when } H_0 \text{ is true}]

  • A Type II error (\beta) means that we fail to reject the null when we should not.

    • \beta = P[\text{fail to reject } H_0 \text{ when } H_1 \text{ is true}]

  • We should choose \alpha based on the level of error we are willing to withstand.

    • The \alpha that is commonly used is \alpha=0.05.

    • Sometimes, smaller \alpha is used. e.g., clinical trial \to \alpha=0.01.

Drawing a Conclusion

  • We draw a conclusion by following the instructions of our rejection region.

  • If p < \alpha, we reject the null hypothesis.

    • There is sufficient evidence to suggest that the alternative is true.

  • If p \ge \alpha, we fail to reject the null hypothesis.

    • There is not sufficient evidence to suggest that the alternative is true.

  • Note the language – “fail to reject” is not the same as “accept.”


We never “accept” H_0.

Interpretation of Results

  • We will always report our results in the context of the problem.

  • Examples:

    • There is sufficient evidence to suggest that the average daily spending of guests at Disney’s Hollywood Studios is greater than $50.

    • There is not sufficient evidence to suggest that the proportion of guests who purchase Lightning Lane passes differs between annual passholders and single-day ticket holders.

Putting it Together

  • For class purposes, you will be reporting all hypothesis test pieces.

    • Hypotheses
    • Test Statistic and p-Value
    • Rejection Region
    • Conclusion and Interpretation

Example 1

  • The Grand Floridian claims average wait time at the Beach Pool bar is 8 minutes, but guests believe it is longer.

  • Hypotheses

    • H_0: \mu \le 8
    • H_1: \mu > 8

  • Test Statistic and p-Value

    • t = 2.14
    • p = 0.021

Example 1

  • The Grand Floridian claims average wait time at the Beach Pool bar is 8 minutes, but guests believe it is longer.

  • Test Statistic and p-Value

    • t = 2.14
    • p = 0.021

  • Rejection Region

    • Reject H_0 if p < \alpha; \ \alpha = 0.05

  • Conclusion and Interpretation

    • Reject H_0. There is sufficient evidence to suggest that the average wait time at the Beach Pool bar is greater than 8 minutes during summer break.

Example 2

  • Disney wonders if the nightly campfires are less popular at Port Orleans French Quarter than at Port Orleans Riverside.

  • Hypotheses

    • H_0: \pi_{\text{POFQ}} \ge \pi_{\text{POR}}
    • H_1: \pi_{\text{POFQ}} < \pi_{\text{POR}}

  • Test Statistic and p-Value

    • z = 1.64
    • p = 0.051

Example 2

  • Disney wonders if the nightly campfires are less popular at Port Orleans French Quarter than at Port Orleans Riverside.

  • Test Statistic and p-Value

    • z = 1.64
    • p = 0.051

  • Rejection Region

    • Reject H_0 if p < \alpha; \ \alpha = 0.05

  • Conclusion and Interpretation

    • Fail to reject H_0. There is not sufficient evidence to suggest that a smaller proportion of guests attend nightly campfires at Port Orleans French Quarter as compared to Port Orleans Riverside.

Example 3

  • For the purpose of beach towel distribution, the lead of Mousekeeping at Art of Animation asks if room type (Little Mermaid, Finding Nemo, Cars, or Lion King) affects pool preference (Big Blue, Cozy Cone, or Flippin’ Fins).

  • Hypotheses

    • H_0: pool preference does not depend on room type
    • H_1: pool preference depends on room type

  • Test Statistic and p-Value

    • \chi^2 = 9.87
    • p = 0.020

Example 3

  • For the purpose of beach towel distribution, the lead of Mousekeeping at Art of Animation asks if room type (Little Mermaid, Finding Nemo, Cars, or Lion King) affects pool preference (Big Blue, Cozy Cone, or Flippin’ Fins).

  • Test Statistic and p-Value

    • \chi^2 = 9.87
    • p = 0.020

  • Rejection Region

    • Reject H_0 if p < \alpha; \ \alpha = 0.05

  • Conclusion and Interpretation

    • Reject H_0. There is sufficient evidence to suggest that pool preference depends on room type.

Practical vs. Statistical Significance

  • Hypothesis testing depends on sample size.

    • As the sample size increases, our test statistics increase.
    • As test statistics increase, our p-values decrease.
    • As p-values decrease, we are more likely to reject the null hypothesis.

  • We must ask ourselves if the value we are testing against makes practical sense.

    • Would you care about a new weight loss medication where the average amount of weight lost was 1 lb over 6 months?

    • Would you care about a new teaching method that only raised final exam scores by 2 points?

Wrap Up

  • point estimation: best guess at the population parameter.

  • interval estimation: range of plausible values of the population parameter.

  • hypothesis testing: formal way to evaluate evidence against a claim.

  • As we move forward, we will see how all of these concepts are connected.

    • Both confidence intervals and hypothesis tests are based on the same sampling distributions of point estimates.

    • In statistical theory, interval estimates are derived from test statistics.