STA2023 Review:
Statistical Inference

Introduction

• This lecture reviews the concepts of statistical estimation and inference:

  • point estimation
  • interval estimation
  • hypothesis testing

• There is an interactive widget in this lecture’s slides – please see class website :)

Point Estimation

• Sample statistics estimate population parameters.

  • \bar{x} estimates \mu
  • s estimates \sigma
  • \hat{p}_1 - \hat{p}_2 estimates \pi_1 - \pi_2

Interval Estimation

• Lower bound: minimum plausible value for the parameter according to the data

• Upper bound: maximum plausible value for the parameter according to the data

• Confidence intervals are found for population parameters.

  • CI for \pi
  • CI for \mu_1-\mu_2
  • CI for \sigma^2

Interval Estimation

• A common level of estimation is 95%.

  • If we were to draw 100 samples from the same population, we expect 95 of the estimated confidence intervals to contain the true value of the population parameter.

Interval Estimation

Confidence Intervals

• Often, confidence intervals will take the form

\text{point estimate} \ \pm \ \text{margin of error}

• Lower bound = point estimate - margin of error

• Upper bound = point estimate + margin of error

• The margin of error is the critical value \times standard error.

  • The critical value will depend on the level of confidence and the sampling distribution of the point estimate.

  • The standard error corresponds to the point estimate.

Example 1

• The Ponyville Spa claims average wait time for a treatment with Rarity and Fluttershy is 8 minutes, but guests believe it is longer.

  • Analyst determines that the 95% CI for \mu is (8.09, 11.91).

  • We are 95% confident the true average wait time at the Ponyville Spa during the summer rush is between 8.09 and 11.91 minutes.

• Takeaway: because the interval is larger than 8, the average wait time is likely longer than 8 minutes.

Example 2

• Ponyville wonders if the nightly campfire sing-alongs are less popular at Sweet Apple Acres than at the Ponyville Schoolhouse.

  • Analyst determines that the 95% CI for \pi_{\text{Acres}} - \pi_{\text{School}} is (– 0.321, 0.021).

  • We are 95% confident the true difference in campfire attendance proportions (Acres – School) is between –0.321 and 0.021.

• Takeaway: because the interval contains 0, we cannot conclude that one location’s campfire is more popular than the other.

Hypothesis Testing

• General steps:

  1. Make a statement about the nature of the population.

  2. Collect evidence to test the statement.

  3. Analyze the data to determine the plausibility of the statement.

Hypotheses

• We will be stating two hypotheses: the null and the alternative.

  • H_0: null hypothesis a mathematical statement about the population parameter; this hypothesis is assumed true until evidence indicates otherwise.

  • H_1: alternative hypothesis a statement about the population parameter that contradicts the null hypothesis; this is also called the research hypothesis.

• Note! H_0 + H_1 should cover the real line, \mathbb{R}.

  • Some textbooks teach that “=” is always used in H_0.

Hypotheses

• One sample tests

  • Two-tailed test
    • H_0: parameter = hypothesized value
    • H_1: parameter \ne hypothesized value
  • Left-tailed test
    • H_0: parameter \ge hypothesized value
      
    • H_1: parameter < hypothesized value
  • Right-tailed test
    • H_0: parameter \le hypothesized value
      
    • H_1: parameter > hypothesized value

Hypotheses

• Examples:

  • “Ponies spend more than 50 bits/day at the Crystal Empire fair”
    • H_1: \mu > 50.
  • “80% of ponies are satisfied with their visit to the fair”
    • H_1: \pi = 0.80.
  • “The variability in daily pony spending is less than $100”
    • H_1: \sigma^2 < 100.

Hypotheses

• Two sample tests

  • Two-tailed test
    • H_0: parameter₁ - parameter₂ = hypothesized difference
    • H_1: parameter₁ - parameter₂ \ne hypothesized difference
  • Left-tailed test
    • H_0: parameter₁ - parameter₂ \ge hypothesized difference
    • H_1: parameter₁ - parameter₂ < hypothesized difference
  • Right-tailed test
    • H_0: parameter₁ - parameter₂ \le hypothesized difference
    • H_1: parameter₁ - parameter₂ > hypothesized difference

Hypotheses

• Examples:

  • “Ponies spend more at the Crystal Empire fair than at the Ponyville fair”

    • H_1: \mu_{\text{CE}} > \mu_\text{PV}.

  • “The proportion of ponies who purchase VIP cloud-seats differs between season-pass holders and single-day ticket holders”

    • H_1: \pi_\text{SP} \ne \pi_\text{SD}.

  • “The wait time for a ride is different in the morning vs. the afternoon on the same day”

    • H_1: \mu_d \ne 0, where \mu_d = \mu_\text{am} - \mu_\text{pm}.

Hypotheses

• Once we have the alternative hypotheses, we know the null hypothesis is in the “opposite direction.”

  • “Ponies spend more than 50 bits/day at the Crystal Empire fair”
    • H_0: \mu \le 50
    • H_1: \mu > 50
  • “80% of ponies are satisfied with their visit to the fair”
    • H_0: \pi = 0.80
    • H_1: \pi \ne 0.80
  • “Ponies spend more at the Crystal Empire fair than at the Ponyville fair”
    • H_0: \mu_{\text{CE}} \le \mu_\text{PV} \to H_0: \mu_{\text{CE}} - \mu_\text{PV} \le 0
    • H_1: \mu_{\text{CE}} > \mu_\text{PV} \to H_1: \mu_{\text{CE}} - \mu_\text{PV} > 0

Test Statistics

• After stating our hypotheses, we will construct an appropriate test statistic.

  • A test statisic is a measure of the distance from what you observed to what you are assuming to he true.

• The choice of test statistic depends on:

  1. The hypotheses being tested.

  2. Assumptions made about the data.

• The value of the test statistic depends on the sample data.

  • If we were to draw a different sample, our calculations would result in a different value for the test statistic.

p-Value

• Mathematically,

  • Right-tailed test: P[\text{distribution} \ge \text{test statistic}]

  • Left-tailed test: P[\text{distribution} \le \text{test statistic}]

  • Two-tailed test: P[\text{distribution} \ne \text{test statistic}]

p-Value

Rejection Regions

• We use rejection regions to determine if we reject the null or fail to reject the null.

• Classical approach: critical values and statistical tables.

• Modern approach: p-values.

  • p-value the probability of observing our test statistic or one more extreme.

• All rejection regions will take the same form:

Errors in Hypothesis Testing

• Wait, what is \alpha?

  • \alpha is the significance level of the test.

• A Type I error (\alpha) means that we reject the null when we should not.

  • \alpha = P[\text{reject } H_0 \text{ when } H_0 \text{ is true}]

• A Type II error (\beta) means that we fail to reject the null when we should not.

  • \beta = P[\text{fail to reject } H_0 \text{ when } H_1 \text{ is true}]

• We should choose \alpha based on the level of error we are willing to withstand.

  • The \alpha that is commonly used is \alpha=0.05.

  • Sometimes, smaller \alpha is used. e.g., clinical trial \to \alpha=0.01.

Drawing a Conclusion

• We draw a conclusion by following the instructions of our rejection region.

• If p < \alpha, we reject the null hypothesis.

  • There is sufficient evidence to suggest that the alternative is true.

• If p \ge \alpha, we fail to reject the null hypothesis.

  • There is not sufficient evidence to suggest that the alternative is true.

• Note the language – “fail to reject” is not the same as “accept.”

Interpretation of Results

• We will always report our results in the context of the problem.

• Examples:

  • There is sufficient evidence to suggest that the average daily spending of ponies at the Crystal Empire fair is greater than 50 bits.

  • There is not sufficient evidence to suggest that the proportion of ponies who purchase VIP cloud-seats differs between season-pass holders and single-day ticket holders.

Putting it Together

• For class purposes, you will be reporting all hypothesis test pieces.

  • Hypotheses
  • Test Statistic and p-Value
  • Rejection Region
  • Conclusion and Interpretation

Example 1

• The Ponyville Spa claims average wait time for a treatment with Rarity and Fluttershy is 8 minutes, but guests believe it is longer.

• Hypotheses

  • H_0: \mu \le 8
  • H_1: \mu > 8

• Test Statistic and p-Value

  • t = 2.14
  • p = 0.021

Example 1

• The Ponyville Spa claims average wait time for a treatment with Rarity and Fluttershy is 8 minutes, but guests believe it is longer.

• Test Statistic and p-Value

  • t = 2.14
  • p = 0.021

• Rejection Region

  • Reject H_0 if p < \alpha; \ \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that the average wait time at the Beach Pool bar is greater than 8 minutes during summer break.

Example 2

• Ponyville wonders if the nightly campfire sing-alongs are less popular at Sweet Apple Acres than at the Ponyville Schoolhouse.

• Hypotheses

  • H_0: \pi_{\text{Acres}} \ge \pi_{\text{Schoolhouse}}
  • H_1: \pi_{\text{Acres}} < \pi_{\text{Schoolhouse}}

• Test Statistic and p-Value

  • z = 1.64
  • p = 0.051

Example 2

• Ponyville wonders if the nightly campfire sing-alongs are less popular at Sweet Apple Acres than at the Ponyville Schoolhouse.

• Test Statistic and p-Value

  • z = 1.64
  • p = 0.051

• Rejection Region

  • Reject H_0 if p < \alpha; \ \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest that a smaller proportion of ponies attend nightly campfire sing-alongs at Sweet Apple Acres as compared to the Ponyville Schoolhouse.

Example 3

• For the purpose of ribbon distribution, the lead organizer at the Ponyville Fair asks if cabin type (Cutie Mark Crusaders, Element of Harmony, Wonderbolts, or Crystal Empire) affects meadow preference (Sweet Apple Orchard, Everfree Edge, or Sparkle Springs).

• Hypotheses

  • H_0: cabin type does not affect meadow preference
  • H_1: cabin type affects meadow preference

• Test Statistic and p-Value

  • \chi^2 = 9.87
  • p = 0.020

Example 3

• For the purpose of ribbon distribution, the lead organizer at the Ponyville Fair asks if cabin type (Cutie Mark Crusaders, Element of Harmony, Wonderbolts, or Crystal Empire) affects meadow preference (Sweet Apple Orchard, Everfree Edge, or Sparkle Springs).

• Test Statistic and p-Value

  • \chi^2 = 9.87
  • p = 0.020

• Rejection Region

  • Reject H_0 if p < \alpha; \ \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that cabin type depends on meadow preference.

Practical vs. Statistical Significance

• Hypothesis testing depends on sample size.

  • As the sample size increases, our test statistics increase.
  • As test statistics increase, our p-values decrease.
  • As p-values decrease, we are more likely to reject the null hypothesis.

• We must ask ourselves if the value we are testing against makes practical sense.

  • Would you care about a new weight loss medication where the average amount of weight lost was 1 lb over 6 months?

  • Would you care about a new teaching method that only raised final exam scores by 2 points?

Wrap Up

• point estimation: best guess at the population parameter.

• interval estimation: range of plausible values of the population parameter.

• hypothesis testing: formal way to evaluate evidence against a claim.

• As we move forward, we will see how all of these concepts are connected.

  • Both confidence intervals and hypothesis tests are based on the same sampling distributions of point estimates.

  • In statistical theory, interval estimates are derived from test statistics.