One-Way ANOVA

Introduction

• Previous: one sample, two samples

• Now: more than two groups, continuous outcome

  • One-way ANOVA

  • Kruskal-Wallis

  • posthoc testing

• This lecture: one-way ANOVA

R Setup

library(tidyverse)
library(ssstats)
commute_times <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-2-commute_times.csv")
citations <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-4-citations.csv")
academy_scores <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-1-academy_scores.csv")

Introduction: Analysis of Variance

• We have previously discussed testing the difference between two groups.

  • What about when there are three or more groups?

• We will use a method called analysis of variance (ANOVA).

  • This method partitions the variance of the outcome into variance due to the groups and variance due to “other” factors.

• Fun fact: the two-sample t-test is a special case of ANOVA.

  • If you perform ANOVA when comparing two means, you will obtain the same results as the two-sample t-test.

Analysis of Variance

• Consider comparing k groups.

ANOVA Tables

• We will see our ANOVA results displayed in a table.

• Here is what we are working towards:

ANOVA: Sums of Squares

• We are partitioning the total variability into different components.

• The total sum of squares measures the overall variability in the data:

\text{SS}_{\text{Tot}} = \sum_{i=1}^k \sum_{j=1}^n (y_{ij} - \bar{y}_{..})^2

• and is partitioned into SS_Trt and SS_E:

\text{SS}_\text{Trt}=n \sum_{i=1}^k \left( \bar{y}_{i.}-\bar{y}_{..} \right)^2 \ \ \ \ \ \ \ \ \ \ \text{SS}_\text{E} = \sum_{i=1}^k \sum_{j=1}^n (y_{ij} - \bar{y}_{i.})^2

ANOVA: Degrees of Freedom

• Next, we have the degrees of freedom. Each sum of square will have its own degrees of freedom.

• The total df:

\text{df}_{\text{Tot}} = n-1

• and is partitioned into df_Trt and df_E:

\text{df}_\text{Trt}=k - 1 \ \ \ \ \ \ \ \ \ \ \text{df}_\text{E} = n - k

ANOVA: Mean Squares and Test Statistic

• To find the mean squares, we simply divide the sum of squares by the appropriate degrees of freedom,

\text{MS}_\text{Trt}=\frac{\text{SS}_{\text{Trt}}}{\text{df}_{\text{Trt}}} \ \ \ \ \ \ \ \ \ \ \text{MS}_\text{E}=\frac{\text{SS}_{\text{E}}}{\text{df}_{\text{E}}}

• Then, we can find our test statistic,

F_{\text{Trt}} = \frac{\text{MS}_{\text{Trt}}}{\text{MS}_{\text{E}}}

ANOVA: Mean Squares and Test Statistic

• Note for the future: the MS and F can be generalized –

\text{MS}_\text{X}=\frac{\text{SS}_{\text{X}}}{\text{df}_{\text{X}}} \ \ \ \ \ \ \ \ \ \ F_{\text{X}} = \frac{\text{MS}_{\text{X}}}{\text{MS}_{\text{E}}}

• When constructing a test statistic in ANOVA, we always divide by MS_E.

ANOVA Tables

• Revisiting the ANOVA table,

• Note the relationships:

  • SS_Trt + SS_E = SS_Tot
  • df_Trt + df_E = df_Tot

Hypothesis Test for \mu_1 = \mu_2 = ... = \mu_k

• Hypotheses

  • H_0: \mu_1 = \mu_2 = ... = \mu_k
  • H_1: at least one mean is different.

• Test Statistic:

  • F_0 = \text{MS}_{\text{Trt}} / \text{MS}_{\text{E}}

Statistical Inference on k Means (R)

• We will use the anovas() function from library(ssstats) to construct the ANOVA table.

1dataset_name |> anovas(outcome = continuous_variable,
2                       grouping = grouping_variable,
3                       alpha = alpha_level)

Example 1

• Okonkwo catches you between meetings: “Officers in Northgate keep grumbling that their commute is worse than everyone else’s. Before I approve any relocation stipends, I want to know if commute time actually differs across our four districts.”

• We will use the commute_times dataset to answer the following question:

  • Does commute time differ across the four districts?

Example 1

• Does commute time differ across the four districts?

• How do we translate this into hypotheses?

  • No direction or hypothesized value.

  • We need to enumerate the number of means expressed in H_0.

• Hypotheses:

  • H_0: \mu_{\text{M}} = \mu_{\text{C}} = \mu_{\text{N}} = \mu_{\text{F}}
  • H_1: at least one district is different

Example 1

• Does commute time differ across the four districts?

commute_times |> anovas(outcome = commute_minutes,
                         grouping = district)

One-way ANOVA

Source      df         SS       MS      F  p-value 
───────────────────────────────────────────── 
district     3   120.0100  40.0033  0.780    0.507 
Residuals  116  5945.7900  51.2568                 
───────────────────────────────────────────── 


Hypotheses (district):
H₀: All group means are equal (no effect of district)
H₁: At least one group mean differs (effect of district exists)
Test statistic: F(3, 116) = 0.78
p-value: 0.507
Conclusion: Fail to reject the null hypothesis (p = 0.507 ≥ α = 0.050)

Example 1

• Hypotheses

  • H_0: \mu_{\text{M}} = \mu_{\text{C}} = \mu_{\text{N}} = \mu_{\text{F}}
  • H_1: at least one district is different

• Test Statistic and p-Value

  • F_0 = 0.780
  • p = 0.507

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest that there is a difference in commute times between the districts.

Example 2

• Okonkwo swings by your desk: “We just finished processing assessment scores for the incoming class. This cycle, recruits went through training at three different academies: Meadowbrook, Coldharbor, and Northgate. Command wants to know if it matters which academy a recruit trains at. Are scores different across the three?”

• We will use the academy_scores dataset to answer the following question:

  • Do assessment scores differ across the three training academies?

Example 2

• Do assessment scores differ across the three training academies (Meadowbrook, Coldharbor, and Northgate)?

• How do we translate this into hypotheses?

  • No direction or hypothesized value.

  • We need to enumerate the number of means expressed in H_0.

• Hypotheses:

  • H_0: \mu_{\text{M}} = \mu_{\text{C}} = \mu_{\text{N}}
  • H_1: at least one training academy is different

Example 2

• Do assessment scores differ across the three training academies (Meadowbrook, Coldharbor, and Northgate)?

academy_scores |> anovas(outcome = score,
                         grouping = academy)

One-way ANOVA

Source     df         SS        MS      F  p-value 
───────────────────────────────────────────── 
academy     2   786.9215  393.4607  5.898    0.004 
Residuals  93  6204.2681   66.7126                 
───────────────────────────────────────────── 


Hypotheses (academy):
H₀: All group means are equal (no effect of academy)
H₁: At least one group mean differs (effect of academy exists)
Test statistic: F(2, 93) = 5.898
p-value: 0.004
Conclusion: Reject the null hypothesis (p = 0.004 < α = 0.050)

Example 2

• Hypotheses

  • H_0: \mu_{\text{M}} = \mu_{\text{C}} = \mu_{\text{N}}
  • H_1: at least one training academy is different

• Test Statistic and p-Value

  • F_0 = 5.898
  • p = 0.004

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that there is a difference in assessment scores between academies.

Wrap Up

• This lecture introduced one-way ANOVA.

  • One-way ANOVA examines the effect of a single factor.

  • (The next module introduces multiple factors!)

• One-way ANOVA only allows us to determine if there is an overall, or omnibus, effect.

  • It does not tell us about pairwise differences.

• In the future:

  • Assessing ANOVA assumptions
  • Nonparametric alternative
  • Posthoc testing