Two-Way ANOVA

Introduction

• Recall that ANOVA allows us to compare the means of three or more groups.

• In one-way ANOVA, we are only considering one factor (grouping variable).

• In this module, we discuss two-way ANOVA, which allows us to consider a second factor (grouping variable).

R Setup

library(tidyverse)
library(ssstats)
exam_scores <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/5-1-exam_scores.csv")
fatigue_scores <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/5-2-fatigue_scores.csv")

Two-Way ANOVA

• Recall in one-way ANOVA, we partitioned SS_Tot into SS_Trt and SS_E.

• Now that multiple factors are involved, we now partition the SS_Trt into the different factors.

  • Here, that is SS_A, SS_B, and SS_AB.

• Recall that SS_E is the “catch all” for unexplained variance.

  • When we add factors to our model, we are moving part of the SS_E into the SS_Trt.

Two-Way ANOVA

• Terminology in two-way ANOVA.

• Factor A has a levels.

  • Donut flavor: glazed, chocolate, maple

• Factor B has b levels.

  • Species: wolf, rabbit, elephant

• There are ab “treatment groups.”

  • Every observation will belong to one level of a and one level of b.

Interaction Effects

• Suppose Okonkwo is testing a new donut recipe for the break room. He has officers taste-test it and rate how much they liked it.

• The outcome is the raters’ taste score from 1–10.

• Then, he’s varying two things at once:

  • Donut flavor: which flavor was baked (Glazed, Chocolate, Maple)

  • Species: what kind of animal is doing the tasting (Wolf, Rabbit, Elephant)

Interaction Effects

• This design allows us to check for an interaction effect, implying the two factors don’t act independently on the outcome.

  — i.e., the effect of one factor on the outcome depends on the level of the other factor.

• In Okonkwo’s donut test, an interaction implies both:

  • The relationship between rating and donut flavor depends on species.
  • The relationship between rating and species depends on donut flavor.

• Remember, these are two descriptions of the same interaction, not two separate effects.

Two-Way ANOVA: Table

• The table we are working towards:

Two-Way ANOVA: Computation

• Let there be a levels of factor A and b levels of factor B.

  • y_{ijk} is the observation on the k^{th} experimental unit receiving the i^{th} level of factor A and the j^{th} level of factor B
  • y_{i.} is the sum for all observations at the i^{th} level of factor A,
  • y_{.j} is the sum for all observations at the j^{th} level of factor B,
  • y_{ij} is the sum for observations at the i^{th} level of factor A and the j^{th} level of factor B,
  • y_{..} is the sum of all observations,
  • (y^2)_{..} is the sum of the squared observations,
  • n is the number in each group (of a \times b treatments)

Two-Way ANOVA: Computation

• To find the SS and df: \begin{align*} \text{SS}_{\text{A}} &= \frac{\sum_i y_{i.}^2}{bn} - \frac{(y_{..})^2}{abn} & \text{df}_{\text{A}} &= a-1 \\ \text{SS}_{\text{B}} &= \frac{\sum_j y_{.j}^2}{an} - \frac{(y_{..})^2}{abn} & \text{df}_{\text{B}} &= b-1 \\ \text{SS}_{\text{AB}}&= \frac{\sum_{ij} y_{ij}^2}{n} - \frac{(y_{..})^2}{abn} - \text{SS}_{\text{A}} - \text{SS}_{\text{B}} & \text{df}_{\text{AB}} &= (a-1)(b-1) \\ \text{SS}_{\text{E}} &= \text{SS}_{\text{Tot}} - \text{SS}_{\text{A}} - \text{SS}_{\text{B}} - \text{SS}_{\text{AB}} & \text{df}_{\text{E}} &= ab(n-1) \\ \text{SS}_{\text{Tot}} &= (y^2)_{..} - \frac{(y_{..})^2}{abn} & \text{df}_{\text{Tot}} &= abn-1 \end{align*}

Two-Way ANOVA: Computation

• To find the MS:

\text{MS}_{\text{X}} = \frac{\text{SS}_{\text{X}}}{\text{df}_{\text{X}}}

• To find the test statistics:

\text{F}_{\text{X}} = \frac{\text{MS}_{\text{X}}}{\text{MS}_{\text{E}}}

Two-Way ANOVA (R)

• To get the two-way ANOVA table in R,

1dataset_name |> anovas(outcome = response_variable,
2                       A = factor_A,
3                       B = factor_B,
4                       interaction = TRUE or FALSE)

Hypothesis Test for an Interaction

• Hypotheses

  • H_0: there is not an interaction between [factor A] and [factor B]
  • H_1: there is an interaction between [factor A] and [factor B]

• Test Statistic and p-Value

  • F_{\text{AB}}
  • p = P[F_{\text{df}_{\text{AB}}, \text{df}_{\text{E}}} \ge F_{\text{AB}}]

Hypothesis Test for an Interaction (R)

• We again use the anovas() function,

dataset_name |> anovas(outcome = response_variable,
                       A = factor_A,
                       B = factor_B,
                       interaction = TRUE,
                       alpha = alpha_level)

Example 1

• Okonkwo pulls up a report on your screen: “Wellness office wants to know: does night shift wear people out the same way across every unit?”

• We will use the fatigue_scores dataset to answer the following question:

  • Does the effect of shift (Day vs. Night) on fatigue depend on which unit an officer works in?

Example 1

• Exploring the levels in shift and unit,

fatigue_scores |> n_pct(shift, unit)

Example 1

• Does the effect of shift on fatigue depend on which unit an officer works in?

fatigue_scores |> anovas(outcome = fatigue_score,
                         A = shift,
                         B = unit)

Two-way ANOVA (with interaction)

Source       df          SS        MS       F  p-value 
───────────────────────────────────────────────── 
shift         1    998.2845  998.2845  10.274    0.002 
unit          2     51.7080   25.8540   0.266    0.767 
shift:unit    2    729.3173  364.6587   3.753    0.025 
Residuals   174  16906.4077   97.1633                  
───────────────────────────────────────────────── 


Hypotheses (shift:unit):
H₀: There is no interaction between shift and unit
H₁: There is an interaction between shift and unit
Test statistic: F(2, 174) = 3.753
p-value: 0.025
Conclusion: Reject the null hypothesis (p = 0.025 < α = 0.050)

Example 1

• Hypotheses

  • H_0: there is not an interaction between shift and unit
  • H_1: there is an interaction between shift and unit

• Test Statistic and p-Value

  • F_0 = 3.753
  • p = 0.025

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that there is an interaction between shift and unit.

Example 2

• Okonkwo again: “Training division ran their numbers and thinks the new Enhanced program is worth the extra budget, but I want a second opinion. Does the training method work the same for everybody?”

• We will use the exam_scores dataset to answer the following question:

  • Does the effect of training method depend on experience level?

Example 2

• Exploring the levels in training method and experience level,

exam_scores |> n_pct(training_method, experience_level)

Example 2

• Does the effect of training method depend on experience level?

exam_scores |> anovas(outcome = exam_score,
                      A = training_method,
                      B = experience_level)

Two-way ANOVA (with interaction)

Source                             df          SS        MS      F  p-value 
────────────────────────────────────────────────────────────────────── 
training_method                     2   1339.2191  669.6096  7.584  < 0.001 
experience_level                    1     88.9014   88.9014  1.007    0.317 
training_method:experience_level    2      4.8084    2.4042  0.027    0.973 
Residuals                         174  15362.8583   88.2923                 
────────────────────────────────────────────────────────────────────── 


Hypotheses (training_method:experience_level):
H₀: There is no interaction between training_method and experience_level
H₁: There is an interaction between training_method and experience_level
Test statistic: F(2, 174) = 0.027
p-value: 0.973
Conclusion: Fail to reject the null hypothesis (p = 0.973 ≥ α = 0.050)

Example 2

• Hypotheses

  • H_0: there is not an interaction between training method and experience level
  • H_1: there is an interaction between training method and experience level

• Test Statistic and p-Value

  • F_0 = 0.027
  • p = 0.973

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest that there is an interaction between training method and experience level.

Two-Way ANOVA: Testing Main Effects

• What if the interaction is not significant?

  • We want to remove the interaction and examine the main effects.
  • That is, we want to look at the individual factors.

• We will reconstruct the ANOVA table. If we were doing this by hand:

  1. Rewrite ANOVA table without interaction; do not change SS, df, and MS for the main effects and total.
  2. Update the error term: add the SS_AB to SS_E and df_AB to df_E.
  3. Recalculate MS_E.
  4. Recalculate F_A and F_B, then find the corresponding p-values.

Hypothesis Test for Main Effects

• Hypotheses

  • H_0: \mu_1 = \mu_2 = … = (\mu_a or \mu_b)
  • H_1: At least one \mu_i is different.

• Test Statistic and p-Value

  • F_{\text{X}}
  • p = P[F_{\text{df}_{\text{X}}, \text{df}_{\text{E}}} \ge F_{\text{X}}]

Hypothesis Test for Main Effects (R)

• We again use the anovas() function,

dataset_name |> anovas(outcome = response_variable,
                       A = factor_A,
                       B = factor_B,
                       interaction = FALSE,
                       alpha = alpha_level)

Example 2

• Are either training method or experience level related to exam scores?

exam_scores |> anovas(outcome = exam_score,
                      A = training_method,
                      B = experience_level,
                      interaction = FALSE)

Two-way ANOVA (no interaction)

Source             df          SS        MS      F  p-value 
────────────────────────────────────────────────────── 
training_method     2   1339.2191  669.6096  7.669  < 0.001 
experience_level    1     88.9014   88.9014  1.018    0.314 
Residuals         176  15367.6668   87.3163                 
────────────────────────────────────────────────────── 


Hypotheses (training_method):
H₀: All means across levels of training_method are equal
H₁: At least one mean across levels of training_method differs
Test statistic: F(2, 176) = 7.669
p-value: < 0.001
Conclusion: Reject the null hypothesis (p = < 0.001 < α = 0.050)

Hypotheses (experience_level):
H₀: All means across levels of experience_level are equal
H₁: At least one mean across levels of experience_level differs
Test statistic: F(1, 176) = 1.018
p-value: 0.314
Conclusion: Fail to reject the null hypothesis (p = 0.314 ≥ α = 0.050)

Example 2

• Hypotheses

  • H_0: \mu_{\text{E}} = \mu_{\text{P}} = \mu_{\text{S}}
  • H_1: at least one training method is different

• Test Statistic and p-Value

  • F_0 = 7.669
  • p < 0.001

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that there is a difference between the training methods.

Example 2

• Hypotheses

  • H_0: \mu_{\text{R}} = \mu_{\text{V}}
  • H_1: \mu_{\text{R}} \ne \mu_{\text{V}}

• Test Statistic and p-Value

  • F_0 = 1.018
  • p = 0.314

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest a difference between rookies and veterans.

Wrap Up

• In this lecture, we have covered two-way ANOVA.

  • Test for interaction.

  • Test for main effects.

• In our following lectures, we will discuss next steps after initial hypothesis testing in two-way ANOVA.

  • posthoc testing

  • profile plots