ANOVA Assumptions

Introduction

• Previous: one sample, two samples

• Now: more than two groups, continuous outcome

  • One-way ANOVA

  • Kruskal-Wallis

  • posthoc testing

• This lecture: ANOVA assumptions

R Setup

library(tidyverse)
library(ssstats)
commute_times <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-2-commute_times.csv")
citations <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-4-citations.csv")
academy_scores <- read_csv("https://raw.githubusercontent.com/samanthaseals/SDSI/refs/heads/main/files/data/lectures/4-1-academy_scores.csv")

ANOVA Assumptions

• We can represent ANOVA with the following model:

y_{ij} = \mu + \tau_i + \varepsilon_{ij}

• where:

  • y_{ij} is the j^{\text{th}} observation in the i^{\text{th}} group,
  • \mu is the overall (grand) mean,
  • \tau_i is the treatment effect for group i, and
  • \varepsilon_{ij} is the error term for the j^{\text{th}} observation in the i^{\text{th}} group.

ANOVA Assumptions

• We assume that the error term follows a normal distribution with mean 0 and a constant variance, \sigma^2. i.e.,

\varepsilon_{ij} \overset{\text{iid}}{\sim} N(0, \sigma^2)

• Very important note: the assumption is on the error term (\varepsilon_{ij}) and NOT on the outcome (y_{ij})!

ANOVA Assumptions

• We will use the residual (the difference between the observed value and the predicted value) to assess assumptions:

e_{ij} = y_{ij} - \hat{y}_{ij}

• Note that in the case of one-way ANOVA, the predicted value of y is the group mean.

ANOVA Assumptions: Graphical Assessment

• Normality: quantile-quantile plot

  • Should have points close to the 45^\circ line
  • We will focus on the “center” portion of the plot

• Variance: scatterplot of the residuals against the predicted values

  • Should be “equal spread” between the groups
  • No “pattern”

Graphical Assessment of Assumptions (R)

• We will use the anova_check() function from library(ssstats) to assess normality.

1dataset_name |> anova_check(outcome = continuous_variable,
2                       grouping = grouping_variable)

• This will provide the the Q-Q plots and variance scatterplot necessary to assess the ANOVA assumptions.

Example 1

• Recall our first question: does commute time differ across the four districts?

commute_times |> anovas(outcome = commute_minutes,
                        grouping = district)

One-way ANOVA

Source      df         SS       MS      F  p-value 
───────────────────────────────────────────── 
district     3   120.0100  40.0033  0.780    0.507 
Residuals  116  5945.7900  51.2568                 
───────────────────────────────────────────── 


Hypotheses (district):
H₀: All group means are equal (no effect of district)
H₁: At least one group mean differs (effect of district exists)
Test statistic: F(3, 116) = 0.78
p-value: 0.507
Conclusion: Fail to reject the null hypothesis (p = 0.507 ≥ α = 0.050)

Example 1

• Checking the assumptions,

commute_times |> anova_check(outcome = commute_minutes,
                             grouping = district)

Example 1

• If it’s hard to “see” the qq plots, we can “zoom in” using one_qq(),

commute_times |> filter(district == "Coldharbor") |> one_qq(commute_minutes)

Example 1

• If it’s hard to “see” the qq plots, we can “zoom in” using one_qq(),

commute_times |> filter(district == "Fairview") |> one_qq(commute_minutes)

Example 1

• If it’s hard to “see” the qq plots, we can “zoom in” using one_qq(),

commute_times |> filter(district == "Meadowbrook") |> one_qq(commute_minutes)

Example 1

• If it’s hard to “see” the qq plots, we can “zoom in” using one_qq(),

commute_times |> filter(district == "Northgate") |> one_qq(commute_minutes)

Example 1

• Final assessment:

  • Normality: fine

  • Variance: ehhhh…?

Brown-Forsythe-Levine Test

• When we are not confident making a call graphically, we can formally check the variance.

  • The analytic approach:

    • Transform y_{ij} by subtracting off its group median.
    • Run ANOVA on these transformed values.

• Hypotheses:

  • H_0: \sigma_1^2=\sigma_2^2=\hdots=\sigma_k^2
  • H_1: population variances are not all equal

• Test Statistic:

  • F_0 on transformed data

Checking Variance (R)

• We will use the multiple_variances() function from library(ssstats) to formally test the equality of variances.

1dataset_name |> multiple_variances(outcome = continuous_variable,
2                                   grouping = grouping_variable,
3                                   alpha = alpha_level)

Example 1

• Checking the variance,

commute_times |> multiple_variances(outcome = commute_minutes,
                                    grouping = district)

Brown-Forsythe-Levene test for equality of variances

s²[Coldharbor] = 50.8543 (s = 7.1312)
s²[Fairview] = 62.2279 (s = 7.8885)
s²[Meadowbrook] = 44.1666 (s = 6.6458)
s²[Northgate] = 47.7784 (s = 6.9122)

Hypotheses:
H₀: All group variances are equal (no effect of district on variance)
H₁: At least one group variance differs (effect of district on variance exists)
Test statistic: F(3, 116) = 0.129
p-value: 0.943
Conclusion: Fail to reject the null hypothesis (p = 0.943 ≥ α = 0.050)

Example 1

• Hypotheses

  • H_0: \sigma^2_{\text{M}} = \sigma^2_{\text{C}} = \sigma^2_{\text{N}} = \sigma^2_{\text{F}}
  • H_1: at least one district is different

• Test Statistic and p-Value

  • F_0 = 0.129
  • p = 0.943

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest that there is a difference in the variance of commute times between the districts.

Example 2

• Do assessment scores differ across the three training academies (Meadowbrook, Coldharbor, and Northgate)?

academy_scores |> anovas(outcome = score,
                         grouping = academy)

One-way ANOVA

Source     df         SS        MS      F  p-value 
───────────────────────────────────────────── 
academy     2   786.9215  393.4607  5.898    0.004 
Residuals  93  6204.2681   66.7126                 
───────────────────────────────────────────── 


Hypotheses (academy):
H₀: All group means are equal (no effect of academy)
H₁: At least one group mean differs (effect of academy exists)
Test statistic: F(2, 93) = 5.898
p-value: 0.004
Conclusion: Reject the null hypothesis (p = 0.004 < α = 0.050)

Example 2

• Looking at the assumptions graphically,

academy_scores |> anova_check(outcome = score,
                              grouping = academy)

Example 2

• Verifying the variance assumption,

academy_scores |> multiple_variances(outcome = score,
                                     grouping = academy)

Brown-Forsythe-Levene test for equality of variances

s²[Coldharbor] = 78.2889 (s = 8.8481)
s²[Meadowbrook] = 69.3266 (s = 8.3263)
s²[Northgate] = 52.5222 (s = 7.2472)

Hypotheses:
H₀: All group variances are equal (no effect of academy on variance)
H₁: At least one group variance differs (effect of academy on variance exists)
Test statistic: F(2, 93) = 0.914
p-value: 0.404
Conclusion: Fail to reject the null hypothesis (p = 0.404 ≥ α = 0.050)

Example 2

• Hypotheses

  • H_0: \sigma^2_{\text{M}} = \sigma^2_{\text{C}} = \sigma^2_{\text{N}}
  • H_1: at least one district’s academy is different

• Test Statistic and p-Value

  • F_0 = 0.914
  • p = 0.404

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Fail to reject H_0. There is not sufficient evidence to suggest that there is a difference in the variance of assessment scores between the academies

Example 3

• Okonkwo’s last one for the day: “Traffic enforcement wants budget for more radar units. Their pitch is that citation activity is way higher in some districts than others. Before I sign off, let’s check the citation counts across the four districts.”

• We will use the citations dataset to answer the following question:

  • Does the number of citations issued per officer differ across the four districts?

Example 3

• Does the number of citations issued per officer differ across the four districts?

citations |> anova_check(outcome = citations,
                         grouping = district)

Example 3

• Does the number of citations issued per officer differ across the four districts?

Example 3

• Does the number of citations issued per officer differ across the four districts?

Example 3

• Verifying the variance assumption,

citations |> multiple_variances(outcome = citations,
                                grouping = district)

Brown-Forsythe-Levene test for equality of variances

s²[Coldharbor] = 4.5766 (s = 2.1393)
s²[Fairview] = 8.7167 (s = 2.9524)
s²[Meadowbrook] = 3.3619 (s = 1.8335)
s²[Northgate] = 8.9677 (s = 2.9946)

Hypotheses:
H₀: All group variances are equal (no effect of district on variance)
H₁: At least one group variance differs (effect of district on variance exists)
Test statistic: F(3, 124) = 4.237
p-value: 0.007
Conclusion: Reject the null hypothesis (p = 0.007 < α = 0.050)

Example 3

• Hypotheses

  • H_0: \sigma^2_{\text{M}} = \sigma^2_{\text{C}} = \sigma^2_{\text{N}} = \sigma^2_{\text{F}}
  • H_1: at least one district is different

• Test Statistic and p-Value

  • F_0 = 4.237
  • p = 0.007

• Rejection Region

  • Reject H_0 if p < \alpha; \alpha = 0.05

• Conclusion and Interpretation

  • Reject H_0. There is sufficient evidence to suggest that there is a difference in the variance of citations between the districts.

Wrap Up

• This lecture introduced assessing assumptions for one-way ANOVA.

  • Normality: graph only

  • Variance: graph and test available

• However, we have not yet discussed what to do about a broken assumption…

• In the future:

  • Nonparametric alternative
  • Posthoc testing