Two-Sample Means

Introduction: Topics

• We previously have reviewed estimation and basic statistical inference.
  • Point estimation
  • Confidence intervals
  • Hypothesis testing
• Today we will focus on the following scenarios today:
  • Two-sample means, independent data (\mu_1-\mu_2)
  • Two-sample means, dependent data (\mu_d)

Introduction: Data

• The Wonderbolts have opened a new Reserve Camp, where 30 cadets spend two weeks training for a shot at the main squad.

• On arrival, each cadet is assigned to one of two training squadrons:

  • Thunder Squadron
  • Lightning Squadron

• The two squadrons train separately and compete against each other in a series of trials.

  • Independent data

• Every cadet is also measured once at the start of camp and once again at the end, after two weeks of identical drills.

  • Dependent data

Introduction: Data

• Eyeballing the dataset,

Independent Data

• Examples:
  • Comparing the cruising speeds of a random sample of Pegasi vs. a random sample of Unicorns flying a short course.
  • Measuring friendship lesson quiz scores for a group of Cutie Mark Crusaders vs. a group of Wonderbolts Cadets.
  • Examining the graduation rates between Unicorns and Pegasi.

Dependent Data

• Examples:
  • Students’ magic‐proficiency scores before and after Princess Celestia’s advanced spell workshop.
  • Applejack’s apple‐yield (in bushels) from Sweet Apple Acres in Spring vs. Fall for the last 10 years.
  • Comparing the “Wonderbolts Tryouts” performance scores for Spitfire and Skyflare (twins).

Independent vs. Dependent Data

• Are the following dependent or independent?
  1. Rainbow Dash times two separate groups, Pegasi trainees and Unicorn cadets, on the same 200-meter aerial course.
  2. Twilight Sparkle measures her own spell‐casting accuracy before and after attending Princess Celestia’s advanced magic workshop.
     
  3. Applejack records bushel counts from Sweet Apple Acres in spring this year and compares them to bushel counts from Sugarcube’s orchard over the same period.
     
  4. The Cutie Mark Crusaders each take a friendship-lesson quiz, and their scores are compared to a completely different group of ponies at the School of Friendship.
     
  5. Fluttershy records the heart rates of the same group of critters before and after she plays soothing music for them.

Confidence Intervals: Two Independent Means

(1-\alpha)100\% confidence interval for \mu_1-\mu_2:

(\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2} \sqrt{\frac{s_1^2 }{n_1} + \frac{s_2^2}{n_2}}

• where
  • (\bar{x}_1-\bar{x}_2) is the point estimate for \mu_1-\mu_2
    • \bar{x}_i is the sample mean for group i
  • t_{\alpha/2} has \min(n_1-1, n_2-1) degrees of freedom
  • s_i^2 is the sample variance for group i
  • n_i is the sample size of group i

Hypothesis Testing: Two Independent Means

• Hypotheses: Two Tailed
  • H_0: \ \mu_1-\mu_2=\mu_0
  • H_1: \ \mu_1-\mu_2 \ne \mu_0
• Hypotheses: Left Tailed
  • H_0: \ \mu_1-\mu_2 \ge \mu_0
  • H_1: \ \mu_1-\mu_2 < \mu_0
• Hypotheses: Right Tailed
  • H_0: \ \mu_1-\mu_2 \le \mu_0
  • H_1: \ \mu_1-\mu_2 > \mu_0

Hypothesis Testing: Two Independent Means

Test Statistic:

t_0 = \frac{(\bar{x}_1-\bar{x}_2)-\mu_0}{{\sqrt{\frac{s_1^2}{n_1} + \frac{s^2_2}{n_2}}}} \sim t_{\text{df}}

• where
  • \bar{x}_i is the mean for group i
  • \mu_0 is the hypothesized difference
  • s_i^2 is the sample variance for group i
  • n_i is the sample size of group i
    
  • \text{df} = \text{min}(n_1-1, n_2-1)

Two Independent Means (R)

• We will use the independent_means function from library(ssstats) to perform statistical inference.

• Generic syntax:

1dataset_name |> independent_means(
2  outcome = outcome_variable,
3  grouping = grouping_variable,
4  alternative = "two" | "less" | "greater",
5  mu = hypothesized_difference,
6  alpha = alpha_level
  ) 

Example 1

• Every cadet runs a timed sprint course as part of camp trials.

• The coaches want us to determine if the Thunder Squadron and Lightning Squadron are finishing the sprint course at different times.

• Looking at the data,

Example 1

• The coaches want us to determine if the Thunder Squadron and Lightning Squadron are finishing the sprint course at different times.

• What is the direction of the test? How do you know?
  
  
  

• What is the hypothesized value? How do you know?
  
  
  

• What are the corresponding hypotheses?

Example 1

• The coaches want us to determine if the Thunder Squadron and Lightning Squadron are finishing the sprint course at different times.

• We will use the independent_means() function,

wonderbolt_camp |> independent_means(outcome = sprint_time_sec,
                                     grouping = squadron, 
                                     alternative = "two",
                                     mu = 0,
                                     alpha = 0.05) 

Example 1

• Running the code,

Independent means (Welch's t-test)

x̅[Lightning Squadron] = 44.4643
x̅[Thunder Squadron] = 41.8188
Point estimate: x̅[Lightning Squadron] − x̅[Thunder Squadron] = 2.6455

95% CI for μ[Lightning Squadron] − μ[Thunder Squadron]: (0.3719, 4.9192)

Hypotheses:
H₀: μ[Lightning Squadron] − μ[Thunder Squadron] = 0
H₁: μ[Lightning Squadron] − μ[Thunder Squadron] ≠ 0
Test statistic: t(27.81) = 2.384
p-value: 0.024
Conclusion: Reject the null hypothesis (p = 0.024 < α = 0.050)

Example 1

• Hypotheses:
  • H_0: \ \mu_{\text{L}} - \mu_{\text{T}} = 0
  • H_1: \ \mu_{\text{L}} - \mu_{\text{T}} \ne 0
• Test Statistic and p-Value
  • t_0 = 2.384, p = 0.024
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha = 0.05
• Conclusion and interpretation
  • Reject H_0. There is sufficient evidence to suggest that there is a difference between the Lightning and Thunder squadrons.

Example 1

• We are also given summary information from the output.

• In particular, we saw that

\bar{y}_{\text{L}} = 44.46 \ \ \text{\&} \ \ \bar{y}_{\text{T}} = 41.82

• Then, our point estimate for \mu_{\text{L}} - \mu_{\text{T}} is 2.65.

• How good is this estimate? Let’s examine the 95% CI for \mu_{\text{L}} - \mu_{\text{T}}:

(0.37, 4.92)

Example 2

• At camp, judges score each cadet’s aerial loop maneuver on a 0–100 accuracy scale during trials.

• Based on her gut feeling, the head judge believes that, on average, the Thunder Squadron is performing more accurate loop maneuvers.

• Looking at the data,

Example 2

• Based on her gut feeling, the head judge believes that, on average, the Thunder Squadron is performing more accurate loop maneuvers.

• What is the direction of the test? How do you know?
  
  
  

• What is the hypothesized value? How do you know?
  
  
  

• What are the corresponding hypotheses?

Example 2

• A quick note about R and subtraction!

• Remember that we are estimating \mu_1 - \mu_2 – how does R determine 1 and 2?

• R is going to subtract in alphabetical order.

  • In our case, with “Thunder” & “Lightning”, we are estimating \mu_{\text{L}} - \mu_{\text{T}}

• If the Thunder Squadron is more accurate \to higher score \to

  • H_0: \mu_{\text{L}} \ge \mu_{\text{T}}
  • H_1: \mu_{\text{L}} < \mu_{\text{T}}

• When specifying the direction in R, we will use the less option.

Example 2

• Based on her gut feeling, the head judge believes that, on average, the Thunder Squadron is performing more accurate loop maneuvers.

• We will use the independent_means() function,

wonderbolt_camp |> independent_means(outcome = loop_accuracy_pct,
                                     grouping = squadron, 
                                     alternative = "less",
                                     mu = 0,
                                     alpha = 0.05) 

Example 2

• Running the code,

Independent means (Welch's t-test)

x̅[Lightning Squadron] = 78.4071
x̅[Thunder Squadron] = 81.2875
Point estimate: x̅[Lightning Squadron] − x̅[Thunder Squadron] = -2.8804

95% CI for μ[Lightning Squadron] − μ[Thunder Squadron]: (-7.606, 1.8453)

Hypotheses:
H₀: μ[Lightning Squadron] − μ[Thunder Squadron] ≥ 0
H₁: μ[Lightning Squadron] − μ[Thunder Squadron] < 0
Test statistic: t(25.18) = -1.255
p-value: 0.111
Conclusion: Fail to reject the null hypothesis (p = 0.111 ≥ α = 0.050)

Example 2

• Hypotheses:
  • H_0: \ \mu_{\text{L}} - \mu_{\text{T}} \ge 0
  • H_1: \ \mu_{\text{L}} - \mu_{\text{T}} < 0
• Test Statistic and p-Value
  • t_0 = -1.255, p = 0.111
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha = 0.05
• Conclusion and interpretation
  • Fail to reject H_0. There is sufficient evidence to suggest that there is a difference between the Lightning and Thunder squadrons.

Example 2

• We are also given summary information from the output.

• In particular, we saw that

\bar{y}_{\text{L}} = 78.41 \ \ \text{\&} \ \ \bar{y}_{\text{T}} = 81.29

• Then, our point estimate for \mu_{\text{L}} - \mu_{\text{T}} is -2.88.

• How good is this estimate? Let’s examine the 95% CI for \mu_{\text{L}} - \mu_{\text{T}}:

(-7.61, 1.85)

Confidence Intervals: Two Dependent Means

\mathbf{(1-\boldsymbol\alpha)100\%} confidence interval for \mathbf{\boldsymbol\mu_d}

\bar{d} \pm t_{\alpha/2} \frac{s_d}{\sqrt{n}}

• where
  • \bar{d} = \text{mean}(x_1-x_2) is the point estimate for \mu_d = \mu_1-\mu_2
  • t_{\alpha/2} has n-1 degrees of freedom
  • s_d is the sample standard deviation of the difference
  • n is the number of pairs of observations

Hypothesis Testing: Two Dependent Means

• Hypotheses: Two Tailed
  • H_0: \ \mu_d=\mu_0
  • H_1: \ \mu_d \ne \mu_0
• Hypotheses: Left Tailed
  • H_0: \ \mu_d \ge \mu_0
  • H_1: \ \mu_d < \mu_0
• Hypotheses: Right Tailed
  • H_0: \ \mu_d \le \mu_0
  • H_1: \ \mu_d > \mu_0
• Note! \mu_d = \mu_1 - \mu_2

Hypothesis Testing: Two Dependent Means

Test statistic:

t_0 = \frac{\bar{d}-\mu_0}{\frac{s_d}{\sqrt{n}}} \sim t_{\text{df}}

• where
  • \bar{d} = \text{mean}(x_1-x_2) is the point estimate for \mu_d = \mu_1-\mu_2
  • \mu_0 is the hypothesized difference
  • s_d is the sample standard deviation of the difference
  • n is the number of pairs of observations
  • \text{df} = n-1

Two Dependent Means (R)

• We will use the dependent_means function from library(ssstats) to perform statistical inference.

• Generic syntax:

1dataset_name |> dependent_means(
2  col1 = first_variable,
3  col2 = second_variable,
4  alternative = "two" | "less" | "greater",
5  mu = hypothesized_difference,
6  alpha = alpha_level
  ) 

Example 3

• Each cadet’s aerial accuracy is scored when they arrive at camp, and scored again at the end after two weeks of drills.

• Coaches want to know whether cadets are actually flying more accurately by the end of camp.

• Looking at the data,

Example 3

• Coaches want to know whether cadets are actually flying more accurately by the end of camp.

• What is the direction of the test? How do you know?
  
  
  

• What is the hypothesized value? How do you know?
  
  
  

• What are the corresponding hypotheses?

Example 3

• Coaches want to know whether cadets are actually flying more accurately by the end of camp.

• We will use the dependent_means() function,

wonderbolt_camp |> dependent_means(col1 = accuracy_score_post,
                                   col2 = accuracy_score_pre,
                                   alternative = "greater", 
                                   mu = 0,
                                   alpha = 0.05) 

Example 3

• Running the code,

Paired (dependent) means

Complete pairs: 30
Point estimate: x̅[accuracy_score_post] − x̅[accuracy_score_pre] = 6.7267
SD of differences: 5.1583

95% CI for μ[accuracy_score_post] − μ[accuracy_score_pre]: (4.8005, 8.6528)

Hypotheses:
H₀: μ[accuracy_score_post] − μ[accuracy_score_pre] ≤ 0
H₁: μ[accuracy_score_post] − μ[accuracy_score_pre] > 0
Test statistic: t(29) = 7.143
p-value: < 0.001
Conclusion: Reject the null hypothesis (p = < 0.001 < α = 0.050)

Example 3

• Hypotheses:
  • H_0: \ \mu_{\text{post}} - \mu_{\text{pre}} \le 0
  • H_1: \ \mu_{\text{post}} - \mu_{\text{pre}} > 0
• Test Statistic and p-Value
  • t_0 = 7.143, p < 0.001
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha = 0.05
• Conclusion and interpretation
  • Reject H_0. There is sufficient evidence to suggest that there is a difference between the Lightning and Thunder squadrons.

Example 3

• We are also given summary information from the output.

• In particular, we saw that our point estimate for \mu_{\text{post}} - \mu_{\text{pre}} is 6.73.

• How good is this estimate? Let’s examine the 95% CI for \mu_{\text{post}} - \mu_{\text{pre}}:

(4.80, 8.65)

Example 4

• Each cadet’s recovery time after a hard trial is timed at the start of camp and again at the end.

• Coaches want to know whether cadets are recovering faster by the end of camp.

• Looking at the data,

Example 4

• Coaches want to know whether cadets are recovering faster by the end of camp.

• What is the direction of the test? How do you know?
  
  
  

• What is the hypothesized value? How do you know?
  
  
  

• What are the corresponding hypotheses?

Example 4

• Coaches want to know whether cadets are recovering faster by the end of camp.

• We will use the dependent_means() function,

wonderbolt_camp |> dependent_means(col1 = recovery_time_min_post,
                                   col2 = recovery_time_min_pre,
                                   alternative = "less", 
                                   mu = 0,
                                   alpha = 0.05) 

Example 4

• Running the code,

Paired (dependent) means

Complete pairs: 30
Point estimate: x̅[recovery_time_min_post] − x̅[recovery_time_min_pre] = -3.26
SD of differences: 3.3987

95% CI for μ[recovery_time_min_post] − μ[recovery_time_min_pre]: (-4.5291, -1.9909)

Hypotheses:
H₀: μ[recovery_time_min_post] − μ[recovery_time_min_pre] ≥ 0
H₁: μ[recovery_time_min_post] − μ[recovery_time_min_pre] < 0
Test statistic: t(29) = -5.254
p-value: < 0.001
Conclusion: Reject the null hypothesis (p = < 0.001 < α = 0.050)

Example 4

• Hypotheses:
  • H_0: \ \mu_{\text{post}} - \mu_{\text{pre}} \ge 0
  • H_1: \ \mu_{\text{post}} - \mu_{\text{pre}} < 0
• Test Statistic and p-Value
  • t_0 = -5.254, p < 0.001
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha = 0.05
• Conclusion and interpretation
  • Reject H_0. There is sufficient evidence to suggest that cadets are recovering faster at the end of camp.

Example 4

• We are also given summary information from the output.

• In particular, we saw that our point estimate for \mu_{\text{post}} - \mu_{\text{pre}} is -3.26.

• How good is this estimate? Let’s examine the 95% CI for \mu_{\text{post}} - \mu_{\text{pre}}:

(-4.53, -1.99)

Let’s Practice! 1

• Spectators toss bits to cadets who pull off a standout run during trials.

• Camp organizers want to know whether one squadron is out-earning the other in prize bits.

• Looking at the data,

Let’s Practice! 2

• Spectators tip cadets with bits after their runs. The amount tipped is recorded once early in camp and once again near the end.

• Camp organizers want to know whether cadets are earning more in tips as they improve over camp.

• Looking at the data,

Let’s Practice! 3

• A sound crew records the peak crowd noise (in decibels) during each cadet’s run.

• Camp organizers are curious whether one squadron draws a louder crowd reaction than the other.

• Looking at the data,

Let’s Practice! 4

• The camp medic tracks how many days each cadet spends recovering from an injury over the two-week camp.

• Coaches want to know whether one squadron’s training style is leading to more injury days than the other’s.

• Looking at the data,

Let’s Practice! 5

• Each cadet’s wing-flap rate while hovering in place is measured on arrival, then measured again after two weeks of drills.

• Coaches want to know whether training actually changes a cadet’s wing-flap rate.

• Looking at the data,

Let’s Practice! 6

• Each cadet rates how confident they feel about their flying on a self-reported scale, once at the start of camp and once at the end.

• Camp staff want to know whether two weeks of training changes how confident cadets feel.

• Looking at the data,

Wrap Up

• Today’s lecture:
  • Independent t-test
  • Dependent t-test
• Wednesday: R lab – bring a laptop!