Correlation

Introduction

• Before today, we discussed methods for comparing continuous outcomes across two or more groups.

• We now will begin exploring the relationships between two continuous variables.

• We will first focus on data visualization and the corresponding correlation.

• The we will quantify the relationship using regression analysis.

Scatterplot: Introduction

• Scatterplot or scatter diagram:

  • A graph that shows the relationship between two quantitative variables measured on the same subject.

• Each individual in the dataset is represented by a point on the scatterplot.

• The explanatory variable is on the x-axis and the response variable is on the y-axis.

• It is super important for us to plot the data!

  • Plotting the data is a first step in identifying issues or potential relationships.

Scatterplot: Introduction

• Positive relationship: As x increases, y increases.

• Negative relationship: As x increases, y decreases.

Scatterplot (R)

• Like we have done before, we will graph using the ggplot() function from library(tidyverse) (or library(ggplot2)).

dataset_name %>% ggplot(aes(x = x_variable, y = y_variable)) + # specify x and y
  geom_point() + # plot points
  labs(x = "x-axis label", # edit x-axis label
       y = "y-axis label") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example Set Up

• In Ponyville, Pinkie Pie is curious about how Gummy’s snack satisfaction (0 to 100) relates to the duration of chew time (in seconds) he spends on crunchy treats. She suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

Scatterplot: Example

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• How should we update the code for a scatterplot?

dataset_name %>% ggplot(aes(x = x_variable, y = y_variable)) + # specify x and y
  geom_point() + # plot points
  labs(x = "x-axis label", # edit x-axis label
       y = "y-axis label") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• How should we update the code for a scatterplot?

gummy_data %>% ggplot(aes(x = chew_time, y = satisfaction)) + # specify x and y
  geom_point() + # plot points
  labs(x = "Chew Time (minutes)", # edit x-axis label
       y = "Gummy's Satisfaction") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• Running the code,

Scatterplot: Example

• What if we switch what we put on the x- and y-axes?

gummy_data %>% ggplot(aes(y = chew_time, x = satisfaction)) + # specify x and y
  geom_point() + # plot points
  labs(y = "Chew Time (minutes)", # edit y-axis label
       x = "Gummy's Satisfaction") + # edit x-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• What if we switch what we put on the x- and y-axes?

Scatterplot: Example Set Up

• Fluttershy wants to determine how the amount of carrots Angel Bunny is given (grams) affects his happiness level (0 to 100). She believes that Angel Bunny tends to be happiest with a moderate amount of carrots.

• Fluttershy records both the weight of the carrot (carrot_weight) and Angel Bunny’s happiness level (happiness) of the last 200 snacks given to Angel Bunny (angel_data).

Scatterplot: Example

• Fluttershy believes that Angel Bunny tends to be happiest with a moderate amount of carrots.

• Fluttershy records both the weight of the carrot (carrot_weight) and Angel Bunny’s happiness level (happiness) of the last 200 snacks given to Angel Bunny (angel_data).

• How should we update the code for a scatterplot?

dataset_name %>% ggplot(aes(x = x_variable, y = y_variable)) + # specify x and y
  geom_point() + # plot points
  labs(x = "x-axis label", # edit x-axis label
       y = "y-axis label") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• Fluttershy believes that Angel Bunny tends to be happiest with a moderate amount of carrots.

• Fluttershy records both the weight of the carrot (carrot_weight) and Angel Bunny’s happiness level (happiness) of the last 200 snacks given to Angel Bunny (angel_data).

• Our updated code,

angel_data %>% ggplot(aes(x = carrot_weight, y = happiness)) + # specify x and y
  geom_point() + # plot points
  labs(x = "Carrot Weight (grams)", # edit x-axis label
       y = "Happiness") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• Running the code,

Scatterplot: Example

• What if we switch the x- and y-axes?

angel_data %>% ggplot(aes(y = carrot_weight, x = happiness)) + # specify x and y
  geom_point() + # plot points
  labs(y = "Carrot Weight (grams)", # edit x-axis label
       x = "Happiness") + # edit y-axis label
  theme_bw() # change theme of graph

Scatterplot: Example

• What if we switch the x- and y-axes?

Correlation: Introduction

• Creating the scatterplot allows us to visualize a potential relationship.

  • e.g., As chew time increases, Gummy’s satisfaction increases.
  • e.g., As carrot weight increaes, Angel Bunny’s happiness increases to a point (~60 grams). After that, as carrot weight increases, Angel Bunny’s happiness decreases.

• Now, let’s discuss quantifying that relationship.

  • Initial quantification: correlation.

  • Further quantification: regression.

Correlation: Definition

• Correlation: A unitless measure of the strength and direction of the linear relationship between two quantitative variables.

  • \rho represents the population correlation coefficient.

  • r represents the sample correlation coefficient.

• Correlation is bounded to [-1, 1].

  • r=-1 represents perfect negative correlation.

  • r=1 represents perfect positive correlation.

  • r=0 represents no correlation.

Correlation: Pearson’s Correlation

• Pearson’s correlation coefficient:

r = \frac{\sum_{i=1}^n \left( \frac{x_i - \bar{x}}{s_x} \right)\left( \frac{y_i - \bar{y}}{s_y} \right)}{n-1}

• x_i is the i^th observation of x; y_i is the i^th observation of y
• \bar{x} is the sample mean of x; \bar{y} is the sample mean of y
• s_x is the sample standard deviation of x; s_y is the sample standard deviation of y
• n is the sample size (number of paired observations)

Correlation: Pearson’s Correlation (R)

• We will use the correlation() function from library(ssstats) to examine correlation.

• For a single pairwise correlation,

dataset_name %>% correlation(x = variable_1, y = variable_2)

• For all pairwise correlations,

dataset_name %>% correlation()

Correlation: Pearson’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• How should we update the code for the correlation between satisfaction level and the chew time?

dataset_name %>% correlation(x = variable_1, y = variable_2)

Correlation: Pearson’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Our updated code,

gummy_data %>% correlation(x = chew_time, y = satisfaction)

Correlation: Pearson’s Correlation

• Running the code,

gummy_data %>% correlation(x = chew_time, y = satisfaction)

Correlation: Pearson’s Correlation

• What happens if we switch x and y?

gummy_data %>% correlation(x = satisfaction, y = chew_time)

Correlation: Pearson’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• How should we update the following code to get the correlation matrix?

dataset_name %>% correlation()

Correlation: Pearson’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Our updated code,

gummy_data %>% correlation()

Correlation: Pearson’s Correlation

• Running the code,

gummy_data %>% correlation()

Hypothesis Testing: Pearson’s Correlation

• We can determine if the correlation is significantly different from 0 (i.e., a relationship exists)

• Hypotheses:

  • H_0: \ \rho_{\text{P}} = 0
  • H_1: \ \rho_{\text{P}} \ne 0

• Test Statistic and p-Value

  • t_0 = \frac{r \sqrt{n-2}}{\sqrt{1-r^2}}
  • p = 2 \times P[t_{n-2}\ge t_0]

• This is default output in our correlation() function.

Hypothesis Testing: Pearson’s Correlation

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Looking at correlation results for this specific relationship,

Hypothesis Testing: Pearson’s Correlation

• Hypotheses:
  • H_0: \ \rho_{\text{P}} = 0
  • H_1: \ \rho_{\text{P}} \ne 0
• Test Statistic and p-Value
  • t_0 = 29.46
  • p < 0.001
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha=0.05.
• Conclusion and Intepretation
  • Reject H_0. There is sufficient evidence to suggest that the correlation is different from zero. That is, a relationship exists between time chewed and Gummy’s satisfaction.

Correlation: Pearson’s Assumptions (R)

• The assumption for Pearson’s correlation is that both x and y are normally distributed.

• We will use the correlation_qq() function from library(ssstats) to examine the normality of x and y.

dataset_name %>% correlation_qq(x = x_variable, y = y_variable)

Correlation: Pearson’s Assumptions

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Let’s now check the assumption that both satisfaction level and the chew time are normally distributed. How should we change the following code?

dataset_name %>% correlation_qq(x = x_variable, y = y_variable)

Correlation: Pearson’s Assumptions

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Let’s now check the assumption that both satisfaction level and the chew time are normally distributed. Our updated code,

gummy_data %>% correlation_qq(x = chew_time, y = satisfaction)

Correlation: Pearson’s Assumptions

• Running the code,

gummy_data %>% correlation_qq(x = chew_time, y = satisfaction)

Correlation: Spearman’s Correlation

• What do we do when we do not meet the normality assumption?

• Spearman’s Correlation: A unitless measure of the strength and direction of the monotone relationship between two variables.

  • Spearman’s correlation only assumes that the data is ordinal.
  • It does not work for nominal data!

• Spearman’s correlation is interpreted the same as Pearson’s correlation.

• To find Spearman’s correlation, the following algorithm is followed:

  1. Rank the x and y values.
     • (x, y) \to (R_x, R_y)
  2. Find Pearson’s correlation for the ranked data.

Correlation: Spearman’s Correlation (R)

• We will use the correlation() function from library(ssstats) to examine correlation.

• For a single pairwise correlation,

dataset_name %>% correlation(x = variable_1, y = variable_2, 
                             method = "spearman")

• For all pairwise correlations,

dataset_name %>% correlation(method = "spearman")

Correlation: Spearman’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• How should we update the code for the Spearman correlation between satisfaction level and the chew time?

dataset_name %>% correlation(x = variable_1, y = variable_2, 
                             method = "spearman")

Correlation: Spearman’s Correlation

• Pinkie Pie suspects that Gummy enjoys snacks more the longer he chews them.

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Our updated code,

gummy_data %>% correlation(x = chew_time, y = satisfaction, 
                           method = "spearman")

Correlation: Spearman’s Correlation

• Running the code,

gummy_data %>% correlation(x = chew_time, y = satisfaction, 
                           method = "spearman")

Hypothesis Testing: Spearman’s Correlation

• We can determine if the correlation is significantly different from 0 (i.e., a relationship exists)

• Hypotheses:

  • H_0: \ \rho_{\text{S}} = 0
  • H_1: \ \rho_{\text{S}} \ne 0

• Test Statistic and p-Value

  • t_0 = \frac{r \sqrt{n-2}}{\sqrt{1-r^2}}
  • p = 2 \times P[t_{n-2}\ge t_0]

• This is default output in our correlation() function.

Hypothesis Testing: Spearman’s Correlation

• Pinkie Pie records both the chew time (chew_time) and Gummy’s satisfaction level (satisfaction) of the last 300 snacks given to Gummy (gummy_data).

• Looking at Spearman’s correlation results for this specific relationship,

Hypothesis Testing: Spearman’s Correlation

• Hypotheses:
  • H_0: \ \rho_{\text{S}} = 0
  • H_1: \ \rho_{\text{S}} \ne 0
• Test Statistic and p-Value
  • t_0 = 570768
  • p < 0.001
• Rejection Region
  • Reject H_0 if p < \alpha; \alpha=0.05.
• Conclusion and Intepretation
  • Reject H_0. There is sufficient evidence to suggest that the correlation is different from zero. That is, a relationship exists between time chewed and Gummy’s satisfaction.

Correlation: Pearson vs. Spearman

• Comparing the two side-by-side,

gummy_data %>% correlation(x = chew_time, y = satisfaction, 
                           method = "pearson")

gummy_data %>% correlation(x = chew_time, y = satisfaction, 
                           method = "spearman")

Wrap Up: Correlation

• Correlation allows us to quantify the relationship between two variables without units.
  • It does not matter if x and y have the same units! Correlation is unitless.
• The closer to -1 or 1, the stronger the relationship.
  • As correlation approaches -1 or 1, x is better at predicting y.
  • As correlation approaches 0, it suggests y is constant as x changes.
• Remember that Pearson’s assumes normality of both x and y. If that is broken, we can use Spearman’s instead.
  • Sometimes r_{\text{P}} > r_{\text{S}}, while other times r_{\text{P}} < r_{\text{S}}.

Wrap Up

• Today we have discussed describing the relationship between two continuous variables.
  • Scatterplots allow us to visualize the relationship.
  • Correlation allows us to quantify the strength and direction of the relationship.
  • Regression allows us to model the relationship, quantify the change in y as x changes, and make predictions.
• Next class meeting:
  • Lab & quiz: Correlation
• Topics for next week:
  • Linear regression.
  • Inference on linear regression.
  • Visualization of models.