Probability Distributions

Introduction

• We covered the ground rules:

  • probabilities are between 0 and 1, and

  • all outcomes sum to 1.

• We covered probability basics:

  • complement

  • independence

  • conditional probability

• Now, we zoom out: instead of asking about one event at a time, we’ll describe probability for any possible event.

Probability Distributions

• Recall from last lecture: a random experiment has a sample space, or the set of everything that could happen

• Two requirements:
  • Every probability is between 0 and 1
  • All the probabilities together add up to 1

Discrete Variables

Before we can describe a variable’s distribution, we have to know what kind of variable it is.

Example: Discrete Variables

• Example: the ZPD role of each recruit

Example: Discrete Variables

• Example: recruits’ exam score categorization

Continuous Variables

Before we can describe a variable’s distribution, we have to know what kind of variable it is.

Example: Continuous Variables

• Example: recruits’ exam score

Example: Continuous Variables

• Example: side hustle earnings

Probability Mass Function (PMF)

• We can formally describe the probability using a function.

• i.e., the PMF is just the bar chart in function form.

  • Pick any value of the variable and the PMF tells you the corresponding probability.

Probability Density Function (PDF)

• We can formally describe the probability using a function.

• i.e., the PDF is just the density plot in function form. However,

  • The curve itself is not a probability – we must find the probability by calculating the area under the curve for a range of values.

  • The total area under the whole curve is 1

Cumulative Distribution Function (CDF)

Both discrete and continuous variables share the cumulative distribution function (CDF).

• The CDF is a running total as you move left to right – it is the cumulative probability

PMF vs CDF

• Comparing the PMF of ZPD assignment to the CMF:

PDF vs CDF

• Comparing the PDF of academy exam scores to the CDF:

Binomial Experiments

• Deputy Chief Okonkwo wants to know:

  • “If I pull n recruits at random, how many will be Patrol?”

• This is a binomial experiment:

  1. The experiment consists of n identical trials (pulling n recruits at random).
  2. Each trial has two possible outcomes: “success” (Patrol) or “failure” (Desk).
  3. The probability of success (\pi) is the same for each trial (0.72).
  4. The trials are independent (the duty assignment of one recruit does not affect the duty assignment of another).
  5. The random variable y is the number of successes (recruits on Patrol) in n trials.

The Binomial Distribution

• We use the binomial distribution to describe the probabilities of binomial experiments,

• If y has a binomial distribution, we say y \sim Bin(n, \pi).

The Binomial Distribution

• The probability of observing y successes in n trials of a binomial experiment is given by

P(y) = \frac{n!}{y!(n-y)!} \pi^y (1-\pi)^{n-y}

• n = number of trials,
• y = number of successes,
• \pi = probability of success on a single trial

The Binomial Distribution

• Deputy Chief Okonkwo wants to know:

  • “If I pull n recruits at random, how many will be Patrol?”

• This question asks us to count the number of “successes” (e.g., Patrol) out of a fixed number of tries

• The binomial distribution is appropriate here, as:

  • n (number of recruits in sample)
  • y (number of recruits in sample that are Patrol)
  • \pi (probability of an individual being Patrol)

The Binomial Distribution

• The shape of the binomial distribution depends on n and p.

• If we have more recruits or a different success rate, our expectations change.

The Normal Distribution

• Deputy Chief Okonkwo wants to know:
  • “How are academy exam scores distributed across all recruits?”
• The normal distribution is the right tool when:
  1. The random variable is continuous (exam scores can take any value along a scale).
  2. The distribution is symmetric, with values clustering around a central average.
  3. Values far from the average become progressively less likely, tapering off evenly in both directions.

The Normal Distribution

• We use the normal distribution to describe continuous variables that cluster symmetrically around an average,

• If y has a normal distribution, we say y \sim N(\mu, \sigma).

The Normal Distribution

• The probability density of a normal distribution at a value y is given by

f(y) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{y - \mu}{\sigma}\right)^2}

• \mu = population mean,
• \sigma = population standard deviation,
• y = a possible value of the variable

The Normal Distribution

• Deputy Chief Okonkwo wants to know:

  • “How are academy exam scores distributed across all recruits?”

• This question asks us to describe the pattern of a continuous variable around its average

• The normal distribution is appropriate here, with:

  • \mu (average academy exam score across recruits)
  • \sigma (spread of academy exam scores around that average)
  • y (a recruit’s individual exam score)

The Normal Distribution

• The shape of the normal distribution depends on \mu and \sigma.

• Note that \mu controls the location and \sigma controls the shape.

The Normal Distribution

• When we change \mu,

The Normal Distribution

• When we change \sigma,

The Normal Distribution

• Finally, we have the standard normal, where z \sim N(0, 1).

The t-Distribution

• Deputy Chief Okonkwo wants to know:

  • “Sahara Square only has 8 recruits — how confident can we be about their average sleep hours?”

• The t-distribution is the right tool when:

  1. The random variable is continuous (sleep hours can take “any” value).
  2. The sample size is small (“small” meaning n < 30).
  3. The population standard deviation (\sigma) is unknown, so we estimate it with the sample standard deviation (s).

• We will see that estimating \sigma with s adds uncertainty.

  • The t-distribution has heavier tails than the normal.

The t-Distribution

• We use the t-distribution to describe a continuous variable estimated from a small sample with unknown population spread,

The t-Distribution

• The probability density of a t-distribution at a value y is given by

f(y) = \frac{\Gamma\left(\frac{\nu+1}{2}\right)}{\sqrt{\nu\pi}\,\Gamma\left(\frac{\nu}{2}\right)} \left(1 + \frac{y^2}{\nu}\right)^{-\frac{\nu+1}{2}} - \nu = degrees of freedom

The t-Distribution

• Deputy Chief Okonkwo wants to know:

  • “Sahara Square only has 8 recruits — how confident can we be about their average sleep hours?”

• This question asks us to estimate a population average from a small sample where we don’t know the true population spread

• The t-distribution is appropriate here, with:

  • n (number of recruits in the Sahara Square sample: 8)
  • s (sample standard deviation of sleep hours)
  • \nu (degrees of freedom = n - 1 = 7 in this case)

The t-Distribution

• The shape of the t-distribution depends on its degrees of freedom (\nu).

• As \nu grows larger (more data), the t-distribution’s tails shrink and it converges toward the standard normal.

The t-Distribution

• Recall: learning probabilities using tables in a book

  • When you hit a “large enough” sample size, it goes to df = \infty and the resulting values are the same as the standard normal.

  • That is because the t-distribution converges to the standard normal as \nu \to \infty.

The \chi^2-Distribution

• Deputy Chief Okonkwo wants to know:

  • “ZPD claims academy exam scores have a standard deviation of 10 — is our recruits’ spread consistent with that?”

• The chi-squared distribution will be used in hypothesis testing, not for describing the shape of a variable itself.

  • We use it strictly as a tool for hypothesis testing.

  • We will see it across several topics, including one-sample tests for variance, two-sample tests for variance, and goodness-of-fit tests.

The \chi^2-Distribution

• We use the \chi^2-distribution to describe how sample variance behaves when testing a claim about population spread,

The \chi^2-Distribution

• The probability density of a chi-squared distribution at a value y is given by

f(y) = \frac{1}{2^{\nu/2}\Gamma\left(\frac{\nu}{2}\right)} y^{\frac{\nu}{2}-1} e^{-y/2}, \quad y \geq 0 - \nu = degrees of freedom

The \chi^2-Distribution

• The shape of the \chi^2-distribution depends on its degrees of freedom (\nu).

The \chi^2-Distribution

• Unlike the t-distribution, the \chi^2-distribution is not symmetric.
  • As \nu grows larger, it becomes more symmetric and bell-shaped.

The F-Distribution

• We use the F-distribution to describe how the ratio of two sample variances behaves when comparing spread across two groups,

The F-Distribution

• The probability density of an F-distribution at a value y is given by

f(y) = \frac{\sqrt{\frac{(\nu_1 y)^{\nu_1} \nu_2^{\nu_2}}{(\nu_1 y + \nu_2)^{\nu_1+\nu_2}}}}{y \, \text{B}\left(\frac{\nu_1}{2}, \frac{\nu_2}{2}\right)}, \quad y \geq 0

• \nu_1 = degrees of freedom, numerator
• \nu_2 = degrees of freedom, denominator

The F-Distribution

• The shape of the F-distribution depends on two degrees of freedom, \nu_1 and \nu_2.

The F-Distribution

• Like the \chi^2-distribution, the F-distribution is not symmetric and only takes non-negative values.

  • As \nu_1 and \nu_2 grow larger, it becomes more symmetric and bell-shaped.

Wrap Up

This lecture gave an overview of probability functions:

1. PMF: probability mass function; for discrete variables.

2. PDF: probability density function; for continuous variables.

3. CDF: cumulative distribution function; for discrete and continuous variables

and distributions:

1. Binomial: for counting successes in a fixed number of independent trials.

2. Normal: for continuous variables that cluster symmetrically around an average.

3. t: for continuous variables estimated from a small sample with unknown population standard deviation.

4. Chi-squared: (hypothesis testing)

5. F: (hypothesis testing)

Looking Forward

• In the next module, you will use these concepts to answer questions such as:

  • “ZPD claims their exam scores have a standard deviation of 10. You have real data. Is that claim believable?”

• We will continue to see these distributions when learning various hypothesis tests.

• Before we learn hypothesis testing, we will learn how to estimate population parameters from sample statistics.

  • Our next lectures cover data description using point estimation and graphical methods.