Linear Models for Linear Effects

• Linear regression fits a straight line through data
• Assumes constant effect: each unit increase in \(X\) changes \(Y\) by the same amount
• Works when the relationship is linear

\[ Y = \beta_0 + \beta_1X + \epsilon \]

Linear Relationships in the Real World

But Real Data Isn’t Always Linear

• Growth curves (exponential, logistic)
• Binary outcomes (survived/died, yes/no)
• Count data (number of events)
• Bounded outcomes (proportions, rates)
• Curved relationships (U-shaped, S-shaped)

Probability of Disease by Age

Fitting a Straight Line Fails

The Problem is the Scale

• Linear regression assumes outcomes can take any value
• But many outcomes are constrained:
  • Probabilities must be between 0 and 1
  • Counts must be non-negative integers
  • Proportions are bounded
• Forcing a straight line violates these constraints

The Core Insight

• Don’t force data onto a straight line
• Transform the scale so linear regression works
• Fit the line on the transformed scale
• Transform back to get predictions
• This is what link functions do.

How Link Functions Work

• The link function connects the linear predictor to the outcome:

  1. Take your outcome \(Y\) (bounded, binary, count, etc.)
  2. Apply a transformation that “stretches” or “squashes” the scale
  3. Fit a linear model on the transformed scale
  4. Transform predictions back to the original scale

The Logit Link for Binary Outcomes

• For binary outcomes (0 or 1), we model the log-odds:

\[ \log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1X \]

• Left side: log-odds (can be any value)

• Right side: linear predictor (can be any value)

• Transform back to get probability: \(p = \frac{1}{1 + e^{-(\beta_0 + \beta_1X)}}\)

• This is logistic regression.

Visualising the Transformation

Logistic Regression Fits an S-Curve

Different Outcomes Need Different Links

• Binary outcomes → Logit link → Logistic regression

• Count data → Log link → Poisson regression

• Continuous positive → Log link → Log-linear models

• Proportions → Logit link → Beta regression

• All follow the same pattern: transform, fit linear model, transform back.

Poisson Regression Uses Log Link

• Model the log of the expected count:

\[ \log(\text{E}[Y]) = \beta_0 + \beta_1X \]

• Left side: log of expected count (can be any value)
• Right side: linear predictor (can be any value)
• Transform back: \(\text{E}[Y] = e^{\beta_0 + \beta_1X}\)
• Predictions are always positive, matching count data.

What We’ve Learned

• Real data often doesn’t fit straight lines.
• Link functions transform non-linear problems into linear ones.
• Logistic regression (logit link) handles binary outcomes.
• Poisson regression (log link) handles count data.
• Linear regression isn’t limited to linear relationships.

The Power of This Approach

• Simplicity - Same core idea (fit a line) works everywhere.
• Flexibility - Adapts to different data types and structures.
• Interpretability - Coefficients still represent effects.
• Extensibility - Once you know linear regression, you can learn anything.

Further Learning

• StatQuest (Josh Starmer) - Logistic Regression
• Andrew Gelman et al. - Regression & Other Stories

Thank You!

Contact:

Code & Slides:

/NHS-South-Central-and-West/code-club

… And don’t forget to give us your feedback.