Regression Foundations
A reading guide for lecture p-07 — the two conditions bias requires, and why no amount of data fixes it.
Section one
The models, the formula, and the two cases, each stated as a claim.
Key concepts
Greek letters mean the truth; Latin letters mean your best guess.
Note the mapping the deck makes explicit: full model plays the role of a population statistic and naïve model the role of a sample statistic. But this is not sampling error — you can have the entire population and still be wrong if a variable is missing.
The main question.
If we have an omitted variable, will our estimate of program impact (b1) sufficiently represent the true program impact (β1)?
We will ALWAYS have omitted variables in observational studies. The real question is not whether it is there, but how much will it affect our estimates?
The three models.
The auxiliary regression puts the omitted variable on the left-hand side.
The take-away, in three lines.
Bias is the indirect effect.
The naïve slope absorbs everything that reaches Y through X1, including the part that really belongs to X2.
Two cases.
A worked example.
5.65 · −0.0099 = −0.056, and −0.433 − (−0.377) = −0.056. The two routes agree. In the multi-model table the bias reaches 51% — the naïve model overestimates program impact by half.
Section two
The mental map. How the pieces connect, and how to read the deck.
This is the whole lecture compressed into one line:
A product is zero if either factor is zero, so omitted variable bias requires both:
Fail either condition and the omission is harmless. This is why the deck can be relaxed about the fact that observational studies always omit something: most omitted variables are irrelevant on at least one of the two channels.
It also tells you what to worry about. The dangerous omitted variable is not the one most strongly related to your outcome — it is the one related to your outcome and to your treatment.
Because bias is a product of two slopes, its direction is the product of two signs. That is a genuinely useful skill, and it works even when the omitted variable is unmeasurable:
| α1 (X1→X2) | β2 (X2→Y) | Bias | Naïve slope is |
|---|---|---|---|
| + | + | + | too large |
| − | − | + | too large |
| + | − | − | too small |
| − | + | − | too small |
So even if you cannot fix the problem, you can often say which way your estimate is wrong — and therefore whether your finding survives. If you estimate a positive effect and reason that the bias is positive, your true effect is smaller than reported.
The deck’s warning attaches here: if the naïve slope is too large it can make it look significant when it’s not.
Put the two lectures side by side and they are the same diagram read in opposite directions:
| p-06: adding a control | p-07: omitting a variable | |
|---|---|---|
| Uncorrelated with X1 | shrinks residual, slope unchanged, SE smaller | no bias |
| Correlated with X1 | eats the overlap, slope changes, SE larger | bias |
Notice that the same variable is harmless in both directions or harmful in both directions. A control uncorrelated with your policy variable neither biases you when omitted nor distorts you when added — it only ever affects precision. A correlated one always does both.
That gives you the decision rule the two lectures jointly imply:
p-06 showed the cost. p-07 shows the reason to pay it. Together they are the accuracy-versus- precision trade from p-05, made concrete.
Slide 2 makes a point worth dwelling on. You are used to Greek-versus-Latin meaning population-versus-sample, and here it means full-model-versus-naïve. The analogy is deliberate but the mechanism is completely different.
Sampling error shrinks as n grows. Omitted variable bias does not. You can have census data on every unit in the population and still get the wrong slope, because the problem is not that you have too few rows — it is that you have too few columns.
This is the clearest illustration of the p-05 distinction: more data buys you efficiency, never unbiasedness. No sample size fixes a missing variable.
Slide 13 is honest about something most treatments gloss over: bias = b1 − β1 is the definition, and it is useless in practice — if you knew β1 you would not be estimating it.
What makes the lecture practical is the other route, bias = α1β2, because both of those can sometimes be reasoned about from theory, prior studies, or partial data even when the full model cannot be run. That is the difference between a definition and a tool.
If you keep one image from this deck, keep the triangle:
Every question about omitted variable bias reduces to: is there a second route from my treatment to my outcome, and how strong is it?
| Slides | What they are doing | What to take away |
|---|---|---|
| 2 | Notation | Greek = truth, Latin = best guess. Bias is not sampling error. |
| 3–4 | The main question | Omission is universal; magnitude is the real question. |
| 5–8 | p-06 recapped | The two control types, restated as a setup. |
| 9–11 | The two cases | Correlated → bias. Uncorrelated → none. |
| 12 | The multi-model table | Bias of 51% in the worked example. |
| 13 | The definition | b1 − β1. True, and unusable. |
| 14–17 | Direct and indirect effects | The path diagram. X2 goes on the left in the auxiliary regression. |
| 19 | Worked calculation | Both routes give −0.056. Check the arithmetic yourself. |
| 20 | The take-away | Three numbered lines. If you memorise one slide, this one. |
| 21 | Why it matters | Case 1 too large, Case 2 too small; significance can be manufactured. |
| 22–26 | When does OVB occur? | The two Venn cases, with the algebra beside each. |
This deck has no closing checklist. Slide 20 is the summary.
Section three
The deck ends on its take-away rather than a question list.