Hypothesis Testing with Dummy Variables

The specification decides which questions you can ask

This is the thesis, and it is a different point from anything earlier in the sequence.

Both models in this deck are the same model. Same four group means, same fitted values, same R², same residuals. Nothing about the description of the data changes when you switch between them.

What changes is which comparisons the software puts a p-value on. Every regression tests each coefficient against zero. So the parameterization you choose determines which contrasts get tested for free — and which ones you would have to compute by hand.

• The cell-means model tests four uninteresting hypotheses (is each group mean zero?).
• The reference-group model tests three interesting ones (is there an environment effect, a program effect, an interaction?).

Choose your reference group so the tested contrast is the one you care about

It follows that omitting a different group changes what the coefficients mean — and the deck shows exactly this. With urban TFA as the reference, b₀ = 66, b₁ = 9, b₂ = −9, b₃ = 9. With suburban regular as the reference, the same data give 75, −18, 0, 9.

Both recover all four group means. But they hand you different default tests.

So the practical workflow is backwards from what students usually do: decide which comparison answers your research question first, then pick the reference category that makes that comparison a single coefficient.

The TFA example is omitted variable bias, in table form

Before any regression appears, slides 3 through 13 have already made the argument from p-07 using nothing but arithmetic. Run it through bias = α₁β₂:

• α₁ ≠ 0: TFA status is correlated with school environment (6 of 10 TFA fellows are in urban schools; 7 of 10 regular teachers are in suburban schools).
• β₂ ≠ 0: school environment strongly affects scores (75 vs 57 for regular teachers).

Both channels are open, so omitting environment biases the program comparison. And here the bias is large enough to cancel the effect exactly — 69.6 versus 69.6.

Notice which direction it runs. This is the case p-05 warned about in reverse: not a false positive manufactured by bias, but a false negative. The naive analysis concludes the program does not work. A program that helps in urban schools gets defunded because more of its teachers are in urban schools.

Why the pooled comparison is not just noisy but wrong

It is tempting to read the 69.6 = 69.6 result as “the effects cancel out.” They do not cancel; they were never comparable in the first place.

The pooled TFA average is 40% suburban and 60% urban. The pooled regular average is 70% suburban and 30% urban. You are comparing two differently-weighted mixes of two different populations. The number is a weighted average of things that should not be averaged together.

This is the same structure as p-10’s heterogeneity bias and the blood-pressure reversal: the between-group comparison and the within-group comparison are different quantities, and only the second answers the causal question.

The interaction always tests “is the combination additive?”

Every example in the back half of the deck is the same test wearing different clothing:

In every case b₀ + b₁ + b₂ is the counterfactual — what you would expect if the two factors acted independently — and b₃ is the gap between that expectation and reality.

Once you see this, the difference-in-differences model from p-08 stops being a special technique and becomes an instance: DiD is just treatment × time, and the parallel-trends assumption is just the claim that the additive counterfactual is the right one.

The two mechanical rules, and why they hold

Why one dummy must be omitted. The full set of dummies for a factor sums to a column of ones, which is exactly what the intercept already is. Including both is perfect multicollinearity — the extreme end of p-10’s second sin — and the software will silently drop one for you. Better to choose which one.

Why same-variable dummies cannot be interacted. White × Black is zero for every observation, because no one is both. An interaction asks “what happens when both conditions hold,” and for mutually exclusive categories that never happens. Interactions are for combinations across factors, not within one.

How to read the slides

1. Explain why the pooled TFA comparison shows no effect. Selection into school environments, which is correlated with both program and outcome.
2. Write both design matrices for a 2×2 and recover all four group means from either.
3. Say why the cell-means model cannot test your hypothesis even though it displays the group means most clearly.
4. Translate a research question into a coefficient test by choosing the reference category.
5. State what an interaction tests — whether the joint effect exceeds the additive expectation.
6. Explain the two mechanical rules: omit one dummy per factor; never interact dummies from the same factor.

Key takeaways

• The design matrix determines which hypotheses get tested, not what the model fits. Both parameterizations describe the data identically.
• Cell-means coding shows group means but tests nothing useful. Reference-group coding tests contrasts you care about.
• Choose the omitted category to match your research question. Different reference groups give different default tests from the same data.
• You can always recover every group mean by multiplying coefficients through the appropriate design-matrix row.
• The TFA example is omitted variable bias producing a false negative — the program works, and the naive comparison says it does not.
• Between-group and within-group comparisons are different quantities. Only the second answers the causal question.
• Every interaction tests the same thing: is the combination additive, or is there something extra?
• Difference-in-differences is an interaction, and its counterfactual is the additive expectation.
• k levels give k−1 dummies, and dummies from one factor can never be interacted with each other.