Applied Econometrics

The building blocks of a bivariate regression model: sample variance and standard deviation, the slope, the intercept, the regression line, and the error term.

Before we can talk about causal impact we need a model that describes the relationship between a treatment and an outcome. The running example asks whether caffeine intake raises heart rate — a question that is easy to model and hard to interpret, which makes it a good place to start.

Confidence intervals are built from standard errors, standard errors are built from residuals, and residuals come out of the regression model. This unit walks that chain backwards until the whole apparatus of hypothesis testing rests on quantities you can calculate by hand.

The distinction that does the most work here is between the distribution of the sample (how spread out your data are) and the sampling distribution (how spread out your estimates would be across repeated samples). Standard deviation describes the first; standard error describes the second.

Moving from bivariate to multiple regression: Y = b0 + b1*X1 + b2*X2 + e, where X1 is always the policy variable and everything after it is a control.

Some controls are correlated with the outcome but not with the treatment — they shrink standard errors and buy precision. Others are correlated with both — they move the slope. Knowing which kind you have added is the difference between a more efficient model and a different answer. Ballentine Venn diagrams are introduced here as the tool for keeping the two straight.

When a variable that belongs in the model is left out, its effect does not disappear. It gets absorbed into the coefficients of the variables that remain, in an amount you can calculate exactly: bias = a1 * B2, where a1 comes from the auxiliary regression of the omitted variable on the policy variable.

The canonical example is the 1989 “crack baby” research, where a strong apparent effect of prenatal cocaine exposure on child development largely dissolved once poverty was added to the models. The doctor-patient racial concordance debate is the same story playing out in a current literature.

Dummy variables are binary 0/1 indicators — one for each level of a categorical variable, minus the reference category:

y = b0 + b1(X) + b2(D) + e

Interactions multiply a covariate by a dummy, which lets the slope itself differ by group:

y = b0 + b1(X) + b2(D) + b3(X)(D) + e

Together they turn a single regression into a set of hypothesis tests about differences between groups — different intercepts, different slopes, or both.

Specification bias is the error you get from fitting the wrong functional form — a straight line through a curved relationship, or a slope dragged around by a highly leveraged outlier. Because OLS minimizes squared error, a residual twice as large has four times the influence, which is why outliers near the extremes of X do so much damage.

The fixes are quadratic terms for non-linearity and log transformations for skew and for modeling growth rates rather than levels. Measurement error is the companion problem: error in the dependent variable inflates standard errors, while error in the independent variable attenuates the slope toward zero.

A consolidated tour of the things that corrupt a regression inference: selection, omitted variable bias, measurement error, specification, group heterogeneity, multicollinearity, and simultaneity. Most of the sequence so far has been a slow walk through the middle of that list; this unit puts the whole set in one place.

Statistical power is the mirror-image question. Before asking whether a program worked, ask whether the study was ever capable of showing that it worked — a null result from an underpowered design carries almost no information.

A causal claim is a counterfactual claim in disguise: to say the program caused the outcome is to say the outcome would not have occurred without it. Since we only get one world, we build comparison groups that stand in for the one we cannot observe.

The running example is a suburban California school district with an unusual number of student suicides. Tested against the state population it looks high; against all high school students it looks average; against suburban high school students it looks low. Same data, same model, three defensible counterfactuals, three opposite conclusions.

Two lectures that belong together. The first asks whether your groups are comparable at all — the selection problem, non-random attrition, and the balance table that every study should report. The second asks what to compute once they are: the reflexive, post-test-only, and difference-in-difference estimators.

The organizing insight is that diff-in-diff is the general case. Reflexive is valid only when there is no secular trend; post-test-only is valid only when the groups were equivalent at baseline. Each of the weaker estimators is a bet that one term drops out.

Every estimator in this course takes the outcome variable on faith. Measurement theory is where that faith gets checked. Two Manhattan neighborhoods can post the same poverty rate while one is trapped in inter-generational poverty and the other is a way station for upwardly mobile immigrant families — same number, different construct.

Some things can be counted directly. Most of what public programs care about — health, happiness, depression, social capital — are latent constructs that can only be measured through an instrument: a survey scale, a coding protocol, a standardized exam. An instrument is valid if it measures the construct it claims to, and reliable if its items measure that construct consistently. Cronbach’s alpha is the workhorse statistic for the second question, and a scale can often be improved by dropping the item that does not belong.

The unit closes with the politics of measurement: once a metric is tied to rewards and penalties, people start managing the metric instead of the outcome.

A framework for evaluating the internal validity of a study as a structured search for competing hypotheses. The program hypothesis says the change came from the program; each Campbell Score item is a rival explanation that has to be ruled out first.

The scoring rule is asymmetric on purpose. Selection and non-random attrition are so common in observational work that they are guilty until proven innocent — a study must show it handled them. The remaining eight items are innocent until proven guilty: you have to argue from evidence in the study that the threat is live.

Chapters 20 and 21 evaluate the same school choice program using the same data and reach opposite conclusions: Chapter 20 finds the program ineffective, Chapter 21 finds it effective. Nothing about the data changed.

Two things differ. Chapter 21 uses a strong counterfactual where Chapter 20 uses a weak comparison group — and, more importantly, each chapter calculates program effects with a different estimator. Work out which of the three each author used and the divergence stops being mysterious.

The time-frame question is subtler than it looks. If a study collects four years of data but analyses only one, its time-frame is one year, not four. Here the time-frame is a function of how effects are calculated, not of how long collection ran.

The synthesis lab. Return to the cognitive-ability study from Unit 08 and calculate program impact three ways — reflexive, post-test only, and difference-in-difference — on the same data.

The regression models are provided. The work is interpretation: read the coefficients correctly, identify which counterfactual each model deploys, run the identification tests, and decide which estimates you actually believe. Some estimators are valid in this context and some are not.

The reflexive counterfactual with enough time points to see a trend. A single unit is observed repeatedly before and after an intervention, and the model separates the pre-existing trajectory from what the intervention changed.

Y = b0 + b1·T + b2·D + b3·P + e, where T counts time, D is a treatment dummy (0 before, 1 after), and P counts time since the intervention. That gives b1 as the pre-intervention trend, b2 as the immediate level change, and b3 as the sustained change in slope. A policy can have an immediate effect, a sustained effect, both, or neither, and the four coefficients tell them apart.

The estimator introduced in Unit 09 becomes a regression here. Two groups, two periods, and an interaction term whose coefficient is the program effect: the treated group’s change minus the comparison group’s change.

Its strength is that the groups need not be equivalent at baseline — level differences difference out. What it does require is that the two groups were moving in parallel before the intervention, which is a testable claim whenever you have more than one pre-period.

With repeated observations on the same units, a fixed effects model gives each unit its own intercept and estimates the slope from variation within units over time. Anything stable about a unit — however unmeasurable — is absorbed and can no longer confound the estimate.

The lecture contrasts the pooled model, where every unit shares one intercept, with the random effects and fixed effects alternatives, and shows the equivalence between the dummy-variable formulation and the demeaned-data formulation.

When the policy variable is correlated with the error term — through omitted variables, measurement error, or simultaneity — OLS is biased and no amount of data fixes it. An instrument is a variable that moves the treatment but affects the outcome only through the treatment.

Two-stage least squares makes the logic explicit: regress the endogenous variable on the instrument, keep the fitted values, and use those in the second stage in place of the original variable.

first.stage  <- lm( x1 ~ z + x2, data=dat )
x1_hat       <- fitted( first.stage )
second.stage <- lm( y ~ x1_hat + x2, data=dat )

When eligibility is determined by whether a running variable crosses a threshold — a test score, an income line, a date of birth — the units just above and just below the cutoff are close to comparable. The jump in outcomes at the threshold is the effect.

The design buys credibility cheaply, and pays for it in generalizability: the estimate is a local effect at the cutoff, and says nothing about units far from it.

When the outcome is yes/no — admitted, employed, enrolled — the linear probability model can predict probabilities outside [0,1]. The logit link keeps predictions in range at the cost of coefficients that are no longer read directly.

The model reports log-odds, which have to be converted to probabilities before they mean anything. And because the curve is not linear, the value of a change in X depends on the baseline: in the law school admissions example, a 10-point LSAT increase is worth very different amounts starting from 120, from 140, and from 170.

The post-test-only estimator needs equivalent groups, and observational data rarely supplies them. Matching builds them after the fact: model each unit’s probability of being treated, then pair treated units with untreated units that had similar probabilities.

The lecture works through distance calculations between treated and untreated students, nearest-neighbour matching with and without a caliper, and matching with and without replacement. What matching cannot do is balance anything you did not measure — which is why it addresses selection on observables only.