test	csize	tqual	ses
504	38	3.793	0.7947
651.3	23	5.277	2.301
623.6	42	6.084	0.4609
539	21	2.654	2.635
673.1	20	5.429	2.703
584.4	46	5.506	0.09412

Solutions

PART I:

Question 1.

Create a scatterplot between Class Size (x-axis) and Test Score (y-axis). This will serve as a visual representation of our baseline model of the relationship between Class Size and Test Score.

plot( dat$csize, dat$test, pch=19, col=gray(0.5,0.4), bty="n", cex=2,
      xlab="Class Size", ylab="Test Scores",
      main="Relationship Between Class Size and Test Scores" )

abline( lm(test~csize,data=dat), col="firebrick", lwd=2 )

# CHANGE THE PLOT STYLE:
#
# pch=19             # change point style
# cex=1.5            # change point size
# col="firebrick"    # change color
# bty="n"            # remove bounding box

Question 2.

Regress Test Score on Teacher Quality while saving the residuals. Now create a scatterplot of Class Size and the residuals of Test Score. What happened to the strength of the relationship? Why?

\(test = b_0 + b_1 \cdot tqual + e_1\)

\(e_1 = b_0 + b_1 \cdot csize + e_{e_1}\)

Unexplained variance removed, primarily impacting the residual.

model.01 <- lm( test ~ tqual, data=dat )
e1.test.score <- model.01$residual

par( mfrow=c(1,2) )

plot( dat$csize, dat$test, pch=19, col=gray(0.5,0.4), bty="n", cex=2,
      xlab="Class Size", ylab="Test Score",
      main="Relationship Between \nClass Size and Test Scores" )
abline( lm(dat$test~dat$csize), col="firebrick", lwd=2 )


plot( dat$csize, e1.test.score, pch=19, col=gray(0.5,0.4), bty="n", cex=2,
      xlab="Class Size", ylab="Test Score (residualized)",
      main="Relationship After Controls",
      ylim=c(-200,200) )
abline( lm(e1.test.score~dat$csize), col="firebrick", lwd=2 )

If a control variable is completely uncorrelated with the policy variable and correlated with the outcome, it will not impact the slope but it will reduce the residual component of the deviations, thus moving data points closer to the regression line and improving model fit.

Aside:

The caveat completely uncorrelated is included here because uncorrelated variables will still mathematically have a non-zero amount of correlation, which will result in small but non-meaningful changes to the slope.

One of the hardest concepts to grasp in regression is the difference between mathematical zero (answer will be the same every time) and statistical zero (zero with some random error). The correlation between two random variables will statistically be zero, meaning you would not be able to reject the null of the correlation being zero. That is why we have stars on correlation tables, to be able to discern whether a correlation of 0.07, for example, is a meaningful relationship (small amount of overlap in a Venn diagram) or random noise that should be interpretted as zero (no overlap).

Question 3.

Regress Test Score on SES and save the residuals. Create a scatterplot of Class Size and the residuals of Test Score. What happened to the strength of the relationship? Why?

\(test = b_0 + b_1 \cdot ses + e_2\)

\(e_2 = b_0 + b_1 \cdot csize + e_{e_2}\)

Variance of X1 targetted, thus impacting the explained component.

model.02 <- lm( test ~ ses, data=dat )
e2.test.score <- model.02$residual

par( mfrow=c(1,2) )

plot( dat$csize, dat$test, pch=19, col=gray(0.5,0.4), bty="n", cex=2,
      xlab="Class Size", ylab="Test Score",
      main="Relationship Between \nClass Size and Test Scores" )
abline( lm(dat$test~dat$csize), col="firebrick", lwd=2 )

plot( dat$csize, e2.test.score, pch=19, col=gray(0.5,0.4), bty="n", cex=2,
      xlab="Class Size", ylab="Test Score (residualized)",
      main="Relationship After Controls" )
abline( lm(e2.test.score~dat$csize), col="firebrick", lwd=2 )

If the control variable is highly-correlated with the policy variable, it will effectively remove the “explained” portion from the simple model (class size and test scores, on the left) but not greatly impact the residual portion, thus reducing the slope.

Note that removing the explained portion of a model is like kicking the kick-stand out from a bike. The regression line falls flat.

Question 4.

These graphs demonstrate the effects of adding the control variables Teacher Quality and SES to the baseline model of the relationship between Class Size and Test Score. Conceptually, the control variable will improve a model by removing variance in the DV to make the estimate either more precise (smaller standard errors) or less bias (adjust the slope of the policy variable).

Q 4-1:

Which control variable do you think is removing UNEXPLAINED portions of the outcome Test Scores? By removing the unexplained portions it will reduce the residuals in the full model.

TEACHER QUALITY (see above)

Q 4-2:

Which control variable is removing the EXPLAINED portion of the variance of test scores (the covariance of class size and test scores)? By removing the explained portion it will weaken the relationship between class size and test scores.

SOCIO-ECONOMIC STATUS (see above)

PART II:

Use the following regression table and graphs to answer the question.

Question 5-1

Based upon the correlation structure reported below, which control variable do you expect would change the slope of caffeine if removed from the model?

Explain your reasoning.

It’s important to focus on the reasoning used to arrive at the results here. Using the examples above we can think about two types of control variables. One serves to improve the statistical power of our model (makes all residuals smaller) without impacting the size of slopes. The other will impact the policy slope.

A variable MUST BE correlated with the policy variable (caffeine in this case) to impact the slope.

Looking at the table below we can see that coffee consumption (our vector for caffeine intake in this non-experiment) will be highly correlated with the stress experienced at work. More stressful jobs require longer hours and more focus, resulting in higher consumption of caffeine.

This is a COMPETING HYPOTHESIS because we see high levels of coffee consumption and high levels of stress together, so it is hard to say which is actually causing increased heart rates. We need to include both in the same model to isolate the independent effects of each. Or similarly, in this case if we remove stress from the model we would expect that the slope associated with caffeine will change.

Question 5-1

Which would result in a larger standard error associated with caffeine if removed from the model?

Explain your reasoning.

Control variables that are correlated with the outcome, but uncorrelated with the policy, decrease the standard errors when added to the model (see Teacher Quality above).

So analogously, we expect the standard errors to increase when we remove (or omit) these controls from the model.

Gym time is uncorrelated with coffee and stress because you can consume lots of coffee and have a stressful job and still make time for the gym (if you have a stressful job you should especially make time for the gym). In reality these might be more correlated (less free time usually means less gym time), but theoretically they don’t have to occur together.


	Dependent variable:

	heart.rate
	(1)	(2)	(3)	(4)

caffeine	0.087^***	0.080^***	0.009	0.037
	(0.021)	(0.008)	(0.121)	(0.047)

gym.time		-1.441^***		-1.440^***
		(0.062)		(0.062)

stress.index			0.414	0.228
			(0.631)	(0.246)

Constant	68.953^***	116.461^***	68.267^***	116.022^***
	(5.454)	(2.942)	(5.568)	(2.982)


Observations	100	100	100	100
R²	0.153	0.872	0.157	0.873

Note:	p<0.1; p<0.05; p<0.01

Lab 03 Control Variables - SOLUTIONS

Load Packages

Load Data

Solutions

PART I:

Question 1.

Question 2.

Question 3.

Question 4.

PART II:

Question 5-1

Question 5-1