Step 3: Substantive Significance/Effect Size

As in a simple linear regression, we can use the numeric values of the coefficients to make imperfect guesses about the magnitudes of their effects. As with interpreting the direction of the effect in a multiple linear regression, we must account for the fact that each coefficient on an $x$ variable is a marginal effect. That is, each coefficient $\beta_i$ represents the change in $y$ when $x_i$ changes and all other $x$ variables are held constant.


	% shift to Trump, 2012-2016

County median income ($1,000s)	-0.158^*	-0.013
	(0.006)	(0.007)

County college experience		-0.344^*
		(0.012)

Constant	8.337^*	20.103^*
	(0.351)	(0.512)


Observations	3,111	3,111
Adjusted R²	0.203	0.371

Note:	* p<0.05

County median income

Our best guess is that a $1,000 increase in county median income is associated with a -0.013% shift to Trump between 2012 and 2016 when education levels are held constant.
We are not confident that this effect is truly negative, because the coefficient is not statistically significant now that county education levels have been controlled for.
Changes in statistical significance and effect size can happen when adding controls to a regression, particularly when there is a strong correlation between the $x$ variables.
Because we are not confident in the direction of the effect of county median income, we do not attempt to interpret the effect size of county income.

County college experience

Our best guess is that a 1% increase in county educational levels is associated with a -0.344% shift to Trump between 2012 and 2016 when income levels are held constant. As noted on the prior page, this is a statistically significant effect, so we are confident it is a negative effect, not null or positive.
The range of the values for county education is from roughly 20% to 90%. That’s 70 units. So a county at the bottom end of the spectrum has shifts to Trump by around $-0.344*70\% = 24$% more than a county at the highest education levels. If the best guess is in the right ballpark, this is a substantively significant change.
You can examine this change in the graphic above. The points described in the calculations have been marked with red diamonds. You can spin the graphic and hover over the red diamonds to see that a change of 70% in college experience is associated with a 24% change in the shift to Trump between 2012 and 2016.
In this case, the estimated effect of education in this model is both statistically and substantively significant.