So we have correlation, which is fine, but there are two big drawbacks to correlation.
First, the thing that is most helpful about it – that it’s unitless – is also a bit of a drawback. While “correlation” is definitely a word regular people understand, it’s pretty impossible to form a real human sentence about a correlation. Considering what we found above: that the correlation between ideology and voting for a centrist candidate is higher among White then Black Americans. That sentence is fine to say but really lacks nuance. How much does ideology matter? What can I practically say about moving from “very liberal” to “moderate” in terms of the probability of voting for a candidate?
The second thing is that our ability to consider a third variable is very blunt. The only way that we could look at the relationship between ideology and voting for a centrist candidate while taking into account race was very bluntly splitting the sample into two groups. What if our third variable had many levels, or was continuous? Or what if we wanted to also consider the effect of education? Correlation doesn’t really allow us to do any of those things.
What is going to solve both of those problems is regression.
To investigate regression let’s look at the relationship between the age and the feeling thermometer for Black Lives Matter:
summary(anes$V201507x)
Min. 1st Qu. Median Mean 3rd Qu. Max.
-9.00 35.00 51.00 49.04 65.00 80.00
Min. 1st Qu. Median Mean 3rd Qu. Max. NA's
0.0 15.0 60.0 53.3 85.0 100.0 936
#Reduce to only where we have non NAs for those two variables#this is not strictly necessary but helps with our "by hand" calculations#down belowanes <- anes[!is.na(anes$age),]anes <- anes[!is.na(anes$blm.therm),]
Before I graph any scatterplot I take a minute to think: what do I expect this graph to look like. What do you expect this graph to look like?
Ah! This looks terrible! It happens, the general problem here is that a lot of people give similar answers on the FT so we have points stacked on top of each other. But still we can also conclude this is not a deterministic relationship: there are lots of 20 year olds that have negative views of BLM and lots of 80 year olds with positive views of BLM.
Now we can use correlation to summarize what’s happening here:
cor(anes$age, anes$blm.therm)
[1] -0.1307171
There is a very weak negative relationship between these two things.
The goal of regression is to draw a line through these data that summarizes what is going on. How might we do that?
Well one way to do this (this is not the right way) would be to look at each value of age and find the average value of the blm feeling thermometer:
So this progression of averages generally slopes downwards, as we would anticipate given the correlation. But, this doesn’t really allow us to give a answer about the relationship between age and feelings towards BLM.
More problematically, let’s not forget the overall goal of this class, which is to form inferences about the population not just to describe the data that we have. Here we are very religiously following the data from age to age. We would absolutely not claim that each and every up and down of this line is something that is present in the population. For example we are not going to conclude that feelings about BLM generally decline to age 40, but then from age 40 to 41 they go up a bunch, and then they go back down. In other words we are “over-fitting” the data that we have here, taking seriously every quirk in the random sample to be true.
Instead, we want to work to summarize this data in a more averaged way, we want to smooth out the random variations caused by sampling to produce a single line which gives us the overall picture of what is happening between these two variables.
Specifically, we are going to estimate the following equation:
\[
\hat{y} = \hat{\alpha} + \hat{\beta}x_i
\]
Where \(\hat{\alpha}\) is the y-intercept, \(\hat{\beta}\) is the slope, and \(x_i\) is each data-points x value.
We are going to get to where these numbers come from shortly, but here are the intercept and slope that the method we are about to cover – Ordinary Least Squares (OLS) regression – will choose:
Here \(\hat{\alpha} = 67.55\) and \(\hat{\beta}=-.27\). What those two pieces of information allow us to do is to draw a line through these data. Specifically, you can evaluate this equation for all values of \(x\), and for each value get a value for \(\hat{y}\), which is the line at that point.
Today we are going to determine (1) how did R choose that line? And on wednesday we will cover (2) How do we interpret that line.
For each data point that we have we can define the residual. The residual is the vertical distance between each data point and the value for the line at that point. Mathematically, the residual is
\[
\hat{u_i} = y_i - \hat{y}
\] We would never figure this out one at a time, but for example our first observation is a 46 year old who gave a value of 0 for the blm therm:
anes[1,c("age","blm.therm")]
age blm.therm
1 46 0
The value of the line at that point would be \(67.55 - .27*46 = 55.13\). As such, the residual for this first individual would be \(0-55.13 = -55.13\).
I mentioned above that the method we use to choose an \(\alpha\) and \(\beta\) is called “Ordinary Least Squares”. It is called this because the method we use to choose these values, is the method which minimizes the sum of the squared residuals.
So specifically, we choose \(\alpha\) and \(\beta\) such that we minimize:
We want to specifically choose the \(\alpha\) and \(\beta\) values so that this equation generates a small a number of possible.
Now I’ve claimed that the alpha and beta chosen by OLS are the alpha and beta that does that. I will prove that to you in two ways. First through simulation and second through calculus.
Let’s first calculate what the sum of squared residuals is for the chosen alpha and beta
Let’s use that sum of the squared residual equation above and first hold constant beta and run through many possibilities for alpha. We should see that the result is the smallest when alpha is equal to 67.55:
In both cases the sum of the squared residuals is at it’s minimum point at the values for alpha and beta that were chosen by OLS.
We can also show the same thing via calculus. If calculus is not your thing, don’t worry too much about this. But this helps me to mechanically understand what’s happening in OLS regression:
We want to take the partial derivative of the sum of squares equation with respect to both alpha and beta. This will tell us the equations to determine the slope at any point on the above graphs.
For alpha, we can take the first derivative via the power and chain rule:
If you are rusty on your calculus, the steps we took generate the equation which, for the graphs above, give the slope of the line at any point.
What we are interested partiuclarly interested in is when the slope is equal to zero. Why zero? Because we want the combination of alpha and beta that leads us to the smallest sum of squared residuals, and that happens preceisely when the slope of that curve is zero (a flat line).
So to find this minimum we will set this equation equal to 0 and isolate both \(\hat{\alpha}\) and \(\hat{\beta}\), the two things we need to estimate:
Now we can use that information to isolate beta and minimize.
First do some re-arranging of the sum of squared residuals equation: \[
\begin{aligned}
\hat{u_i}^2 = \sum_{i=1}^n (y_i - \hat{\alpha} - \hat{\beta}x_i)^2
\end{aligned}
\]
Sub in the (now) known definition of \(\hat{\alpha}\)
Set equal to 0 and divide by -2: \[
\begin{aligned}
0 = \sum_{i=1}^n (y_i - \bar{y})(x_i - \bar{x}) -\hat{\beta}(x_i -\bar{x})^2
\end{aligned}
\] Move the last term to the other side, and remove the constant \(\beta\) from the summation
\[
\begin{aligned}
\hat{\beta} \sum_{i==1}^n(x_i -\bar{x})^2 = \sum_{i=1}^n (y_i - \bar{y})(x_i - \bar{x})
\end{aligned}
\] Divid both sides by \(\sum_{i==1}^n(x_i -\bar{x})^2\):
So a little bit of calculus produces the equations for the two paramaters of the line that produce the line which minimizes the sum of the squared residuals.
Looking at the equation for \(\hat{\beta}\) should look familiar.
If we think about the equation for covariance and for variance they are:
We have uncovered the equations used to generate an alpha and beta to draw the line we see above. Now that we have that line, how do we interpret the main regressionwe have been discussing?
Min. 1st Qu. Median Mean 3rd Qu. Max.
-62.690 -33.381 4.049 0.000 32.202 54.049
The two numbers for alpha and beta are present in the “Estimate” column. These are generated using the equations we’ve derived.
The remaining columns give a hypothesis test. How could we determine which hypothesis is being tested? Well a t value is “how many standard errors the estimate is from the null. So:
So we are getting hypothesis tests that these coefficients are equal to 0, or not.
This makes a lot of sense for the regression coefficient on age. We care a lot about whether a coefficient is 0, or not, because 0 is a flat line! This has a very high t-value and a very low p-score, so we would reject the null hypothesis that the slope in the population is 0. More next week on this hypothesis test!
Do we care if alpha is 0 or not? Well… in real terms what is alpha?
It is the value of \(\hat{y}\) where the line crosses the y axis. We can see that mathematically:
So it is the predicted value of y when x is 0. What does an x of 0 mean here? A newborn? This is how newborns feel about BLM? Do we care about that at all?
No! Of course not. Having an \(\alpha\) is a pre-requisite to drawing a line. Like mechanically, the two ingredients to a line are its intercept and its slope. So we need to define this value, but that does not mean that it is a meaningful number.
What’s more, the hypothesis test is definitely not a meaningful hypothesis test. Do we care if the predicted level of support of BLM of newborns is 0, or not? No! Definitely not. I actually don’t even like that they give you a statistical test for this value.
Let’s move on to beta, which is the estimate for age. We have seen that it is -.27, what does that mean?
You have seen a slope before being defined as rise over run, and this is just what this number is. For every 1 unit increase in X, Y goes down by .27.
When we say 1 unit in a regression, we mean 1 unit in our number system. As in the difference between 1 and 2, and 100 and 101, and 567 and 568. This will be true for absolutely every OLS regression we ever run and you should really internalize it right now. The beta coefficient is how a 1 unit change in x relates to a \(\beta\) change in y.
The best thing about this is how it allows us to form normal human sentences about this relationship. As a person gets 1 year older their feelings about BLM drop .27 points on average.
Note also that we can scale this up or down: If a one unit change in x is -.27, what is a 10 unit change in x? 2.7! So we might also say that as a person gets 10 years older, their feelings about BLM drop 2.7 points on average. Also good!
What this means, however, is that if we re-scale our x variable then \(\beta\) will change, even if the underlying relationship does not.
Call:
lm(formula = blm.therm ~ age.months, data = anes)
Residuals:
Min 1Q Median 3Q Max
-62.690 -33.381 4.049 32.202 54.049
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 67.549664 1.328699 50.84 <2e-16 ***
age.months -0.022498 0.002031 -11.08 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 35.12 on 7062 degrees of freedom
Multiple R-squared: 0.01709, Adjusted R-squared: 0.01695
F-statistic: 122.8 on 1 and 7062 DF, p-value: < 2.2e-16
Visually we can see that the relationship looks the same, we have just re-scaled the x-axis of the graph. Age explains no more or less than it did when it was measured in years.
Looking at the coefficient on age.months we see that it is smaller than it was before. Because we know that this is how a 1-unit change in x relates to a change in y, we know that this does not mean that age is “less important” in this second regression. Instead, we are now measuring the impact of a change in 1 month on BLM feelings instead of 1 year.
Indeed if we multiply that coefficient by 12:
m2$coefficients["age.months"]*12
age.months
-0.2699798
We return the original coefficient.
Think this through in relation to what we discovered last time that the regression coefficient is simply the covariance of x and y divided by the variance of x. When we were discussing covariance we found that it is completely sensitive to the scale of the variables. Correlation divided by the product of the variance of x and y which standardized the variable to 0,1. Regression, on the other hand, only divides by the variance of the explanatory variable. This means that regression coefficients are standardized based on whatever x is scaled to be at the current moment.
How does this interpretation of the beta coefficient relate to other types of variables?
Let’s look at how being a liberal vs being a conservative or moderate influences your opinions of BLM:
Call:
lm(formula = blm.therm ~ liberal, data = anes)
Residuals:
Min 1Q Median 3Q Max
-78.439 -23.054 1.946 21.561 61.946
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 38.0544 0.4774 79.71 <2e-16 ***
liberal 40.3844 0.7898 51.13 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 29.66 on 6080 degrees of freedom
(982 observations deleted due to missingness)
Multiple R-squared: 0.3007, Adjusted R-squared: 0.3006
F-statistic: 2614 on 1 and 6080 DF, p-value: < 2.2e-16
Here we have an intercept of 38.05, and a beta of 40.38.
Let’s start with the intercept, what does this intercept represent? Remember, the intercept is the average value of y when x=0, as shown mechanically by the regression equation.
This is the average value of the BLM thermometer among non-liberals. That is what it means to be 0 on this variable.
What does the slope on liberal mean? This is the effect of a 1 unit change in x on y, so moving one unit on y increases the average person’s blm thermometer by 40.38 points. Ok but what does this mean in real human terms? A one unit change in this variable indicates us moving from one group to another. We have specifically set up this variable in a way that works extremely well with regression, because changing “1-unit” brings us from one group to another. So the difference in blm feelings between liberals and non-liberals is 40.38 points.
What is the average level of feeling towards BLM among liberals?
We can answer this via the regression equation. Our predicted y is equal to
\[
\hat{y} = 38.05 + 40.38*Liberal_i
\]
So we can turn liberal on and off to get that answer:
Let’s compare that to the t.test of the difference in means between these two groups on this variable, which we have already learned about:
t.test(anes$blm.therm ~ anes$liberal)
Welch Two Sample t-test
data: anes$blm.therm by anes$liberal
t = -57.181, df = 6005.6, p-value < 2.2e-16
alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
95 percent confidence interval:
-41.76891 -38.99987
sample estimates:
mean in group 0 mean in group 1
38.05440 78.43879
The two means are 38.05 and 78.43. So OLS regression with a binary x variable exactly uncovers the means of the two groups, with beta representing the difference between those two means.
This is super helpful, and one of the reasons why I’ve made such a big deal about indicator variables throughout the semester. Indicator variables are great because they work extremely well with the math of OLS. OLS uncovers the effect of 1 unit shifts, and a 1 unit shift in an indicator indicates group membership.
What about if we use the un-altered ideology variable as our dependent variable? Remember that ideology is a 7-point scale where 1 is “very liberal” and 7 is “very conservative”.
table(anes$ideology)
1 2 3 4 5 6 7
327 1091 804 1537 696 1267 360
plot(anes$ideology, anes$blm.therm)
m4 <-lm(blm.therm ~ ideology, data=anes)
What does the intercept mean in this case? This is the average value of y when x is 0. Can x be zero? No! Definitely not, so this is a completely meaningless number.
What does beta mean in this case? For every 1 unit change in x your feelings about BLM drop 14.5 points. What does that mean in terms of this variable? Every 1 step more conservative you get you drop 14.5 points in terms of your feelings about BLM.
Now here’s a question, if we use our regression equation to determine the predicted y at 1, 2, 3…7, will that also recover the mean blm thermometer at each of those points?
They are definitely not the same! The red line, the regression line, is constrained to being a straight line. the blue line, the connected means, is not. Which of these, in this case, better represents this data?
Regression assumes that everything you put into it is a continuous variable. That means it thinks that the variable ranges from negative to positive infinity, and that each number is evenly spaced. We are necessarily treating, with the red line, the jump from very liberal to liberal the same as the jump from somewhat liberal to moderate. Looking at the blue line, it’s somewhat clear that each of these jumps is not uniformly important. The jump from 1 to 2 and from 6 to 7 is smaller than the jumps in between. Regression returns none of that nuance, it just gives us a straight line that averages across these values.
Neither of these are right or wrong, they just present two different perspectives. But it’s important to know what’s going on under the hood in regression so you understand the assumptions you are making about your variables.
13.3 An example using what we’ve learned so far
Let’s use what we’ve learned so far to investigate the Arizona data from the 2022 election we’ve worked with before:
I want to investigate, relative to the population we are studying (likely Arizona voters) who is in this sample. We are not going to get in to how weighting works in this class, but the basic intuition is that we define groups, and then give individual higher weights if not enough people from their groups are in the sample, relative to the population. So if we run a sample and we don’t have enough women, all women will get a higher weight to make up for it. A weighting algorithm does this simultaneously for many groups (here it is age, gender, education, race, and 2020 presidential vote). But if we see a relationship between a particular variable and the weight variable, it means that we are using weights to make up for the fact that not enough people from that group are in our sample.
Here is the relationship between age and the weight variable
Bit of a messy relationship that is hard to summarize without regression. Here is the regression output for this relationship:
m <-lm(weight ~ age, data=sm.az)summary(m)
Call:
lm(formula = weight ~ age, data = sm.az)
Residuals:
Min 1Q Median 3Q Max
-0.6194 -0.2602 -0.1573 0.2313 3.8457
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.676267 0.031362 21.563 < 2e-16 ***
age -0.001591 0.000545 -2.918 0.00354 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.5105 on 3045 degrees of freedom
Multiple R-squared: 0.002789, Adjusted R-squared: 0.002462
F-statistic: 8.517 on 1 and 3045 DF, p-value: 0.003544
What is the interpretation for the coefficient on the Intercept here. The average weight when age is 0 is .67. Again, this is a fairly useless bit of information. We don’t really care what happens when age is 0. This is just a mechanically necessary number to have in order to draw a regression line.
What is the interpretation for the coefficient on age? For every additional year of age, the weight goes down by around .001. If higher weights mean that individuals from that group are less likely to be in our sample, what does this result mean? It means that younger people were harder to get into our sample (despite being online!).
What about the relationship between race and the weight variable?
Can we put this variable into the regression as-is? (We actually can, which we will see later on, but for right now….) No. Regression takes numeric variables and this is a bunch of words. We can’t run a regression on this. We have to do some re-coding.
Let’s create a variable that splits the sample into white and non-white:
Again, these are just words, but in this case the words have an order. Let’s convert this into a numbered ordinal variable and then put that into the regression:
Call:
lm(formula = weight ~ biden.approval.num, data = sm.az)
Residuals:
Min 1Q Median 3Q Max
-0.5869 -0.2573 -0.1651 0.2282 3.9108
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.580937 0.012720 45.671 <2e-16 ***
biden.approval.num 0.006676 0.007563 0.883 0.377
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.5093 on 3022 degrees of freedom
(23 observations deleted due to missingness)
Multiple R-squared: 0.0002578, Adjusted R-squared: -7.302e-05
F-statistic: 0.7793 on 1 and 3022 DF, p-value: 0.3774
What is the interpretation of the Intercept? Is it meaningful? What is the interpretation of the coefficient? Looking ahead, what does it mean that our p-value is .377?
13.4 Regression with a binary dependent variable
So far all of the regressions that we have run have had a continuous dependent variable. Is that our only option? What happens if we want to use regression on a binary dependent variable?
Let’s make a 0,1 variable of whether someone plans to vote for Mark Kelly (the Democrat) or Blake Masters (the Republican).
That doesn’t even touch any of the data-points! Is that helpful at all?
Let’s think about what the regression output says:
m4 <-lm(vote.kelly~age, data=sm.az)summary(m4)
Call:
lm(formula = vote.kelly ~ age, data = sm.az)
Residuals:
Min 1Q Median 3Q Max
-0.5482 -0.5241 0.4627 0.4753 0.4989
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.4907882 0.0357856 13.715 <2e-16 ***
age 0.0005743 0.0006037 0.951 0.342
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.4996 on 2611 degrees of freedom
(434 observations deleted due to missingness)
Multiple R-squared: 0.0003465, Adjusted R-squared: -3.64e-05
F-statistic: 0.9049 on 1 and 2611 DF, p-value: 0.3416
What is the mathematical interpretation of the intercept?
Remember, that alpha is the average value of y when x is equal to 0. What does it mean when we take the average of a binary/indicator variable? That gives us the probability of that variable being equal to 1. It’s something we have seen repeatedly in this course. So the practical interpretation of this intercept is that the probability of voting for Kelly among a newborn (I know) is approximately 49%.
If that’s what the intercept is giving us, what does the age coefficient mean? For each additional 1 unit change in x, y goes up .0005, on average. This means that every additional year someone ages their probability of voting for Kelly increases by 1 half of a percentage point.
In other words, when we have a binary dependent variable, it converts all of our explanations into probability.
Let’s do another example (with a variable that actually affects Kelly vote prob…)
Let’s look at how the Biden approval variable above relates to voting for Kelly:
Call:
lm(formula = vote.kelly ~ biden.approval.num, data = sm.az)
Residuals:
Min 1Q Median 3Q Max
-1.1296 -0.1267 -0.1267 0.2047 0.8733
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.126699 0.007313 17.33 <2e-16 ***
biden.approval.num 0.334291 0.004225 79.13 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.2704 on 2595 degrees of freedom
(450 observations deleted due to missingness)
Multiple R-squared: 0.707, Adjusted R-squared: 0.7069
F-statistic: 6261 on 1 and 2595 DF, p-value: < 2.2e-16
What is the interpretation of the intercept now? Well we set this variable to be equal to strongly disapprove of Biden at 0, so this is the probability of voting for Kelly when you strongly disapprove of Biden. What is the interpretation of the coefficient on biden.approval.num now? This is now the effect of going from strongly to somewhat disapprove, from somewhat disapprove to somewhat approve, etc. That effect .33. For every step up the approval chain the probability of voting for Kelly increases by 33%. That makes sense!
Now if we try to visualize this it’s going to look terrible:
We can somewhat improve this graph by using the jitter feature, which adds a bit of noise to our data so that we can see that each of these dots is actually a fair number of people:
But this graph also reveals another problem here. What is the predicted probability of voting for Kelly at all the levels of Biden approval? How can we mathematically determine that?
How would we interpret this last value? If you strongly approve of Biden you have a 112% chance of voting for him! Uh oh! Broken laws of probability!
When we run an OLS regression with a binary dependent variable, it transforms what we are doing into a “Linear Probability Model” or LPM. As we’ve seen, it allows us to interpret our coefficients in terms of the probability of the dependent variable being 1. LPMs are great, and most of the time they work fantastically. I’ve use them throughout my career with no problems.
The major downside to using an LPM is that OLS regression doesn’t know or care what scale your variable are on. It treats every variable like it is a continuous variable that ranges from negative infinity to infinity. There is nothing constraining OLS, in other words, to draw a line that leads to us making a prediction that is outside the interval of \([0,1]\). This is legitimately a big problem if you are specifically using regression to make a prediction, but less of a problem if you are using regression to search for an explanation. Here I’m not super bothered that this prediction falls outside of 0,1 because I am mostly interested in saying that Biden approval has a strong, positive impact on the probability that you are going to vote for Kelly.
If you are interested in using regression for prediction and your outcome variable is binary it is often the case that using an LPM is not appropriate. An example of this would be generating a likely voter model, which definitely needs to range between 0 and 1. In those cases we use a logit or probit model, which are beyond the scope of this course, but are specifically designed to work with binary outcome variables and will not generate predictions outside of the 0,1 interval.
Ultimately any numerical variable can be put into either side of a regression. That being said, you really really have to think about what is happening each and every time. You really have to understand the scale of the variables that you are using in order to correctly interpret what is going on. Every single time.
13.5 Goodness of fit
The final bit of information we get in the regression output is at the bottom, and presents a few different measures of how good your regression fit is, overall.
The value I want to focus on is \(R^2\). Now I may mentioned I have beef with AP stats before. I’ve never actually read an AP stats textbook but for some reason a lot of people come to college with the belief that \(R^2\) is the be all end all of regression. It’s important! But much less so then you might think.
Ok but what is it? \(R^2\) is a measure of how well the \(x\) variable explains the \(y\) variable. It ranges from 0-1 and tells you what percent of the variation in \(y\) is explained by \(x\).
So if we visually think about a single point, the SST is how much each point varies from it’s mean. Remember, this is literally the formula for variance, just without the 1/n. So this is giving us the total variation in the y variable.
SSR instead looks at how the predicted variables vary from y. So we are seeing the improvement in fit that is down to having the regression line as opposed to just having the data points.
We can calculate \(R^2\) for the main regression of age on blm.them:
Call:
lm(formula = weight ~ age, data = sm.az)
Residuals:
Min 1Q Median 3Q Max
-0.6194 -0.2602 -0.1573 0.2313 3.8457
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.676267 0.031362 21.563 < 2e-16 ***
age -0.001591 0.000545 -2.918 0.00354 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.5105 on 3045 degrees of freedom
Multiple R-squared: 0.002789, Adjusted R-squared: 0.002462
F-statistic: 8.517 on 1 and 3045 DF, p-value: 0.003544
Yes they are the same!
Why is this sometimes helpful to know, but sometimes not?
As above, it really comes down to whether we want to use regression for explanation or prediction.
We will cover prediction in a later week, but right now it should be clear mathematically who we might be able to say: we have a regression for how age predicts BLM thermometer scores. What would our prediction be for how a 120-year-old person would feel about BLM? We can very easily just plug that value in for age and OLS will form a prediction. In these cases, how well your explanatory variable (or variables) is explaining the outcome will be critical in determining whether this is a good or bad prediction.
For explanation we are way less concerned with goodness of fit. Here we might be interested in the question of whether age is an explanation of how people feel about BLM. We find above that it is statistically significant, but the \(R^2\) is very low. Someone might come to me and say: your \(R^2\) is low so your regression is junk! But I don’t really care about that. Of course lots of things influence how someone feels about BLM, so my regression is not going to fully explain each individuals feelings simply based on their age. That’s not what i’m trying to do. I’m simply trying to show that this variable has a non-zero impact.
So \(R^2\) is a helpful number, but a low \(R^2\) does not mean your regression is junk. You have to know what you are using it for.
