R Squared and Pearson Correlation: How Good Is a Regression Relationship?

A regression line gives us a prediction rule. This lesson asks the next question: how close are the predictions, how much variation does the line explain, and how strong is the linear relationship between X and Y?

On this page

1. Regression accuracy
2. Total, explained, and unexplained variation
3. R squared
4. Pearson correlation
5. Correlation and causation
6. Properties and regression coefficients

What You Should Be Able to Do After This Lesson

• Explain the standard error of estimate.
• Distinguish observed Y from predicted Y hat.
• Explain total, explained, and unexplained variation.
• Calculate and interpret r squared.
• Calculate and interpret Pearson r.
• Explain why correlation does not prove causation.
• Use r = ±√(b_yxb_xy).

Part A: Regression Accuracy

Actual Values, Predicted Values, and Errors

The data give us an observed value of Y. The fitted regression line gives us a predicted value, written as Y hat. Their difference is the prediction error.

e_i = Y_i − Ŷ_i

A positive error means the observed point is above the fitted line. A negative error means it is below the line.

Standard Error of Estimate

The standard error of estimate summarizes the typical vertical spread of observed Y values around the fitted regression line.

s_y.x = √[ Σ(Y_i − Ŷ_i)² / (n − 2) ]

For our example, the sum of squared errors is 4. With five observations:

s_y.x = √(4/3) = 1.155

The standard error is about 1.15 Y units.

Total, Explained, and Unexplained Variation

Worked Variation Calculation

The mean of Y is Ȳ = 14.

Total variation = Σ(Y − Ȳ)² = 44

Unexplained variation = Σ(Y − Ŷ)² = 4

Explained variation = Σ(Ŷ − Ȳ)² = 40

44 = 40 + 4

Coefficient of Determination, R Squared

R squared is the proportion of total variation in Y explained by its linear relationship with X in the fitted simple regression model.

r² = Explained Variation / Total Variation

r² = 1 − Unexplained Variation / Total Variation

r² = 40/44 = 0.9091

About 90.9 percent of the variation in Y is explained by the fitted linear relationship. About 9.1 percent remains unexplained by this model.

Part B: Pearson Correlation

What Correlation Measures

Regression and correlation answer different questions. Regression explains or predicts Y from X. Correlation measures the strength and direction of linear association between two variables.

Correlation Versus Covariance

Covariance shows whether two variables tend to move in the same or opposite direction. Its size depends on the units. Correlation standardizes this movement.

ρ = Cov(X,Y) / √[Var(X) Var(Y)]

Positive, Negative, and Near Zero Linear Correlation

Pearson Product Moment Correlation Coefficient

For the worked data, X̄ = 3 and Ȳ = 14.

The value is positive and close to plus 1, so these data have a strong positive linear association.

Computational Formula

Both formulas give the same answer apart from rounding.

Range and Interpretation

−1 ≤ r ≤ 1

The Link Between r and r Squared

In simple linear regression, the square of Pearson r is the coefficient of determination.

For our example, r = 0.9535 and r² = 0.9091. This matches the explained variation calculation.

Correlation Does Not Imply Causation

A high correlation does not prove that changes in X cause changes in Y. A third variable can affect both. The direction of cause can also be unclear.

Model answer: Correlation means two variables move together, but a hidden variable, reverse direction, or another explanation may produce that movement. Extra evidence is needed before making a causal claim.

Properties of Pearson Correlation

Regression Coefficients and Correlation

r = ±√(b_yxb_xy)

The sign of r follows the common sign of the two regression coefficients.

Regression Versus Correlation

Common Mistakes

• Calling r squared the percentage of Y caused by X.
• Saying r near zero proves that no relationship exists.
• Forgetting that r squared cannot be negative.
• Confusing actual Y with predicted Y hat.
• Using covariance and correlation as though they have the same scale.
• Treating strong correlation as proof of cause and effect.

Formula Revision Table

Frequently Ask Questions

1. What does R squared mean in regression?
   R squared, also called the coefficient of determination, shows the proportion of variation in the dependent variable Y that is explained by its linear relationship with X. For example, if r2=0.80, then 80 percent of the variation in Y is explained by the fitted linear regression relationship, while 20 percent remains unexplained.

2. What is the difference between R squared and Pearson correlation?
   Pearson correlation, r, measures the strength and direction of a linear relationship between two variables. R squared measures the proportion of variation explained by the linear regression relationship. In simple linear regression, the coefficient of determination is r2.

3. What does the Pearson correlation coefficient tell us?
   The Pearson correlation coefficient measures the direction and strength of a linear relationship. Its value ranges from −1 to +1. A positive value shows a positive relationship, a negative value shows a negative relationship, and a value near zero shows little or no linear relationship.

4. Does a high correlation mean that one variable causes the other?
   No. Correlation shows association, not proof of cause and effect. Two variables may move together because another variable affects both, or because the direction of influence is unclear. The chapter specifically warns students not to interpret correlation as causation.

5. What is the relationship between correlation and regression coefficients?
   The correlation coefficient can be found from the two regression coefficients using:

r=±√b_yx​b_xy​​

The sign of r follows the common sign of the regression coefficients. If both regression coefficients are positive, r is positive. If both are negative, r is negative.