Introduction to Weighted Least Squares in R
This post will discuss the implementation of Weighted Least Squares (WLS) in R. The OLS method minimizes the sum of squared residuals, while the WLS weights the square residuals. The WLS technique is used when the OLS assumption related to constant variance in the errors is violated.
Table of Contents
The WLS technique is also known as weighted linear regression it is a generalization of ordinary least squares (OLS) and linear regression in which knowledge of the variance of observations is incorporated into the regression.
WLS in R Language
Let us perform the WLS in R.
Here we will use the mtcars
dataset. You need to load this data set first using the attach()
function. For example,
attach(mtcars)
Consider the following example regarding the weighted least squares in which the reciprocal of $wt$ variable is used as weights. Here two different weighted models are performed and then check the fit of the model using anova()
function.
# Weighted Model 1 w_model1 <- lm(mpg ~ wt + hp, data = mtcars) # Weighted Model 2 w_model2 <- lm(mpg ~ wt + hp, data = mtcars, weights = 1/wt)
Check/ Test The Model Fit
To check the model fit, summary statistics of the fitted model, and different diagnostic plots of the fitted model, one can use the built-in functions as,
anova(w_model1, w_model2 ) summary(w_model1) summary(w_model2) plot(w_model1) plot(w_model2)
The output of the First Weighted Model is:
Call: lm(formula = mpg ~ wt + hp, data = mtcars) Residuals: Min 1Q Median 3Q Max -3.941 -1.600 -0.182 1.050 5.854 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 37.22727 1.59879 23.285 < 2e-16 *** wt -3.87783 0.63273 -6.129 1.12e-06 *** hp -0.03177 0.00903 -3.519 0.00145 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 2.593 on 29 degrees of freedom Multiple R-squared: 0.8268, Adjusted R-squared: 0.8148 F-statistic: 69.21 on 2 and 29 DF, p-value: 9.109e-12
The output of the Second Weighted Model is
Call: lm(formula = mpg ~ wt + hp, data = mtcars, weights = 1/wt) Weighted Residuals: Min 1Q Median 3Q Max -2.2271 -1.0538 -0.3894 0.6397 3.7627 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 39.002317 1.541462 25.302 < 2e-16 *** wt -4.443823 0.688300 -6.456 4.59e-07 *** hp -0.031460 0.009776 -3.218 0.00317 ** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 1.554 on 29 degrees of freedom Multiple R-squared: 0.8389, Adjusted R-squared: 0.8278 F-statistic: 75.49 on 2 and 29 DF, p-value: 3.189e-12
Graphical Representation of Models
The graphical representation of both models is:
Learn about Generalized Least Squares