What Is a Biased Estimator In Linear Regression?
In linear regression, estimator biasness refers to the systematic deviation of the estimated coefficients from their true population values. A biased estimator consistently produces estimates that, on average, differ from the true values in a predictable manner.
In the context of linear regression, the biases of an estimator can occur for different reasons:
Omitted Variable Bias: If relevant variables are excluded from the regression model, the estimated coefficients may be biased. Omitted variable bias arises when the omitted variables are correlated with both the independent variables included in the model and the dependent variable. This correlation leads to a systematic bias in the estimated coefficients.
Measurement Error: If there is a measurement error in the dependent variable or the independent variables, it can introduce bias to the coefficient estimates. Measurement error refers to inaccuracies or imprecisions in the measurement or recording of the variables. In the presence of measurement error, the estimated coefficients may deviate from their true values, resulting in bias.
Functional Form Misspecification: If the functional form of the regression model is misspecified, meaning it does not accurately represent the true relationship between the dependent and independent variables, the estimated coefficients may be biased. For example, if a linear relationship is assumed when a nonlinear relationship exists, the estimated coefficients will likely be biased.
Violation Of Assumptions: Linear regression relies on certain assumptions, such as linearity, independence of errors, homoscedasticity, and normality of errors. If these assumptions are violated, the estimated coefficients may be biased. For instance, if the errors exhibit heteroscedasticity (varying levels of error variance), the estimated coefficients may be biased.
It’s important to note that unbiasedness is a desirable property for estimators in linear regression. An unbiased estimator provides estimates that, on average, are equal to the true population values. However, in practice, bias may arise due to various factors, and it is crucial to assess and address the sources of bias to obtain reliable and accurate coefficient estimates.
Super Note:
- For example, Bias in a coin refers to it will always give you a head or tail whenever you flip it.
- An examinor will always give good marks to girls instead of boys.
- Similarly, a biased Linear Regression model will always overestimate or underestimate the predicting variable giving a positive or a negative error always.