Seminar 4 in Applied Statistics
Consider the standard linear regression model with one regressor:
\[ Y_i = \beta_0 + \beta_1 X_i + \varepsilon_i. \]
a) Verify that the standard scalar OLS estimators of \(\beta_0\) and \(\beta_1\) coincide with
\[ \hat{\beta} = (X^T X)^{-1}X^T Y, \]
where \(X\) is the \(n\times 2\) design matrix whose \(i\)-th row is \((1,X_i)\).
b) Show that
\[ P_X = X(X^T X)^{-1}X^T, \qquad Q_X = I-P_X \]
are projection matrices.
Consider the regression model
\[ Y_i = \beta_0+\beta_1X_i+\beta_2d_i+\beta_3X_i d_i+\varepsilon_i, \]
where \(d_i\) is a dummy variable, i.e. \(d_i\in\{0,1\}\), and \(\varepsilon_i\) is the regression error.
Show that the OLS estimators of \(\beta_0\) and \(\beta_1\) in the full model coincide with the OLS estimators obtained by running the regression
\[ Y_i=\beta_0+\beta_1X_i+\varepsilon_i \]
only on the subsample for which \(d_i=0\).
Suppose that \((Y_i,X_i)\), \(i=1,\ldots,n\), are i.i.d. copies of random variables \((Y,X)\), and consider the linear regression model
\[ Y_i=\alpha+\beta X_i+\varepsilon_i, \]
where \(\varepsilon_i\) is an unobserved regression error.
a) Find the probability limits of the OLS estimators \(\hat{\alpha}\) and \(\hat{\beta}\).
b) Under what conditions is the OLS estimator \(\hat{\beta}\) consistent for \(\beta\)?
Let \(Y,X_1,\ldots,X_p,\varepsilon\) be random variables, and define
\[ X=(X_1,\ldots,X_p)^T. \]
Assume that
\[ \operatorname{Cov}(X,\varepsilon)=0_p. \]
Consider the linear model
\[ Y=X^T\beta+\varepsilon. \]
Suppose that \(X\) can be partitioned into two groups:
\[ X= \begin{pmatrix} D\\ W \end{pmatrix}, \]
where \(D\) is a \(p_1\)-dimensional vector of target regressors and \(W\) is a \(p_2\)-dimensional vector of other regressors (control variables). Thus,
\[ Y=D^T\beta_1+W^T\beta_2+\varepsilon. \]
The coefficient \(\beta_1\) describes how the prediction of \(Y\) changes with \(D\) while holding \(W\) fixed.
Define the population partialling-out operator \(Q_W\) by
\[ Q_W(V)=\widetilde V = V-W^T\gamma_{VW}, \]
where
\[ \gamma_{VW} = \operatorname*{arg\,min}_{b\in\mathbb R^{p_2}} \mathbb E\left(V-W^Tb\right)^2. \]
If \(V\) is a random vector, the operator is applied componentwise. Assume that all relevant random variables have finite second moments.
a) Show that \(Q_W\) is a linear operator.
b) Apply \(Q_W\) to both sides of
\[ Y=D^T\beta_1+W^T\beta_2+\varepsilon. \]
Is the resulting regression error uncorrelated with the regressors? Express the population regression coefficient \(\beta_1\) in terms of
\[ \widetilde Y=Q_W(Y), \qquad \widetilde D=Q_W(D). \]
c) How should the definition of \(Q_W\) be modified if the regression model also contains quadratic terms in \(W\)?