Problem set 6: Properties of Estimators

Let \(X_1,\ldots,X_n\) be a sample from \(\operatorname{U}(0,\theta)\). Check whether each of the following estimators of the parameter \(\theta\) is unbiased, consistent, strongly consistent, and asymptotically normal:

  1. \(2\overline{X}\);

  2. \(\overline{X}+\dfrac{X_{(n)}}{2}\);

  3. \((n+1)X_{(1)}\);

  4. \(X_{(1)}+X_{(n)}\);

  5. \(\dfrac{n+1}{n}X_{(n)}\).

Let \(\theta_n^*(X)\) be an asymptotically normal estimator of the parameter \(\theta\). Prove that \(\theta_n^*(X)\) is a consistent estimator of \(\theta\).

Let \(X_1,\ldots,X_n\) be a sample from some distribution with parameter \(\sigma^2\). Suppose, in addition, that \[ \operatorname{Var}(X_1)=\sigma^2. \]

Define the sample variance by \[ s^2=\frac{1}{n}\sum_{i=1}^n (X_i-\overline{X})^2. \]

Prove that:

  1. \[ s^2=\overline{X^2}-\overline{X}^{\,2}; \]

  2. \(s^2\) is a strongly consistent estimator of \(\sigma^2\);

  3. if \(\mathbb{E}X_1^4<\infty\), then \(s^2\) is an asymptotically normal estimator of \(\sigma^2\);

  4. Is \(s^2\) an unbiased estimator of \(\sigma^2\)?

Let \(X_1,\ldots,X_n\) be a sample from an exponential distribution with parameter \(\theta\), i.e., \[ p_\theta(t)=\theta e^{-\theta t}\mathbf{1}(t>0). \]

Show that, for every \(k\in\mathbb{N}\), the statistic \[ \left(\frac{k!}{\overline{X^k}}\right)^{1/k} \] is an asymptotically normal estimator of the parameter \(\theta\).

Find its asymptotic variance.

Let \(X_1,\ldots,X_n\) be a sample from \(\operatorname{U}(0,\theta)\). Find a number \(\delta>0\) and a non-degenerate distribution \(P_\theta\) such that \[ n^\delta\bigl(\theta-X_{(n)}\bigr) \xrightarrow{d_\theta} \xi\sim P_\theta. \]

Let \(X_1,\ldots,X_n\) be a sample from the distribution \(\operatorname{Bern}(\theta)\). Suppose that the function \(\tau\) is such that there exists an unbiased estimator of \(\tau(\theta)\).

Prove that \(\tau\) is a polynomial of degree at most \(n\).

Does every such polynomial admit an unbiased estimator?

Let \(X_1,\ldots,X_n\) be a sample from the uniform distribution on \([0,\theta]\). Compare the following estimators of the parameter \(\theta\) in terms of mean squared error: \[ 2\overline{X}, \qquad (n+1)X_{(1)}, \qquad \frac{n+1}{n}X_{(n)}. \]

Let \(\theta_1^*(x)\) and \(\theta_2^*(x)\) be two optimal estimators of the parameter \(\theta\) in terms of mean squared error, with the same expectations.

Prove that for every \(\theta\) they coincide almost surely, i.e., \[ \theta_1^*(x)=\theta_2^*(x) \qquad P_\theta\text{-a.s.} \]