Estimators | Approximation & Estimation | A Level Maths Revision Notes

Contents hide

1 Summary

1.1 Sample Mean

1.2 Sample variance

1.3 Bias and Unbiased of Estimator

1.3.1 Example #1

1.3.1.1 Reference

Summary

Estimator is a statistic intended to approximate a parameter governing the distribution of the data.
Estimator is denoted by a hat i.e $\hat { \theta }$ denotes an estimator of $\theta$ .
Sample Mean : $\overline { X } \quad =\quad \frac { { X }_{ 1 }\quad +\quad X_{ 2 }\quad +\quad .....\quad { X }_{ n } }{ n }$
Sample variance: ${ S }^{ 2 }\quad =\quad \sum { \frac { { (X\quad -\quad \overline { X } ) }^{ 2 } }{ n } }$
An estimator is said to be consistent if the variance of the estimator tends to zero as $n\quad \rightarrow \quad \infty$
An absolute efficiency of an estimator as the ratio between the minimum variance and the actual variance.
$bias(\hat { \theta } )\quad =\quad E\hat { \theta } \quad -\quad \theta \quad$
An estimator $\hat { \theta }$ is unbiased if its $bias(\hat { \theta } )\quad =\quad 0$

In statistics, estimation refers to the process by which one draws certain conclusions about a population, based on information obtained from a sample.

An estimator is any quantity calculated from the sample data which is used to give information about an unknown quantity in the population. Much of the theory of statistics is called Estimators. We can also say estimator is a statistic intended to approximate a parameter governing the distribution of the data.

To understand what statistic is let’s assume we have some data:

$D\quad =\quad ({ X }_{ 1 },\quad { X }_{ 2 },\quad ........\quad { X }_{ n })$ where the X’s are random variables.

Statistic can be explained as a random variable R that is a function of the data D and can be written in the form R = f(D).

Estimator is denoted by a hat i.e $\hat { \theta }$ denotes an estimator of $\theta$ .

Sample Mean

Sample Mean is defined as $\overline { X } \quad =\quad \frac { { X }_{ 1 }\quad +\quad X_{ 2 }\quad +\quad .....\quad { X }_{ n } }{ n }$

Sample variance

And sample variance is defined as ${ S }^{ 2 }\quad =\quad \sum { \frac { { (X\quad -\quad \overline { X } ) }^{ 2 } }{ n } }$

Remember that the best or most efficient estimator of a population parameter is one which give the smallest possible variance. Also an estimator is said to be consistent if the variance of the estimator tends to zero as $n\quad \rightarrow \quad \infty$ . We can thus define an absolute efficiency of an estimator as the ratio between the minimum variance and the actual variance.

Bias and Unbiased of Estimator

Let’s now move on to discuss what bias and unbiased of estimator is.

We can say that a bias of an estimator $\hat { \theta }$ is the expected value of the estimator minus the true value of the estimator. We can write it in the form of a formula:

$bias(\hat { \theta } )\quad =\quad E\hat { \theta } \quad -\quad \theta$

Secondly, an estimator $\hat { \theta }$ is unbiased if its $bias(\hat { \theta } )\quad =\quad 0$ , otherwise it’s considered bias. The sample mean and sample variance that is defined above are said to be unbiased estimators of the mean and variance.

Example #1

Q. A distribution has a known mean $\mu$ and variance ${ \sigma }^{ 2 }$ . If two independent observations ${ X }_{ 1 }$ and ${ X }_{ 2 }$ are made, find the values of a and b such that $a{ X }_{ 1 }\quad +\quad b{ X }_{ 2 }$ is an unbiased and efficient estimator of $\mu$ .