Estimators: Linear and Unbiased

Best Linear Unbiased Estimators

Definition

Set of data x[n]={x[0],x[1],….,x[N-1]} with a scaled PDF p(x:) that is dependent on the unknown parameter . Because the BLUE stops the estimator to be linear in data, the parameter estimate can be represented as a linear group of data examples with some values an.

=n=0Nanxn=aTx……..(1)

a is a vector of constants whose esteem we look for to meet the plan boundaries. Thus, the whole gauge issue decreases down to deciding the vector of constants- a . The previous condition could bring about various answers for the vector a. Notwithstanding, we should choose the arrangement of a values that produce unprejudiced appraisals with the least change.

Why do we need blue?

When attempting to determine a variable’s Minimum Variance Unbiased (MVU), there are various issues to consider.

• The PDF (Probability Density Function) is unknown.
• The PDF is tough to model.
• Even in circumstances where the PDF is available, estimating the lowest variance is problematic.

The approach to use BLUE

In such cases, the recommended method is to apply a suboptimal estimator and constrain it to linearity.

• This estimator is not influenced in any way.
• Considering both the first and second moments of the probability density function, the minimum variance can be calculated (PDF).
• It is more effective because the entire PDF is never needed.
• This estimate has the smallest variance of any unbiased linear estimator.

Thus, to construct a BLUE estimator with the lowest variance, the set of values for a must satisfy the conditions listed below.

• Characterize a direct assessor.
• It should have the property of being unbiased.
• Minimum variance is then calculated.
• The circumstances under which the base change is figured are not entirely settled.
• This then, at that point, should be placed as a vector.

To find BLUE, we need to first understand the important constraints which help us find BLUE. They are

• Linearity Constraint 
• Unbiased Estimate Constraint
• Linearity Constraint

Assuming every one of the details of a limitation is of the primary request, the imperative is supposed to be straight. This implies the requirement doesn’t contain a variable squared, cubed, or raised to any power other than one, a term isolated by a variable, or factors duplicated by one another. As referenced above, on the grounds that the BLUE compels the assessor to be straight in information, the parameter estimator can be represented as a linear group of data examples with some values an.

=n=0Nanxn=aTx……..(1)

• Unbiased Estimate Constraint

The mean of the estimate must be identical to the true value of the estimation for the estimate to be termed unbiased.

E[]=θ…….(2)

Hence,

n=0NanExn=θ………(3)

Equating both Equation (1) and (2), we get

E[θ]=n=0NanExn=aTx=θ……..(4)

We can meet both the limitations or constraints just when the assumption is direct. That is x[n] is of the structure x[n]=s[n], where is the a not known boundary that we wish to assess.

The linear form to be estimated as a sample is given below,

xn=snθ+wn………….(5)

wn = Zero mean 

The estimation of the equation given above is,

Exn=Esn=snθ……(6)

Substituting both equations (6) and (4), we get

E[θ]=n=0NanExn=θE[θ]=n=0Nansn=θaTs=θ…….(7)

Equality.

aTs=θ…….(8)

Ad can be satisfied if and only if

aTs=1……(9)

If this criterion is met, the next step is to minimise the estimate’s variance. Keeping the estimate’s variance to a minimum,

var=E[(n=0Nanxn-En=0Nanxn)2] 

=E[(aTx-aTEx)2] 

=E[(aTx-Ex)2] 

=E[aTx-Ex[x-Ex]Ta] 

=E[aTCa] 

=aTCa …………………..(10)

How to find BLUE?

As mentioned above, to find a BLUE estimator for a given data set, two requirements – linearity ,and unbiased estimators – should be fulfilled and the change of the variance should be least. Considering constraint as the subject, we need to minimize the variance. As this is a Langrangian Multiplier problem, we get

J=aTCa+aTs-1…….(11)

w.r.t to zero,

∂J∂a=2Ca+s=0 ⇒a=-2C-1s………(12)

Substituting equation (12) and (9)

aTs=-2C-1s=1 ⇒-2=1sTC-1s………..(13)

Now, we finally get the coefficients of the BLUE which is given below

a=C-1ssTC-1s…………(14)

Finally, we get the BLUE and the Variance of the estimates,

BLUE=aTx=C-1ssTC-1s

var()=1sTC-1s

Advantages of BLUE

• The Gauss-Markov theorem can be used to find the BLUE if data can be modeled to have linear observations in noise. The BLUE conclusion is generalized by the Markov theorem to the case where the ranks are less than full.
• BLUE is useful for estimating the magnitude of known signals in noise. It should be noted, however, that noise does not have to be Gaussian in nature.
• The main downside of BLUE is that it is already sub-optimal in nature, and thus is not always the best fit for the task at hand.

Example:

Estimate DC level in colored noise: xn=A+w[n]

n=0,1,……,N-1 

w=[w0,w1,…..,w[N-1]]T (Coloured noise with zero mean)

EwwT=C (Covariance matrix)

BLUE is

A=(hTC-1h)-1hTC-1x=1TC-1x1TC-11

And its variance,

var(A)=11TC-11

Assume the Cholesky Factorization (C)-1= DTD, THEN THE BLUE od A is,

A=1TDTDx1TDTD1=(D1)TDx1TDTD1=n=0N-1dxxtransf[n]

where,

dn=[D1]n/1TDTD1

Conclusion

The best considered, linear unbiased estimator (BLUE), which likewise utilizes the fluctuation of the assessors. BLUE a vector of assessors is BLUE assuming that it is the base fluctuation, direct fair-minded assessor. To show this property, we utilize the Gauss-Markov Theorem.