Understanding the Tests for Linear Hypotheses in Detail

Tests for linear hypotheses

In this article, we will discuss hypothesis testing in the normal linear regression model, where the OLS estimators of the coefficients have a normal distribution conditional on the matrix of regressors. We will also show how to carry out hypothesis tests in linear regression models where the OLS estimator can be proved to be asymptotically normal.

In both parts, the regression model is

yi= xi + i

where yi is an output variable, xi is a 1+k vector of inputs, is a K*1 vector of coefficients and i is an error term. There are N observations in the sample, so that i=1,2,3,….N.

We also denote:

by y the N*1 vector of output

by X the N*k matrix of inputs

by the N*1vector of errors

Using this notation, we can write

y=X+

Moreover, the OLS estimator of is

へ= (XTX)-1XTy

We assume that the design matrix X has full-rank, so that the matrix XTX is invertible.

The normal linear regression model’s hypothesis test

In this section we derive tests about the coefficients of the normal linear regression model. In this model the vector of errors is assumed to have a multivariate normal distribution conditional on X, with mean equal to 0 and covariance matrix equal to 2I; where, I  is the N*Nidentity matrix and 2 is a positive constant.

It can be clearly seen that the assumption of conditional normality justifies that:

• the OLS estimator へ is conditionally multivariate normal with mean $eta $ and covariance matrix 2(XTX)-1 ;
• the adjusted sample variance of the residuals

 2へ=1N-Ki=1N(yi-Xiへ)2

is an unbiased estimator of 2; furthermore, it has a Gamma distribution with parameters N-K and 2;

• へis conditionally independent of 2へ.

The single coefficient’s Restriction Test (t-test)

In The single coefficient’s Restriction Test, we test the null hypothesis

H0: k=q; where k is the kth entry of the vector of coefficients and qℝ .

In other words, our null hypothesis is that the kth coefficient is equal to a specific value.

This hypothesis is usually tested with the test statistic

t=へ-q2へSkk; where, Skk is the kth diagonal entry of the matrix (XTX)-1.

The test statistic t has a standard Student’s t distribution with N-K degrees of freedom. For this particular reason, it is termed as t statistic and the test is termed as the t test.

Explanation

The null hypothesis is not considered if t falls outside the acceptance range.

The acceptance region is determined depending not only on the desired size of the test but also on the two-tailed test (if we consider that kis both smaller or larger than q) or one-tailed test (if we consider that only one of the two things, that means smaller or larger, is possible).

Test of a set of linear restrictions (F test)

When testing a set of linear restrictions, we test the null hypothesis

R=q; where, R is an L*K matrix and q is an L*1 vector. L is the number of restrictions.

Example Suppose that is 2*1 and that we want to test the hypothesis1+2=1. We can write it in the form R=q by setting

 R= [1  1]

q=1

Example 

Suppose that is 3*1 and that we want to test whether the two restrictions 1=2 and 3=0 hold simultaneously. The first restriction can be represented as

1-2=0

So we have

This hypothesis is generally tested with the test statistic

F=1L(Rへ-q)T[2へR((XTX)-1RT]-1((Rへ-q)

which has an F distribution with L and N-K degrees of freedom. That’s the reason, it is called the F statistic and the test is called the F test.

Explanation

The F test is generally one-tailed. A critical value on the right tail of the F distribution is chosen to achieve the desired size of the test. Then, the null hypothesis is rejected if the F statistics is greater than the critical value.

When you are using a statistical package to run linear regression, you could get a regression output that includes the value of the F statistic. Generally, this is obtained by performing the F test of the null hypothesis that each regression coefficient is equal to zero (excluding the coefficient on the intercept).

Maximum Likelihood Procedures’s Test (Wald, Lagrange multiplier, likelihood ratio)

The maximum likelihood estimator of the vectors of coefficients of the normal linear regression model is always equal to the OLS estimator へ. Hence as a consequence, all the usual tests based on maximum likelihood procedures could be employed to conduct hypothesis test.

Hypothesis tests when the OLS estimator is normal asymptotically 

Here we will learn how to perform hypothesis tests about the coefficients of a linear regression model when the OLS estimator is normal, asymptotically.

It can be easily proved that:

the OLS estimator $widehat{eta }$ is asymptotically normal, that is,

N(へ-)d

Where, ddenotes convergence in distribution (as the sample size N tends to infinity), and is a multivariate normal random vector with mean 0 and covariance matrix V; the value of the K*K matrix V depends on the set of assumptions made about the regression model;

it is possible to derive a consistent estimator Vへof V, that is,

VへP V

Where, P denotes convergence in probability (again as N tends to infinity). The estimator Vへ is an easily computable function of the observed inputs xi and outputs yi.

To derive the asymptotic distribution of the test statistics used in hypothesis testing these two properties are used.

The single coefficient’s restriction test (z test)

In the z test the null hypothesis is the basic restriction on a single coefficient:

H0: k=q

Where, k is the kth entry of the vector of coefficients and qℝ .

The test statistic is

zN=へk-qVへkkN

where Vへkk is the kth diagonal entry of the estimator Vへ of the asymptotic covariance matrix.

The test statistic zN, converges in distribution to the standard normal distribution as the sample size N increases. That’s the reason, it is called the z statistic (because the letter z is majorly used to denote the standard normal distribution) and the test is called the z test.

Explanation

When N is comparatively large, we approximate the actual distribution of zN with the standard normal. We then employ the test statistic zN in the usual manner: based on the desired size of the test and the distribution of zN, we determine the critical value(s) and the acceptance region. The null hypothesis is rejected if zN falls outside the acceptance region.

Test of a set of linear restrictions (Chi-square test)

In the Chi-square test, the null hypothesis is a set of L linear restrictions

R=q

where R is a L*K matrix and q is a L*1 vector.

The test statistic is

X2N=(Rへ-q)T[1NRVへRT]-1(Rへ-q)

which converges to a Chi-square distribution with L degrees of freedom. That is the reason, it is called the Chi-square statistic and the test is called the Chi-square test.

Explanation

When setting up the test, the actual distribution of X2N is approximated by the asymptotic one (Chi-square).

Just like the F test,  the Chi-square test is also usually one-tailed. The desired size of the test could be achieved by appropriately choosing a critical value in the right tail of the Chi-square distribution. The null is not accepted if the Chi-square statistics is bigger than the critical value. 

Conclusion

Tests of linear hypotheses are now clearly explained to you in detail. You can read the above article carefully to understand the hypothesis meaning. We have also discussed the tests of the hypothesis along with its types. A proper explanation is also available in the article above.