ESTIMATING A LINEAR REGRESSION USING MLE

 

The purpose of this session is to introduce you to the MLE of the normal general linear model. This approach to linear regression forms the statistical basis for hypothesis testing found in most econometrics textbooks. Thus, the normal general linear model is important from the standpoint of justifying least squares. More important, this model serves as a gateway for understanding maximum likelihood estimation of many time series models, models with heteroskedastic disturbances, and models with non-normal disturbances.

/* This file demonstrates maximum likelihood estimation of normal linear regression models */

Reset $

/* The next line will read a data file. Change the path to find data */
Read ; File=OSTROM.DAT ; Nvar=3 ; Nobs=22 ; Names=YEAR,US,USSR $

/* Set the sample from observation 1 through 22 */

Sample; 1-22 $

/* Now let's print the data and compute descriptive statistics */

List ; Year,US,USSR $
Dstat ; Rhs=Year,US,USSR ; Output=3 $

/* Now let's fit a regression the easy way, plot and save the residuals
and do a Wald test for whether the coefficient on USSR=0. */

Regress ; Lhs = US ; Rhs = One,USSR ; plot ; res=e ; Test: B(2)=0 $

/* Now let's fit the Normal Maximum Likelihood model. Notice that
Limdep asks for the likelihood function for a single observation
and does the summation internally. Thus, N/2 becomes 1/2 below */

Maximize ; labels=a0,a1,Sig
; Start=15,.9,13
; Fcn=-(1/2)*log(2*pi)-(1/2)*log(Sig^2)
-(1/(2*Sig^2))*(US-a0-a1*USSR)^2
; Alg=Newton
; Output=4 $

/* Note that Alg=Newton calls for optimization by Newton's method. Other available algorithms are DFP, BFGS, BHHH, and the Method of Steepest Ascent. The default is DFP, which usually works quite well. The Output= # command sets the technical details of estimation in the output file. Note that the highest level is 4. We could also have specified a convergence criterion using Tlf, Tlb, or Tlg. The default is change in the function (Tlf), coefficients (Tlb), or gradient (Tlg) less than a small constant, c=.0001. */

/* The maximum likelihood estimate of the variance is biased. Correct by multiplying by n/n-k */

Calc ; List ; SigUnb = Sqr(Sig^2*(n/(n-2))) $

/* The square of the t-statistic on USSR is a Wald test that USSR=0. We might want to test this hypothesis with other methods, including either a likelihood ratio or LaGrange multiplier. Therefore,we save the log-likelihood for later testing below. */

CALC ; LIST ; LU=LOGL $

/* Now let's do a likelihood ratio test that the coefficient on USSR=0 using LU from above.*/

Maximize ; labels=a0,Sig
; Start=140,77
; Fcn=-(1/2)*log(2*pi)-(1/2)*log(Sig^2)
-(1/(2*Sig^2))*(US-a0)^2
; Alg=Newton $
CALC ; LIST ; LR=LOGL $
CALC ; LIST ; LRATIO= 2 * ( LU - LR)
; PVALUE=1-CHI(LRATIO,1)
; TABLEVLU=CTB(.95,1) $

/* We can also do a LaGrange Multiplier test that the coefficient on USSR=0. This can be done as follows. */

Maximize ; labels=Sig,a0
; Start=77,140
; Fcn=-(1/2)*log(2*pi)-(1/2)*log(Sig^2)
-(1/(2*Sig^2))*(US-a0)^2
; Alg=Newton $

Maximize ; labels=Sig,a0,a1
; Start=B,0
; Fcn=-(1/2)*log(2*pi)-(1/2)*log(Sig^2)
-(1/(2*Sig^2))*(US-a0-a1*USSR)^2
; Maxit=0 $

/* Notice above that the B in the second Start=command feeds in the values of the final coefficients from the prior estimation.*/

/* In some instances it will be useful to perform matrix calculations on MLE outputs. For example, in computing quadratic forms. You may also want to use the NAMELIST command in simplifying your commands. As an example of using matrices, let's illustrate the preceding regression using some of the matrix capabilities of LIMDEP */

/* First, define the data in terms of named matrices. Note that we could also have read the data directly into the matrix on the READ command by using the MATRIX= option. */

Namelist ; X=one,USSR ; y=US $

/* Compute coefficient estimates using matrices. Note that < > below is LIMDEP's language for matrix inverse. */

Matrix ; List ; bols=<x'x>*x'y $

/* Compute predicted values and residuals using matrices */

Matrix ; List ; yhat=x*bols $
Matrix ; List ; errors=y-yhat $

/* Compute the maximum likelihood estimate of the error variance */

Matrix ; List ; SSE=errors'errors $
Calc ; List ; S2=(1/n)*SSE $

/* Compute the maximum likelihood covariance matrix of coefficients. The diagonal of this matrix contains the standard errors. */

Matrix ; List ; CovB=S2*<x'x> $

 

/* Clear the data area for another program. */

Delete ; * $

_