ESTIMATING THE HETEROSKEDASTIC/AUTOCORRELATED LINEAR REGRESSION USING MLE

 

The purpose of this session is to show you how to estimate and test the heteroskedastic and/or autocorrelated normal general linear model using MLE.

 

HETEROSKEDASTICITY: Heteroskedasticity can be treated directly in the context of the normal MLE simply by specifying an equation to reflect the form of the heteroskedasticity in place of the variance term in the log likelihood function. This equation can take many different forms to correspond with the type of heteroskedasticity. There could simply be a weight that reflects the variance of each disturbance (White's approach), or the heteroskedasticity could be some linear combination of independent variables that may (or may not) be in the equation for the conditional mean. Other forms include dependent variable heteroskedasticity, where the weighting term is some function of the dependent variable (logs, absolute values, etc.), ARCH, and GARCH. All of these are easily treated in the context of the normal MLE.

 

The example below shows one form, but this program could easily be modified to include all of the other forms.

 

/* This file demonstrates maximum likelihood estimation of normal models with heteroskedasticity of various forms. */

 

Reset $

 

/* The next line will read a data file from Greene, Chapter 11. Change the path to find data */

 

Read ; File=Credit.txt ; Nvar=7 ; Nobs=100 ;

Names=MDR,ACC,AGE,INCOME,AVGEXP,OWNRENT,SELFEMPL $

 

/* Set the sample and generate a required variable to replicate Greene. */

 

Sample; 1-100 $

REJECT ; AVGEXP=0 $

CREATE ; INCOME2=INCOME^2 $

 

/* First, let's estimate the regression using least squares and look at the residuals.  This makes it clear that income is the source

of the heteroskedasticity.  */

 

Regress ; Lhs = AVGEXP ; Rhs =One,AGE,INCOME,INCOME2,OWNRENT ; res=e $

plot ; lhs=income ; rhs=e ; limit=-500,2000 ; endpoints=0,10 $

 

/* Now, let's estimate the heteroskedastic regression the easy way. The following perform's White's procedure for robust standard errors. */

 

Regress ; Lhs = AVGEXP ; Rhs =One,AGE,INCOME,INCOME2,OWNRENT ; hetero $

 

/* You can also specify a weighted least squares procedure. */

 

Regress ; Lhs = AVGEXP ; Rhs =One,AGE,INCOME,INCOME2,OWNRENT ; wts=income $

 

/* Alternatively, you can estimate a "canned" heteroskedastic MLE using the HREG procedure. Note that Rh2 below can contain any variables.

No constant term is necessary, since one is automatically included. */

 

Hreg ; Lhs = AVGEXP ; Rhs =One,AGE,INCOME,INCOME2,OWNRENT ; Rh2=INCOME $

 

/* Now, let's develop our own procedure using the maximize command. First, fit the Normal Maximum Likelihood model, with constant variance.

Use starting values from the linear regression above.*/

 

Maximize ; labels=a0,a1,a2,a3,a4,b0

; Start=0,0,0,0,0,0

; Fcn=-(1/2)*log(2*pi)-(1/2)*log(exp(b0))

-(1/(2*(exp(b0))))*(AVGEXP-a0-a1*AGE-A2*INCOME-A3*INCOME2-A4*OWNRENT)^2 $

 

/* Below save the log likelihood from this model for later hypothesis testing */

 

Calculate ; LR=LOGL $

 

/* Now, let's fit the heteroskedastic Normal Maximum Likelihood model, with variance a function of a right side variable. */

 

Maximize ; labels=a0,a1,a2,a3,a4,b0,b1

; Start=0,0,0,0,0,0,0

; Fcn=-(1/2)*log(2*pi)-(1/2)*log(exp(b0+b1*INCOME))

-(1/(2*(exp(b0+b1*INCOME))))*(AVGEXP-a0-a1*AGE-a2*INCOME-a3*INCOME2-a4*OWNRENT)^2 $

 

/* Below save the log likelihood from this model for hypothesis testing. Then construct a likelihood ratio test for no heteroskedasticity. */

 

Calculate ; LU=LOGL

; List

; LRtest=2*(LU-LR)

; PVALUE=1-Chi(LRtest,1)

; TableVLU=Ctb(.95,1) $

Type ; LRtest,PVALUE,TableVLU $

 

/* Clear the data area below for another program. */

 

Delete ; * $

 

 

 

AUTOCORRELATION: A standard correction for autocorrelation estimates the autocorrelation coefficient simultaneously with the coefficients. This is the approach taken, for example, by Beach and MacKinnon. LIMDEP offers a range of options for estimating autocorrelation models. Below we demonstrate the "canned" procedure, as well as a procedure of our own using the Maximize command. Note that our procedure is not fully efficient because we dropped the first observation. With only 22 observations, this is somewhat of a problem.

 

/* This file demonstrates maximum likelihood estimation of an AR-1 model using the approach of Beach and MacKinnon. */

 

Reset $

 

/* The next line will read a data file. Change the path to find data */

 

Read ; File =OSTROM.DAT ; Nvar=3 ; Nobs=22 ; Names=YEAR,US,USSR $

 

/* Set the Sample */

 

Sample; 1-22 $

 

/* First, let's fit the AR regression the easy way. Note that there are various algorithms which could be used on the Alg option.*/

 

Regr ; Lhs = US ; Rhs = One,USSR ; AR1 ; Alg=MLE $

 

/* Now let's fit the Normal Maximum Likelihood AR1 model. Drop obs 1. Also, since the log likelihood function is for a single observation note that the 1/2 term below is 1/2n, from Beach and MacKinnon */

 

Create ; USLAG1=US[-1] $

Create ; USSRLAG1=USSR[-1] $

Sample ; 2-22 $

Maximize ; labels=a0,a1,Sig,Ro

; Start=15,.9,14,0

; Fcn=-(1/2)*log(2*pi)-(1/2)*log(Sig^2)

-(1/(2*Sig^2))*(US-a0-a1*USSR

-ro*(USLAG1-a0-a1*USSRLAG1))^2

+(1/(2*n))*log(1-ro^2) $

 

/* Clear the data area for another program. */

 

Delete ; * $