THE HETEROSKEDASTIC LINEAR REGRESSION MODEL

 

The purpose of this session is to show you how to estimate and test the heteroskedastic general linear models.

HETEROSKEDASTICITY: There are various ways of testing for and estimating heteroskedastic models within LIMDEP. You can use the canned procedures. Alternatively, heteroskedasticity can be treated directly in the context of the normal MLE simply by specifying an equation to reflect the form of the heteroskedasticity in place of the variance term in the log likelihood function. Using either approach, the heteroskedasticity can take many different forms to correspond with the type of heteroskedasticity. There could simply be a weight that reflects the variance of each disturbance (White's approach), or the heteroskedasticity could be some linear combination of independent variables that may (or may not) be in the equation for the conditional mean. Other forms include dependent variable heteroskedasticity, where the weighting term is some function of the dependent variable (logs, absolute values, etc.), ARCH, and GARCH.

/* First, let's replicate the results in chapter 11 of Greene. */

RESET $
DELETE ; * $
/*===============================================
Example 11.1. Heteroscedastic Regression
*/===============================================
Read ; Nobs = 100 ; Nvar = 7
; Names =
Derogs,Card,Age,Income,Exp,OwnRent,SelfEmpl $
0 1 38 4.52 124.98 1 0
0 1 33 2.42 9.85 0 0
0 1 34 4.50 15.00 1 0
0 1 31 2.54 137.87 0 0
0 1 32 9.79 546.50 1 0
0 1 23 2.50 92.00 0 0
0 1 28 3.96 40.83 0 0
0 1 29 2.37 150.79 1 0
0 1 37 3.80 777.82 1 0
0 1 28 3.20 52.58 0 0
0 1 31 3.95 256.66 1 0
0 0 42 1.98 0.00 1 0
0 0 30 1.73 0.00 1 0
0 1 29 2.45 78.87 1 0
0 1 35 1.91 42.62 1 0
0 1 41 3.20 335.43 1 0
0 1 40 4.00 248.72 1 0
7 0 30 3.00 0.00 1 0
0 1 40 10.00 548.03 1 1
3 0 46 3.40 0.00 0 0
0 1 35 2.35 43.34 1 0
1 0 25 1.88 0.00 0 0
0 1 34 2.00 218.52 1 0
1 1 36 4.00 170.64 0 0
0 1 43 5.14 37.58 1 0
0 1 30 4.51 502.20 0 0
0 0 22 3.84 0.00 0 1
0 1 22 1.50 73.18 0 0
0 0 34 2.50 0.00 1 0
0 1 40 5.50 1532.77 1 0
0 1 22 2.03 42.69 0 0
1 1 29 3.20 417.83 0 0
1 0 25 3.15 0.00 1 0
0 1 21 2.47 552.72 1 0
0 1 24 3.00 222.54 0 0
0 1 43 3.54 541.30 1 0
0 0 43 2.28 0.00 0 0
0 1 37 5.70 568.77 1 0
0 1 27 3.50 344.47 0 0
0 1 28 4.60 405.35 1 0
0 1 26 3.00 310.94 1 0
0 1 23 2.59 53.65 0 0
0 1 30 1.51 63.92 0 0
0 1 30 1.85 165.85 0 0
0 1 38 2.60 9.58 0 0
0 0 28 1.80 0.00 0 1
0 1 36 2.00 319.49 0 0
0 0 38 3.26 0.00 0 0
0 1 26 2.35 83.08 0 0
0 1 28 7.00 644.83 1 0
0 0 50 3.60 0.00 0 0
0 1 24 2.00 93.20 0 0
0 1 21 1.70 105.04 0 0
0 1 24 2.80 34.13 0 0
0 1 26 2.40 41.19 0 0
1 1 33 3.00 169.89 0 0
0 1 34 4.80 1898.03 0 0
0 1 33 3.18 810.39 0 0
0 0 45 1.80 0.00 0 0
0 1 21 1.50 32.78 0 0
2 1 25 3.00 95.80 0 0
0 1 27 2.28 27.78 0 0
0 1 26 2.80 215.07 0 0
0 1 22 2.70 79.51 0 0
3 0 27 4.90 0.00 1 0
0 0 26 2.50 0.00 0 1
0 1 41 6.00 306.03 0 1
0 1 42 3.90 104.54 0 0
0 0 22 5.10 0.00 0 0
0 1 25 3.07 642.47 0 0
0 1 31 2.46 308.05 1 0
0 1 27 2.00 186.35 0 0
0 1 33 3.25 56.15 0 0
0 1 37 2.72 129.37 0 0
0 1 27 2.20 93.11 0 0
1 0 24 4.10 0.00 0 0
0 1 24 3.75 292.66 0 0
0 1 25 2.88 98.46 0 0
0 1 36 3.05 258.55 0 0
0 1 33 2.55 101.68 0 0
0 0 33 4.00 0.00 0 0
1 1 55 2.64 65.25 1 0
0 1 20 1.65 108.61 0 0
0 1 29 2.40 49.56 0 0
3 0 40 3.71 0.00 0 0
0 1 41 7.24 235.57 1 0
0 0 41 4.39 0.00 1 0
0 0 35 3.30 0.00 1 0
0 0 24 2.30 0.00 0 0
1 0 54 4.18 0.00 0 0
2 0 34 2.49 0.00 0 0
0 0 45 2.81 0.00 1 0
0 1 43 2.40 68.38 0 0
4 0 35 1.50 0.00 0 0
2 0 36 8.40 0.00 0 0
0 1 22 1.56 0.00 0 0
1 1 33 6.00 474.15 1 0
1 1 25 3.60 234.05 0 0
0 1 26 5.00 451.20 1 0
0 1 46 5.50 251.52 1 0
?
Sample ; 1-100 $
Create ; Incomesq = Income^2 $
Namelist ; X = One,Age,OwnRent,Income,Incomesq $
Reject ; Exp = 0 $
?----------------------------------------------------------
? Heteroscedastic Regression
?----------------------------------------------------------
Regress ; Lhs = Exp ; Rhs=X ; Res = u $ (OLS)
?
? Test for presence of income and square in regression
?
Regress ; Lhs = Exp ; Rhs = X ; Cls:b(4)=0,b(5)=0 $
?
Plot ; Lhs = Income ; Rhs = u ; Grid
; title=Plot of Residuals Against Income $
/*====================================================
Example 11.2. Using the White Estimator
*/====================================================
?
Regress ; Lhs = Exp;Rhs=X ; Hetero $ (White)
?
? See if income and square are still significant when the
? White corrected covariance matrix is used. (Yes)
?

Matrix ; bi=b(4:5)
; Vbi=part(varb,4,5,4,5)
; List ; WaldStat = bi'<Vbi>bi$
?
? Davidson and MacKinnon recommended corrections if it is still significant.
?
Calc ; Scale = N/(N-Col(X)) $
Matrix ; DM1 = Scale * VARB ; Stat(B,DM1) $
Matrix statistical results: Coefficients=B Variance=DM1
Matrix ; XXI=<X'X> $
Create ; v = u^2 /(1-qfr(X,XXI))$
Matrix ; DM2=XXI * X'[v]X * XXI $
Matrix ; Stat(b,DM2)$
Matrix statistical results: Coefficients=B Variance=DM2
?
?
/*====================================================
Testing for Heteroscedasticity
*/====================================================
?
? Get a set of residuals
?
Regress ; Lhs = Exp ; Rhs = X ; Res = E $
Create ; U = E^2 $
?
? White's test.
?
Create ; Age2=Age^2 ; Income3=Income^3 ; Income4=Income^4
; Agerent=Age*Ownrent ; Ageinc=Age*Income
; Ageinc2=Age*Incomesq ; Rentinc=Ownrent*Income
; Rentinc2=Ownrent*Incomesq$

Namelist ; Z=X,Age2,Agerent,Ageinc,Ageinc2,RentInc,Rentinc2,
income3,income4$
Regr ; Lhs=u ; Rhs=z$
Calc ; List ; White=n * Rsqrd $
?
? Goldfeld and Quandt test
?
Sort ; Lhs = Income ; Rhs = * $ (Carry all variables)
Create ; j=trn(1,1)$ (Sequence numbers)
Reject ; j > 36 $
Regress ; Lhs = Exp ; Rhs = X $
Calc ; E1E1 = Sumsqdev $
Sample ;all $
Reject ; Exp = 0 $
Reject ; j <= 36 $
Regress ; Lhs = Exp ; Rhs = X $
Calc ; E2E2 = Sumsqdev ; List ; GQ = e2e2/e1e1 $
?
? Breusch and Pagan - Godfrey LM test
?
Sample ; All $
Reject ; Exp=0 $
Calc ; een = e'e/n $
Create ; g = u/een - 1$
Namelist ; Z = One,Income,IncomeSQ $
Matrix ; list ; LMtest = .5 * g'Z * <Z'Z> * Z'g $
?
? Koenker and Bassett variant
?
Create ; devu=u-een $
Calc ; Vdu=Devu'Devu/n $
Matrix ; list ; LMKB=1/Vdu * devu'Z * <Z'Z> * Z'devu $
?
? Glesjer's tests. Uses White VC and tests a=0.
?
Create ; w1=u ; w2=abs(e) ; w3=log(w2)$
Regress ; Lhs=w1 ; Rhs=Z ; het ; cls:b(2)=0,b(3)=0 $
?
Regress ; Lhs=w2 ; Rhs=Z ; het ; cls:b(2)=0,b(3)=0 $
?
Regress ; Lhs=w3 ; Rhs=Z ; het ; cls:b(2)=0,b(3)=0 $
?
/*=======================================================
Two Step Estimation of a Heteroscedastic
Regression
*/=======================================================
?----------------------------------------------------------
? 1. Unweighted Least Squares
Regress ; Lhs = Exp ; Rhs = X $
?
? 2. Variance proportional to income
Create ; wt = 1/income $
Regress ; Lhs = Exp ; Rhs = X ; Wts = wt $
?
? 3. Variance proportional to income squared
Create ; wt = 1/IncomeSq $
Regress ; Lhs = Exp ; Rhs = X ; Wts = wt $
?
? 4. Variance proportional to z'alpha. Z=Income,IncomeSq
?
Regress ; Lhs=Exp ; Rhs=X ; res=e$ Initial OLS
Create ; ee=e*e$
Namelist ; Z = income,incomesq $
Matrix ; alpha = <Z'Z>*Z'ee $
Create ; wt = 1/Z'Alpha $
Regress ; Lhs = Exp ; Rhs = X ; Wts = wt $
?
? 5. Variance proportional to (z'alpha)^2. Z=Income,IncomeSq
?
Create ; ee=abs(e)$
Matrix ; alpha = <Z'Z>*Z'ee $
Create ; wt = 1/Z'Alpha $
Regress ; Lhs = Exp ; Rhs = X ; Wts = wt $
?
? 6. Variance proportional to exp(z'alpha)Z=Income,IncomeSq
?
Create ; ee=log((abs(e))^2)$
Namelist ; Z=one,income,incomesq$
Matrix ; alpha = <Z'Z>*Z'ee $
Create ; wt = 1/exp(Z'Alpha) $
Regress ; Lhs = Exp ; Rhs = X ; Wts = wt $
?
?Two Step Estimation of a Heteroscedastic Regression
?
Regress ; Lhs = Exp ; Rhs = X ; Res = e $
Create ; loginc=log(income)$
Namelist ; Z=one,loginc$
Procedure
Create ; Logee=log(e*e)$
Matrix ; Bold=B ; Alpha = <Z'Z> * Z'Logee$
Create ; Wt= 1/exp(Z'alpha)$
Regress; Lhs=exp ; Rhs=X ; Wts=wt ; Res=e$
Matrix ; Delta=B-Bold ; Cnv=Delta'Delta$
Endproc
Calc ; Cnv=1$
Execute Procedure ; While Cnv > .00001 $
Hreg ; Lhs=Exp ; Rhs = X ; Rh2 = Loginc $
/*=======================================================
Maximum Likelihood Estimation
*/=======================================================
?
? This routine produces a plot of the concentrated log
? likelihood
?
Create ; Loginc = Log(Income) $
Calc ; SumL = Sum(Loginc) ; i = 0 $
Matrix ; Alpha = init(101,1,0.); LogLHREG=Alpha$
Procedure
Create ; Wt= 1/income^a$
Matrix ; Bw = <X'[wt]x> * X'[Wt]Exp $
Create ; GSQ=(exp-X'bw)^2 * Wt$
Calc ; Ssw=Sum(GSQ)/n
; loglw=-n/2*(1+log(2*pi)+log(ssw))-(a/2)*SumL
; i=i+1$
Matrix ; Alpha(i)=a ; LogLHREG(i)=loglw$
Endproc
Execute; a=0,5,.05$
?
Mplot ; Lhs=alpha;Rhs=Loglhreg;fill;grid
; Title=Plot of Concentrated Log Likelihood$
/*=======================================================
Multiplicative Heteroscedasticity
*/=======================================================
Namelist ; X = One,Age,OwnRent,Income,IncomeSq
; Z = One,Income,IncomeSq $
? Collect ordinary least squares results.
Regress ; Lhs = Exp ; Rhs = X ; Res = E $
Calc ; LIST; LoglR = LogL ; c1 = log(e'e/n) ; Cnv=1 ; iter=0$
Matrix ; LIST; c = [c1/0/0] ; beta = b $
Namelist ; Z = One,Income,IncomeSq $
Create ; h=e*e/exp(c1) ; g = h-1
; Wts=1/Exp(Z'c) ; logfi=c1$
Matrix ; List ; LMTest=.5*g'Z*<Z'[h]Z> * Z'g $
?
? This is actually unnecessary, as the HREG command
? fits this model. We do it here for illustration.
?
Procedure for iteration for multiplicative model.
Matrix ; Beta = <X'[Wts]X> * X'[Wts]Exp
; delta = <Z'Z> * Z'g ; c=c+Delta $
Create ; E = Exp - X'Beta ; Wts=1/exp(Z'c)
; h = e*e *Wts ; g=h-1 ; logfi=Z'c$
Calc ; LoglU = -n/2*log(2*pi)-1/2*Sum(logfi)-1/2*sum(h)
; Iter=Iter+1 ; Cnv = Delta'Delta $
EndProcedure
Execute ; While Cnv > .0000001$
?
? Display estimation results. Then test hypothesis.
Matrix ; Vbeta =<X'[wts]X> ; Stat(beta,Vbeta)
; Vc =2*<Z'Z> ; Stat(c,Vc)
; Alpha = Part(c,2,3) ; Valpha=Part(Vc,2,3,2,3)
; List ; WaldTest = Alpha' * <Valpha> * Alpha $
Matrix statistical results: Coefficients=BETA Variance=VBETA
Matrix statistical results: Coefficients=C Variance=VC
Wald ; fn1 = sqr(exp(gamma1))
; start = c ; Var = VC ; Labels = gamma1,c2,c3 $
Calc ; List ; LogLR ; LogLU ; LRTest = -2*(LogLR - LogLU) $