Vuong's closeness test

In statistics, the Vuong closeness test is likelihood-ratio-based test for model selection using the Kullback-Leibler information criterion. This statistic makes probabilistic statements about two models. They can be nested, non-nested or overlapping. The statistic tests the null hypothesis that the two models are equally close to the true data generating process, against the alternative that one model is closer. It cannot make any decision whether the "closer" model is the true model.

With non-nested models and iid exogenous variables, model 1 (2) is preferred with significance level α, if the z statistic

$Z=\frac{LR_N(\beta_{ML,1},\beta_{ML,2})} {\sqrt{N}\omega_N}$

with

${LR_N(\beta_{ML,1},\beta_{ML,2})} = L^1_N-L^2_N-\frac{K_1-K_2} {2} \log N$

exceeds the positive (falls below the negative) (1 − α)-quantile of the standard normal distribution. Here K₁ and K₂ are the numbers of parameters in models 1 and 2 respectively.

The numerator is the difference between the maximum likelihoods of the two models, corrected for the number of coefficients analogous to the BIC, the term in the denominator of the expression for Z, $\omega_N \,$, is defined by setting $\omega_N^2$ equal to either the mean of the squares of the pointwise log-likelihood ratios $\ell_i\,$, or to the sample variance of these values, where

$\ell_i = \log\frac{f_1(y_i|x_i,\beta_{ML,1})}{f_2(y_i|x_i,\beta_{ML,2})}.$

For nested or overlapping models the statistic

$2LR_N(\beta_{ML,1},\beta_{ML,2})\,$

has to be compared to critical values from a weighted sum of chi squared distributions. This can be approximated by a gamma distribution:

$M_m(.,\bold\lambda)\sim \Gamma(b,p)\,$

with

$\bold\lambda=(\lambda_1, \lambda_2, \dots, \lambda_m),\,$

$m=K_1+K_2,\ b=\frac 1 2 \frac {\sum\lambda_i} {\sum\lambda_i^2}$

and

$p=\frac 1 2 \frac {{(\sum\lambda i)}^2} {\sum\lambda_i^2}.$

$\bold\lambda$ is a vector of eigenvalues of a matrix of conditional expectations. The computation is quite difficult, so that in the overlapping and nested case many authors only derive statements from a subjective evaluation of the Z statistic (is it subjectively "big enough" to accept my hypothesis?).

Vuong's test for non-nested models has sometimes been used to determine whether zero-inflation is present in data. As a given model and its zero-inflated counterpart are not non-nested, this is an erroneous use of the test

References

• Vuong, Quang H. (1989). "Likelihood Ratio Tests for Model Selection and non-nested Hypotheses". Econometrica. 57 (2): 307–333. JSTOR 1912557.

• Genius, Margarita; Strazzera, Elisabetta (2002). "A note about model selection and tests for non-nested contingent valuation models". Economics Letters. 74 (3): 363–370. doi:10.1016/S0165-1765(01)00566-3.

• Wilson, Paul (2015). "The Misuse of The Vuong Test For Non-Nested Models to Test for Zero-Inflation". Economics Letters. 127 (2): 151–153. doi:10.1016/j.econlet.2014.12.029.