Penalized Flexible Bayesian Quantile Regression

Ali Alkenani, Rahim Alhamzawi, Keming Yu

Mathematical Science Department, School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, UK.
Mathematical Science Department, School of Information Systems, Computing and Mathematics, Brunel University, Uxbridge, UK&Statistics Department, College of Administration and Economics, Al-Qadisiyah University, Al Diwaniyah, Iraq.

DOI: 10.4236/am.2012.312A296                

Abstract

The selection of predictors plays a crucial role in building a multiple regression model. Indeed, the choice of a suitable subset of predictors can help to improve prediction accuracy and interpretation. In this paper, we propose a flexible Bayesian Lasso and adaptive Lasso quantile regression by introducing a hierarchical model framework approach to enable exact inference and shrinkage of an unimportant coefficient to zero. The error distribution is assumed to be an infinite mixture of Gaussian densities. We have theoretically investigated and numerically compared our proposed methods with Flexible Bayesian quantile regression (FBQR), Lasso quantile regression (LQR) and quantile regression (QR) methods. Simulations and real data studies are conducted under different settings to assess the performance of the proposed methods. The proposed methods perform well in comparison to the other methods in terms of median mean squared error, mean and variance of the absolute correlation criterions. We believe that the proposed methods are useful practically.

Keywords

Adaptive Lasso; Lasso; Mixture of Gaussian Densities; Prior Distribution; Quantile Regression

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	R. Koenker, “Quantile Regression,” Cambridge University Press, Cambridge, 2005. doi:10.1017/CBO9780511754098
[2]	K. Yu, Z. Lu and J. Stander, “Quantile Regression: Applications and Current Research Areas,” The Statistician, Vol. 52, No. 3, 2003, pp. 331-350. doi:10.1111/1467-9884.00363
[3]	R. Koenker and G. Bassett Jr., “Regression Quantiles,” Econometrica, Vol. 46, No. 1, 1978, pp. 33-50. doi:10.2307/1913643
[4]	R. Koenker and J. Machado, “Goodness of Fit and Related Inference Processes for Quantile Regression,” Journal of the American Statistical Association, Vol. 94, No. 448, 1999, pp. 1296-1310. doi:10.1080/01621459.1999.10473882
[5]	K. Yu and R. A. Moyeed, “Bayesian Quantile Regression,” Statistics and Probability Letters, Vol. 54, No. 2, 2001, pp. 437-447. doi:10.1016/S0167-7152(01)00124-9
[6]	E. Tsionas, “Bayesian Quantile Inference,” Journal of Statistical Computation and Simulation, Vol. 73, No. 9, 2003, pp. 659-674. doi:10.1080/0094965031000064463
[7]	K. Yu and J. Stander, “Bayesian Analysis of a Tobit Quantile Regression Model,” Journal of Econometrics, Vol. 137, No. 1, 2007, pp. 260-276. doi:10.1016/j.jeconom.2005.10.002
[8]	M. Geraci and M. Bottai, “Quantile Regression for Longitudinal Data Using the Asymmetric Laplace Distribution,” Biostatistics, Vol. 8, No. 1, 2007, pp. 140-154. doi:10.1093/biostatistics/kxj039
[9]	H. Kozumi and G. Kobayashi, “Gibbs Sampling Methods for Bayesian Quantile Regression,” Technical Report, Kobe University, Kobe, 2009.
[10]	C. Reed and K. Yu, “An Efficient Gibbs Sampler for Bayesian Quantile Regression,” Technical Report, Brunel University, Uxbridge, 2009.
[11]	D. F. Benoit and D. Van den Poel, “Binary Quantile Regression: A Bayesian Approach Based on the Asymmetric Laplace Distribution,” Journal of Applied Econometrics, Vol. 27, No. 7, 2012, pp. 1174-1188. doi:10.1002/jae.1216
[12]	S. Walker and B. Mallick, “A Bayesian Semi-Parametric Accelerated Failure Time Model,” Biometrics, Vol. 55, No. 2, 1999, pp. 477-483. doi:10.1111/j.0006-341X.1999.00477.x
[13]	A. Kottas and A. Gelfand, “Bayesian Semi-Parametric Median Regression Modelling,” Journal of the American Statistical Association, Vol. 96, No. 465, 2001, pp. 1458-1468. doi:10.1198/016214501753382363
[14]	T. Hanson and W. O. Johnson, “Modeling Regression Error with a Mixture of Polya Trees,” Journal of the American Statistical Association, Vol. 97, No. 460, 2002, pp. 1020-1033. doi:10.1198/016214502388618843
[15]	N. L. Hjort, “Topics in Nonparametric Bayesian Statistics,” In: P. J. Green, S. Richardson and N. L. Hjort, Eds., Highly Structured Stochastic Systems, Oxford University Press, Oxford, 2003, pp. 455-475.
[16]	N. L. Hjort and S. Petrone, “Nonparametric Quantile Inference Using Dirichlet Processes,” In: V. Nair, Ed., Advances in Statistical Modeling and Inference: Essays in Honor of Kjell A. Doksum, World Scientific, Singapore City, 2007, pp. 463-492. doi:10.1142/9789812708298_0023
[17]	M. Taddy and A. Kottas, “A Nonparametric Model-Based Approach to Inference for Quantile Regression,” Technical Report, UCSC Department of Applied Math and Statistics, 2007.
[18]	A. Kottas and C. M. Krnjaji?, “Bayesian Nonparametric Modelling in Quantile Regression,” Scandinavian Journal of Statistics, Vol. 36, No. 3, 2009, pp. 297-319.
[19]	B. Reich, H. Bondell and H. Wang, “Flexible Bayesian Quantile Regression for Independent and Clustered Data,” Biostatistics, Vol. 11, No. 2, 2010, pp. 337-352. doi:10.1093/biostatistics/kxp049
[20]	Q. Li, R. Xi and N. Lin, “Bayesian Regularized Quantile Regression,” Bayesian Analysis, Vol. 5, No. 3, 2010, pp. 1-24. doi:10.1214/10-BA521
[21]	R. Tibshirani, “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society Series B, Vol. 58, No. 1, 1996, pp. 267-288.
[22]	J. Fan and R. Li, “Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties,” Journal of the American Statistical Association, Vol. 96, No. 456, 2001, pp. 1348-1360. doi:10.1198/016214501753382273
[23]	H. Zou, “The Adaptive Lasso and Its Oracle Properties,” Journal of the American Statistical Association, Vol. 101, No. 476, 2006, pp. 1418-1429. doi:10.1198/016214506000000735
[24]	T. Park and G. Casella, “The Bayesian Lasso,” Journal of the American Statistical Association, Vol. 103, No. 482, 2008, pp. 681-686. doi:10.1198/016214508000000337
[25]	J. Bradic, J. Fan and W. Wang, “Penalized Composite Quasi-Likelihood for Ultrahigh-Dimensional Variable Selection,” Journal of Royal Statistics Society Series B, Vol. 73, No. 3, 2010, pp. 325-349. doi:10.1111/j.1467-9868.2010.00764.x
[26]	R. Koenker, “Quantile Regression for Longitudinal Data,” Journal of Multivariate Analysis, Vol. 91, No. 1, 2004, pp. 74-89. doi:10.1016/j.jmva.2004.05.006
[27]	Y. Yuan and G. Yin, “Bayesian Quantile Regression for Longitudinal Studies with Non-Ignorable Missing Data,” Biometrics, Vol. 66, No. 1, 2010, pp. 105-114. doi:10.1111/j.1541-0420.2009.01269.x
[28]	H. Wang, G. Li and G. Jiang, “Robust Regression Shrinkage and Consistent Variable Selection through the LAD,” Journal of Business and Economic Statistics, Vol. 25, No. 3, 2007, pp. 347-355. doi:10.1198/073500106000000251
[29]	Y. Li and J. Zhu, “L1-Norm Quantile Regressions,” Jour- nal of Computational and Graphical Statistics, Vol. 17, No. 1, 2008, pp. 1-23.
[30]	Y. Wu and Y. Liu, “Variable Selection in Quantile Regression,” Statistica Sinica, Vol. 19, No. 2, 2009, pp. 801-817.
[31]	R. Alhamzawi, K. Yu and D. Benoit, “Bayesian Adaptive Lasso Quantile Regression,” Statistical Modelling, Vol. 12, No. 3, 2012, pp. 279-297. doi:10.1177/1471082X1101200304
[32]	R. W. Johnson, “Fitting Percentage of Body Fat to Simple Body Measurements,” Journal of Statistics Education, Vol. 4, No. 1, 1996, pp. 236-237.
[33]	X. He, “Quantile Curves without Crossing,” The American Statistician, Vol. 51, No. 2, 1997, pp. 186-192. doi:10.1080/00031305.1997.10473959
[34]	D. F. Andrews and C. L. Mallows, “Scale Mixtures of Normal Distributions,” Journal of Royal Statistics Society Series B, Vol. 36, No. 1, 1974, pp. 99-102.
[35]	J. A. Hoeting, D. Madigan, A. E. Raftery and C. T. Volinsky, “Bayesian Model Averaging: A Tutorial,” Statistical Science, Vol. 14, No. 4, 1999, pp. 382-417.
[36]	C. Leng, M. N. Tran and D. Nott, “Bayesian Adaptive Lasso,” Technical Report, 2010. http://arxiv.org/PScache/arxiv/pdf/1009/1009.2300v1.pdf