A New Algorithm for Generalized Least Squares Factor Analysis with a  Majorization Technique

Kohei Adachi

doi:10.4236/ojs.2015.53020

Home > Journals > Physics & Mathematics > OJS

Open Journal of Statistics > Vol.5 No.3, April 2015

A New Algorithm for Generalized Least Squares Factor Analysis with a Majorization Technique ()

Kohei Adachi

Graduate School of Human Sciences, Osaka University, Osaka, Japan.

DOI: 10.4236/ojs.2015.53020 PDF HTML XML 2,778 Downloads 3,598 Views Citations

Abstract

Factor analysis (FA) is a time-honored multivariate analysis procedure for exploring the factors underlying observed variables. In this paper, we propose a new algorithm for the generalized least squares (GLS) estimation in FA. In the algorithm, a majorization step and diagonal steps are alternately iterated until convergence is reached, where Kiers and ten Berge’s (1992) majorization technique is used for the former step, and the latter ones are formulated as minimizing simple quadratic functions of diagonal matrices. This procedure is named a majorizing-diagonal (MD) algorithm. In contrast to the existing gradient approaches, differential calculus is not used and only elmentary matrix computations are required in the MD algorithm. A simuation study shows that the proposed MD algorithm recovers parameters better than the existing algorithms.

Keywords

Exploratory Factor Analysis, Generalized Least Squares Estimation, Matrix Computations, Majorization

Share and Cite:

Adachi, K. (2015) A New Algorithm for Generalized Least Squares Factor Analysis with a Majorization Technique. Open Journal of Statistics, 5, 165-172. doi: 10.4236/ojs.2015.53020.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Harman, H.H. (1976) Modern Factor Analysis. 3rd Edition, The University of Chicago Press, Chicago.
[2]	Mulaik, S.A. (2010) Foundations of Factor Analysis. 2nd Edition, CRC Press, Boca Raton.
[3]	Yanai, H. and Ichikawa, M. (2007) Factor Analysis. In: Rao, C.R. and Sinharay, S., Eds., Handbook of Statistics, Vol. 26: Psychometrics, Elsevier, Amsterdam, 257-296.
[4]	Anderson, T.W. and Rubin, H. (1956) Statistical Inference in Factor Analysis. In: Neyman, J., Ed., Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Vol. 5, University of California Press, Berkeley, 111-150.
[5]	Lange, K. (2010) Numerical Analysis for Statisticians. 2nd Edition, Springer, New York.
[6]	ten Berge, J.M.F. (1993) Least Squares Optimization in Multivariate Analysis. DSWO Press, Leiden.
[7]	Jöreskog, K.G. (1967) Some Contributions to Maximum Likelihood Factor Analysis. Psychometrika, 32, 443-482. http://dx.doi.org/10.1007/BF02289658
[8]	Jennrich, R.I. and Robinson, S.M. (1969) A Newton-Raphson Algorithm for Maximum Likelihood Factor Analysis. Psychometrika, 34, 111-123. http://dx.doi.org/10.1007/BF02290176
[9]	Jöreskog, K.G. and Goldberger, A.S. (1972) Factor Analysis by Generalized Least Squares. Psychometrika, 37, 243-250. http://dx.doi.org/10.1007/BF02306782.
[10]	Lee, S.Y. (1978) The Gauss-Newton Algorithm for the Weighted Least Squares Factor Analysis. Journal of the Royal Statistical Society: Series D (The Statistician), 27, 103-114. http://dx.doi.org/10.2307/2987906
[11]	Harman, H.H. and Jones, W.H. (1966) Factor Analysis by Minimizing Residuals (Minres). Psychomerika, 31, 351-369. http://dx.doi.org/10.1007/BF02289468
[12]	Rubin, D.B. and Thayer, D.T. (1982) EM Algorithms for ML Factor Analysis. Psychometrika, 47, 69-76. http://dx.doi.org/10.1007/BF02293851
[13]	Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38.
[14]	Groenen, P.J.F. (1993) The Majorization Approach to Multidimensional Scaling: Some Problems and Extensions. DSWO Press, Leiden.
[15]	Unkel, S. and Trendafilov, N.T. (2010) A Majorization Algorithm for Simultaneous Parameter Estimation in Robust Exploratory Factor Analysis. Computational Statistics and Data Analysis, 54, 3348-3358. http://dx.doi.org/10.1016/j.csda.2010.02.003
[16]	Unkel, S. and Trendafilov, N.T. (2010) Simultaneous Parameter Estimation in Exploratory Factor Analysis: An Expository Review. International Statistical Review, 78, 363-382. http://dx.doi.org/10.1111/j.1751-5823.2010.00120.x
[17]	Adachi, K. (2012) Some Contributions to Data-Fitting Factor Analysis with Empirical Comparisons to Covariance-Fitting Factor Analysis. Journal of the Japanese Society of Computational Statistics, 25, 25-38. http://dx.doi.org/10.5183/jjscs.1106001_197
[18]	Kiers, H.A.L. and ten Berge, J.M.F. (1992) Minimization of a Class of Matrix Trace Functions by Means of Refined Majorization. Psychometrika, 57, 371-382. http://dx.doi.org/10.1007/BF02295425
[19]	Kiers, H.A.L. (1990) Majorization as a Tool for Optimizing a Class of Matrix Functions. Psychometrika, 55, 417-428. http://dx.doi.org/10.1007/BF02294758
[20]	Costa, P.T. and McCrae, R.R. (1992) NEO PI-R Professional Manual: Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI). Psychological Assessment Resources, Odessa, FL.
[21]	Kaiser, H.F. (1958) The Varimax Criterion for Analytic Rotation in Factor Analysis. Psychometrika, 23, 187-200. http://dx.doi.org/10.1007/BF02289233
[22]	Gower, J.C. and Dijksterhuis, G.B. (2004) Procrustes Problems. Oxford University Press, Oxford. http://dx.doi.org/10.1093/acprof:oso/9780198510581.001.0001
[23]	Adachi, K. (2013) Factor Analysis with EM Algorithm Never Gives Improper Solutions When Sample Covariance and Initial Parameter Matrices Are Proper. Psychometrika, 78, 380-394. http://dx.doi.org/10.1007/s11336-012-9299-8