Share This Article:

Student’s Academic Efficacy or Inefficacy? An Example on How to Evaluate the Psychometric Properties of a Measuring Instrument and Evaluate the Effects of Item Wording

Abstract Full-Text HTML Download Download as PDF (Size:3752KB) PP. 484-493
DOI: 10.4236/ojs.2014.46046    3,341 Downloads   3,975 Views   Citations

ABSTRACT

This study evaluated the effect of item inversion on the construct validity and reliability of psychometric scales and proposed a theoretical framework for the evaluation of the psychometric properties of data gathered with psychometric instruments. To this propose, we used the Maslach Burnout Inventory, which is the most used psychometric inventory to measure burnout in different professional context (Students, Teachers, Police, Doctors, Nurses, etc…). The version of the MBI used was the MBI-Student Survey (MBI-SS). This inventory is composed of three key dimensions: Exhaustion, Cynicism and Professional Efficacy. The two first dimensions—which have positive formulated items—are moderate to strong positive correlated, and show moderate to strong negative correlations with the 3rd dimension—which has negative formulated items. We tested the hypothesis that, in college students, formulating the 3rd dimension of burnout as Inefficacy (reverting the negatively worded items in the Efficacy dimension) improves the correlation of the 3rd dimension with the other two dimensions, improves its internal consistency, and the overall MBI-SS’ construct validity and reliability. Confirmatory factor analysis results, estimated by Maximum Likelihood, revealed adequate factorial fit for both forms of the MBI-SS (with Efficacy) vs. the MBI-SSi (with Inefficacy). Also both forms showed adequate convergent and discriminant related validity. However, reliability and convergent validity were higher for the MBI-SSi. There were also stronger (positive) correlations between the 3 factors in MBI-SSi than the ones observed in MBI-SS. Results show that positively rewording of the 3rd dimension of the MBI-SS improves its validity and reliability. We therefore propose that the 3rd dimension of the MBI-SS should be named Professional Inefficacy and its items should be positively worded.

Conflicts of Interest

The authors declare no conflicts of interest.

Cite this paper

Maroco, J. , Maroco, A. and Campos, J. (2014) Student’s Academic Efficacy or Inefficacy? An Example on How to Evaluate the Psychometric Properties of a Measuring Instrument and Evaluate the Effects of Item Wording. Open Journal of Statistics, 4, 484-493. doi: 10.4236/ojs.2014.46046.

References

[1] Pais-Ribeiro, J.L. (2013) Medida na Avaliação Psicológica. Psicologia, Saúde & Doenças, 14, 245-263.
[2] Borsboom, D. and Mellenbergh, G.J. (2002) True Scores, Latent Variables, and Constructs: A Comment on Schmidt and Hunter. Intelligence, 30, 505-514.
http://dx.doi.org/10.1016/S0160-2896(02)00082-X
[3] Bollen, K.A. (2002) Latent Variables in Psychology and the Social Sciences. Annual Review of Psychology, 53, 605-634. http://dx.doi.org/10.1146/annurev.psych.53.100901.135239
[4] Zucoloto, M.L., et al. (2014) Psychometric Properties of the Oral Health Impact Profile and New Methodological Approach. Journal of Dental Research, 93, 645-650.
http://dx.doi.org/10.1177/0022034514533798
[5] Anastasi, A. (1988) Psychological testing. 6th Edition, Macmillan Publishing Company, New York.
[6] Streiner, D.L. and Norman, G.R. (2008) Health Measurement Scales: A Practical Guide to Their Development and Use. 4th Edition, Oxford University Press, New York.
http://dx.doi.org/10.1093/acprof:oso/9780199231881.001.0001
[7] Campos, J.A.D.B., Bonafé, F.S.S., Dovigo, L.N. and Maroco, J. (2013) Avaliação Psicométrica da Escala de Atitudes em Relação à Estatística. Revista Brasileira de Biometria, 31, 327-337.
[8] Hambleton, R.K., Swaminathan, H. and Rogers, H.J. (1991) Fundamentals of Item Response Theory. Sage, Newbury Park, CA.
[9] Muthén, B. (1984) A General Structural Equation Model with Dichotomous, Ordered Categorical, and Continuous Latent Variable Indicators. Psychometrika, 49, 115-132.
http://dx.doi.org/10.1007/BF02294210
[10] Muthén, L.K. and Muthén, B. (2001) Mplus: Statistical Analysis with Latent Variables, User’s Guide. Muthén & Muthén, Los Angeles.
[11] Maroco, J. (2014) Análise de Equações Estruturais. 2nd Edition, Report Number, Lisboa.
[12] Elosua, P. and Iliescu, D. (2012) Tests in Europe: Where We Are and Where We Should Go. International Journal of Testing, 12, 157-175.
http://dx.doi.org/10.1080/15305058.2012.657316
[13] Kane, M. (2013) Validating the Interpretations and Uses of the Test Scores. Journal of Educational Measurement, 50, 1-73. http://dx.doi.org/10.1111/jedm.12000
[14] Schaufeli, W.B., Martinez, I.M., Pinto, A.M., Salanova, M. and Bakker, A.B. (2002) Burnout and Engagement in University Students—A Cross-National Study. Journal of Cross-Cultural Psychology, 33, 464-481.
http://dx.doi.org/10.1177/0022022102033005003
[15] Hair, J.F., Black, W.C., Babin, B., Anderson, R.E. and Tatham, R.L. (2005) Multivariate data analysis. 6th Edition, Prentice Hall, Upper Saddle River.
[16] Campos, J.A.D.B. and Maroco, J. (2012) Maslach Burnout Inventory—Student Survey: Portugal-Brazil Cross-Cultural Adaptation. Revista de Saúde Pública, 46, 816-824.
http://dx.doi.org/10.1590/S0034-89102012000500008
[17] Bresó, E., Salanova, M. and Schaufeli, W. (2007) In Search of the “Third Dimension” of Burnout: Efficacy or Inefficacy? Applied Psychology, 56, 460-478.
http://dx.doi.org/10.1111/j.1464-0597.2007.00290.x
[18] Nunnally, J.C. and Bernstein, I.H. (1994) Psychometric Theory. 3rd Edition, McGraw-Hill, New York.
[19] AERA, APA and NCME (1999) The Standards for Educational and Psychological Testing. Aera Publication, Washington DC.
[20] Kline, R.B. (1998) Principles and Practice of Structural Equation Modeling. The Guilford Press, New York.
[21] Bollen, K.A. (1989) Structural Equations with Latent Variables. John Wiley & Sons, New York.
[22] Byrne, B.M. (2001) Structural Equation Modeling with Amos: Basic Concepts, Applications and Programming. Lawrence Erlbaum Associates, New Jersey.
[23] Fornell, C. and Larcker, D.F. (1981) Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research, 18, 39-50.
http://dx.doi.org/10.2307/3151312
[24] Maroco, J. and Garcia-Marques, T. (2006) Qual a fiabilidade do alfa de Cronbach? Questões antigas e soluções modernas? Laboratório de Psicologia, 4, 65-90.
[25] Batista-Foguet, J.M., Coenders, G. and Alonso, J. (2004) Análisis factorial confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud. Medicina Clinica, 122, 21-27.
http://dx.doi.org/10.1157/13057542
[26] Yeh, W.Y., Cheng, Y., Chen, C.J., Hu, P.Y. and Kristensen, T.S. (2007) Psychometric Properties of the Chinese Version of Copenhagen Burnout Inventory among Employees in Two Companies in Taiwan. International Journal of Behavioral Medicine, 14, 126-133.
http://dx.doi.org/10.1007/BF03000183
[27] Campos, J.A.D.B., Carlotto, M.S. and Maroco, J. (2013) Copenhagen Burnout Inventory—Student Version: Adaptation and Transcultural Validation for Portugal and Brazil. Psicologia: Reflexão e Crítica, 26, 87-97.
http://dx.doi.org/10.1590/S0102-79722013000100010
[28] Halbesleben, J.R.B. and Demerouti, E. (2005) The Construct Validity of an Alternative Measure of Burnout: Investigating the English Translation of the Oldenburg Burnout Inventory. Work and Stress, 19, 208-220.
http://dx.doi.org/10.1080/02678370500340728
[29] Kristensen, T.S., Borritz, M., Villadsen, E. and Christensen, K.B. (2005) The Copenhagen Burnout Inventory: A New Tool for the Assessment of Burnout. Work and Stress, 19, 192-207.
http://dx.doi.org/10.1080/02678370500297720
[30] Schaufeli, W.B. and Salanova, M. (2007) Efficacy or Inefficacy, That’s the Question: Burnout and Work Engagement, and Their Relationships with Efficacy Beliefs. Anxiety, Stress and Coping, 20, 177-196.
http://dx.doi.org/10.1080/10615800701217878
[31] Simbula, S. and Guglielmi, D. (2010) Depersonalization or Cynicism, Efficacy or Inefficacy: What Are the Dimensions of Teacher Burnout? European Journal of Psychology of Education, 25, 301-314.
http://dx.doi.org/10.1007/s10212-010-0017-6
[32] Schaufeli, W.B., Leiter, M.P. and Maslach, C. (2009) Burnout: 35 Years of Research and Practice. Career Development International, 14, 204-220.
http://dx.doi.org/10.1108/13620430910966406

  
comments powered by Disqus

Copyright © 2019 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.