Towards More Efficient Assessments: Increasing Information from Objective Examinations

Abstract

Objective examination questions are widely used to assess students knowledge, but the standard MCQ with a stem followed by 4 - 6 possible answers one of which is chosen as correct is very inefficient. Simple changes to the format can treble the information gained. Information theory is the tool for assessing the information content of electronic and other communications. The “bit” is the unit of information and equals on true/false choice. I have applied basic information theory to objective questions. The standard MCQ with a stem, a choice of 4 possible answers (mark one answer true) and an expected correct answer rate of 70% yields 1.36 bits. A MTFQ with 4 choices where the student must answer true/false to every possible answer gives 3.52 bits of information. By adding a “dont know” option the same MTFQ gives 4.72 bits of information, 350% of the standard MCQ. Thirty MTFQ with dont know give the same information about students knowledge as 100 standard MCQs. The effort needed to set, sit and mark is the same for both. Small changes to the format of objective questions give large gains in efficiency. We should balance these gains against possible disadvantages.

Share and Cite:

Dugdale, A. (2013). Towards More Efficient Assessments: Increasing Information from Objective Examinations. Creative Education, 4, 39-41. doi: 10.4236/ce.2013.46A007.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Ben-Simon, A., Budescu, D. V., & Nevo, B. (1997). A comparative study of measures of partial knowledge in multiple-choice questions. Applied Psychological Measurement, 21, 65-88. doi:10.1177/0146621697211006
[2] Harden, R. M. G., Brown, R. A., Biran, L. A., Dallas Ross, W. P., & Wakeford, R. E. (1976). Multiple choice questions, to guess or not to guess. Medical Education, 10, 27-32. doi:10.1111/j.1365-2923.1976.tb00527.x
[3] Moulton, R. (2010). A short simple introduction to information theory. URL. www.moultano.wordpress.com/article/
[4] Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27, 379-423, 623-656.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.