Scenario-Based, Single Best, Multiple-Choice Questions (SB-SB-MCQs) in Basic Medical Sciences: An Exploratory Study about the Staff Awareness, Knowledge and Difficulties Encountered ()
1. Introduction
To deliver effective medical science education, the assessment system must have the ability to examine students’ knowledge, learning attitude and practical skills [1]. The assessment of competency changes students learning behavior and improves information gaining from the teachers [2] [3]. Multiple-choice questions (MCQs) have been known and in use as a tool of assessment for a long time in both under and postgraduate medical examinations [4] [5]. They are consistent, fair, unbiased, cost effective, trustworthy, and easily differentiate between the high and low achievers [6]. A multiple-choice question consists of a problem or story, known as the stem, a lead-in question, and a number of suggested answers, known as alternatives or options. The answers consist of correct one known as SBA (Single Best Answer) and the other suggested answers are known as distractors [7]. Scenario-based questions can be used as early as in the first-year undergraduate. This will ensure the students will use and apply their knowledge so far and try to understand the scenario and hence choose the best single answer [8]. In this way, their practice will simulate the real-life of clinical practice when they graduate where they must apply their knowledge whenever they encounter a clinical event. MCQs are used in assessment of different Bloom’s Taxonomy levels; from factual recall to more complex levels such as evaluation and reasoning [9]. Moreover, because they facilitate a wider and more varied content, they are considered a suitable format of assessment especially for certificate and licensing assessments [9] [10]. On the other hand, they have a downside face as they are time-consuming, not easy to construct, and only well-trained staff are capable of writing them ( [5] [9] [11]). Since student learning is driven by tests, careful test construction is an important skill for educators to develop [12]. Staff needs continuous training courses to improve MCQS writing skills, the training must be regular and in repeated programs [13]. Assessment must be of high quality and valid, this requires the founding of strict procedures to check the test quality before and after conduction of the test.
Assessment is deemed to promote the educational skills of teachers as expressed by almost (75%) of directors of medical education centers, deans and academic chairs [14]. Guidelines and examination committees encourage excellent practices and promote educators skills and knowledge [15].
This a study to explore the awareness of the staff, involved in teaching of basic medical sciences, about their practice of scenario-based single best answer (SBA) MCQs and the difficulties they encounter. An electronic Likert-scale format questionnaire is used for data collection. The number of staff who responded and filled in the questionnaire is 101.
2. Methodology
This study is meant to explore the awareness of the university basic medical sciences’ staff about the scenario-based (SB) SBA (single best answer) MCQs as a tool of assessment; their knowledge and difficulties they encounter, if any, in writing this type of questions.
An electronic Likert-scale format questionnaire is used as a tool for data collection. The survey planet platform was used as a mean to facilitate data collection. The rationale for using the Likert Scale because it uses a universal method of collecting data and easy to understand and draw conclusions and results from the responses. The questionnaire consisted of twelve questions; background of staff member (medical/non-medical), previous experience dealing with scenario-based MCQs, courses or degrees attended in relation with MCQs and personal thought about shifting towards SB SBA MCQs.
The data was analyzed manually because of its small size.
3. Results
Two thirds of the research population are male university staff and the rest are females. More than half of the population researched are currently working in different universities in the kingdom of Saudi Arabia, a little more than one third are in the Sudan. The rest are sited in other different countries. Three quarters have a medical background, and the rest are non-medical.
Almost one third are anatomist, nearly one-fifth (20%) are pathologists, more than one-tenth (15%) are physiologists and few from each of the other basic medical sciences.
More than half of the population are at the Assistant professor level, one-tenth (10%) holding full professorship in their specialties, less than one-tenth (8%) are at the Associate professor level and the rest are at the lecturer and senior lecturer stages.
Two thirds of the staff researched have a wealth of experience in teaching for nine or more years. Less than one-fifth (18%) have 5 to 8 years and 15% less than 5 years.
More than two thirds of the population have a degree in medical education (or other related subjects). These ranges from a diploma to a PhD and less than one third do not have a degree in medical education or any other related subjects.
The majority (86%) are familiar with courses or workshops related to MCQs writing and assessment in general, a small minority have not attended any.
The majority (86%) had some experience in writing MCQs. Only, a small percentage has not tried writing this type of MCQs.
Overall, two-thirds think constructing this type of MCQs is easy and currently are doing it, another group (nearly 15%) also think it is easy but are not currently involved in writing them. Around 20% think it is difficult or not necessary.
Nearly 60% thinks it takes time to construct, one-fifth relates the difficulty to language barrier, one-third lack the knowledge to do and nearly one-fifth simply declares it as unnecessary.
The majority (96%) are in support of shifting to scenario-based MCQs in basic medical sciences
4. Discussion
Analysis of the results of the interview of the staff involved in teaching of the basic medical sciences revealed positive facts regarding construction and usage of scenario-based SBA (MCQs) in basic medical sciences and few negative points. The majority (86%) are familiar with and attended courses or workshops related to MCQs writing and assessment in general, a small minority have not attended any. Raza and H. Zainab (2019) found that faculty to faculty feedback can improve item writing considerably and regular training will perfect the faculty writing skills [16]. Rajaraman reported question writing should be upgraded by training of faculty on writing and a re-analysis is to be assessed after training [17]. The majority are used to this type of questions in contrast to a small minority (<15%) who lack previous experience. These finding are more or less similar to the findings of Abdulghani et al. (2015) where he pointed out that faculty needs a long duration of training courses to correct flaws in MCQS writing and the training must be continuous and in repeated fashions [13]. Regarding the difficulty issue of writing scenario-based questions, two thirds of the staff who responded to the interview think constructing scenario-based type of MCQs is easy and they are used to and nearly 20% are facing difficulties constructing this type of questions. Similar finding reported by Karthikeyan et al., who found that item writing could be affected by institutional and individual barriers, faculty development and quality assurance process. However, they failed to point-out specific challenges [18]. In the time factor, most of those interviewed pointed out that it is time consuming to construct good type of questions. Some have related this to lack of expertise or language barrier. These findings are nearly similar to the results of Karthikeyan et al., 2019 [19] who think allocation of reasonable time for item writing, beside bonding the experienced writers with new writers for advisership could enhance writer commitment. Others (Bligh and Brice, 2009) have indirectly highlighted the issue of time needed in the construction of the scenario-based questions as medical educators have many duties among them is writing high quality items [20]. Most of the participants (96%) are in support of shifting to scenario based MCQs in basic medical sciences, this in agreement with P. Lal, who found that shifting to case scenario is mandatory and it sound good in the era of reasoning in constructing exam question [21].
A renovation in the assessment methods is needed for assessing the learners understanding of the different anatomy subjects. This will ensure the learning of students will happen in a higher cognitive domain far away from the routine factual recall. It is done over a suitable theoretical basis of assessment which meets the expectation of all those involved in the educational process.
Eventually, the students will gain the necessary competencies in knowledge, skills and attitude to prepare them for practicing in the different medical fields. Consequently, this will positively influence the whole educational process [22]. The most commonly used tool of assessing learners in medical and health professions programs for anatomy are MCQs [23]. Case-scenario MCQs compared to stand-alone MCQs was found to offer prospects for integration of sub-specialties in assessment in line with PBL.
They are consistent and practical in assessing students’ cognitive skills. The critical and logical thinking is encouraged by different levels of item difficulties. The higher students’ scores in the CS-MCQ examination suggest improved understanding of the subject and/or well-written question.
Increasing the number of scenario questions will certainly mean wider course content is included in the examination. [24]
Clinical scenario questions present students with information on the clinical presentation, complications, laboratory and radiological investigations. The students have to interpret and manage specific clinical conditions which will necessitates the test items to be written in a high cognitive format [25].
5. Conclusions
The majority (86%) had some experience in writing MCQs and many of this majority are currently involved in writing MCQs.
Nearly 60% think it takes time to construct, one-fifth relate the difficulty to language barrier, one-third lack the knowledge to do and nearly one-fifth simply declare it as unnecessary. Most of those responded are in favor and support of shifting to scenario-based MCQs in basic medical sciences.
Author Contributions
Both authors contributed equally in all the steps of the research and preparation of the paper draft.
Limitation of Current Study
Small number of respondents (101).