Kazan State Medical University Survey after the Use of CyberPatientTM during COVID-19 ()
1. Introduction
Asynchronous delivery of knowledge in general has become acceptable by many educational organizations, including medical education, but particularly when its use pertains to the acquisition of practical knowledge (Messman et al., 2020). Recently, accredited courses have been made available for medical education through asynchronous delivery for undergraduate, post graduate and continuous professional development. However, the acquisition of practical knowledge and competencies for medical practice is still largely delivered within a hospital setting or through simulation laborites with the physical presence of students and physical interaction of students with patients, simulated patients (SP) or other physical simulators. Despite the introduction of the online virtual simulations by many investigators and meta-analyses performed (Valleé et al., 2020), it has been widely believed that for practical knowledge, students have to be in the hospital and have physical interaction with a real person or a sophisticated physical simulator.
The COVID-19 pandemic has changed the situation. One of the important conditions for control of COVID-19 is social distancing, which has created serious barriers for acquisition of practical knowledge in medical schools around the world. In the past four months many symposiums, conferences, workshops and town hall meetings have been conducted to find a solution for acquisition of practical knowledge during COVID-19 pandemic when students cannot attend classes and rotations in the hospital ( Ashokka et al., 2020 ; Ataniyazova, 2020; Choi et al., 2020; Almarzooq et al., 2020 ).
CyberPatientTM (CP) is an innovative online simulation solution for the acquisition of practical medical knowledge and experience in a virtual environment delivered asynchronously. This technology provides the opportunity for students to interact with simulated patients and practice their clinical competencies in a virtual, clinical environment. This innovative technology was applied in a group of 3rd - 5th year medical students in Kazan State Medical University (Russia)—equivalent to North American Year 1st, 2nd and 3rd (pre-clerkship)—for one semester as part of their curriculum.
It was hypothesized that students would find the CyberPatientTM platform suitable and complementary to the clinical environment in acquisition of practical knowledge and competency gain during the COVID-19 pandemic.
The main goal of this survey was to investigate the satisfaction of the students in relation to the usability, value and benefit of the CP platform after the platform was used for a semester (three months) during the COVID-19 pandemic.
1) The specific aims of this survey were to determine the following:
2) The utility of CyberPatientTM when it is an integral part of the curriculum or given to students as supplementary learning material.
3) If the use of CyberPatientTM distracts students from other online teaching materials.
4) The intuitiveness of CyberPatient’sTM user interface and content.
5) The perceived value of CyberPatientTM by students.
6) The perceived efficacy of CyberPatientTM by students.
7) The perceived impact of CyberPatientTM by students.
2. Materials and Methods
Since March 2020, at Kazan State Medical University (Tatarstan, Russia), students’ education has been transferred to distance learning. KSMU has two separate faculties of medicine. In one, the KMSU curriculum is delivered for native students in the native (Russian) language and in the second, the KSMU curriculum is carried out in English for foreign students. This survey was carried out as part of the faculty of medicine for foreign students with an English curriculum. Online education for these students was carried out on the KSMU’s own educational portal and for the English speaking students, the CyberPatientTM platform was provided as an additional distance learning tool. The use of CyberPatientTM technology, however, was mandatory only for 3rd, 4th and 5th year students who attended general surgery and internal medicine pre-clerkship and clerkship (equivalent to 1st, 2nd and 3rd years in North America). The rotations where CP was part of their curriculum will hereafter be called “Mandatary Group’’ or MG. For students in other clinical rotations which did not receive CP as part of their curriculum, CP was freely available to use as self-directed study. This group will be called thereafter “Voluntary Group” or VG.
Each student in the English language faculty had their own full account on the CyberPatient platform. In the beginning of the semester, an introductory lecture was given to the MG to familiarize students to the CP’s user interface, how classes with CP will be organized and how CP can be used for self-directed study and assessment. The main objective for the implementation of CP into the curriculum was to fill the gap that was created by the COVID-19 pandemic and the resulting inability of students to attend the clinical sessions in the hospitals.
Before starting the classes, the MG received a copy of the powerpoint lecture on CyberPatientTM, in addition to a live online lecture, to brush up on their skills in the use of CP platform. During the semester, instructors were given tasks (1 - 2 cases) on the topic of each lesson to students. For each discipline there were 10 lessons in the semester. At the end of the lesson, students sent their results to the instructors via the KSMU online system.
At the end of the semester, both the MG and the VG groups completed the online survey. The VG, students who did not use CP as a mandatory part of their curriculum, were asked not to fill the rest of the survey questions after they completed the general questions on the online delivery methods proposed by KSMU. The total number of students participating in this survey was 125 from which 59 students were in the MG groups and 66 students in the VG group.
The total number of students that used CP in the VG group were 32 of the 66. The slim majority of students in the voluntary group did not use CP Platform. Students who used CP on a voluntary basis (VG) did not participate in the second section of the survey.
The completion of this questionnaire was strictly anonymous. The questionnaire was presented to students as a Google Form. The main objective of the survey was to investigate the satisfaction of the students for usability, value and benefit of the CP platform, and the questionnaire was designed with this objective in mind. The survey questionnaire consisted of two sections. The first section (A) was devoted to general questions about remote learning. The specific aims for the first part of the questionnaire were twofold. One was to investigate if CP was used more as a freely accessible resource for students or if it was used more when integrated into their curriculum as a mandatory resource. The second aim was to see if the addition of the CP workload into the already heavy coursework of students negatively affected the use of other online resources. Questions related to the section A of the survey are presented in Table 1. After the completion of this section, and in order to identify students of the MG, the survey asked if they used CP as part of their curriculum assignment. Students who answered “yes” to this question continued to answer the rest of the survey. The survey ended for those students who answered “No” to the mandatory use of CP.
In the second section (B) questions were specific to the use of CP and consisted of three specific aims. The first specific aim was to determine the opinion of the students on the usability of CP platform. For this purpose, the following questions were asked of the students:
1) How difficult it was to navigate and operate CP?
2) Did you use the CP tutorial before starting the cases?
3) Did you use any other instruction materials to learn how to navigate/operate CP?
4) How many different cases of CP did you finish during this term?
5) What is the maximum number of times you repeated the same case?
6) How many times did you repeat a case?
7) Please identify your three favourite CP features.
8) Please rate your overall satisfaction with the number of cases on the platform.
9) Please rate your overall satisfaction with the quality of cases in the platform.
10) Please rate your overall satisfaction with the variety of cases in the platform
The second specific aim was to determine the opinion of students on the value of CP. For this purpose, the following statements were posed for the students to agree or disagree with:
1) CP is more engaging than a textbook.
2) CP is more stress-free than dealing with SPs or real patients.
3) With Cyberpatient I feel safe making hard clinical decisions.
4) CP gives me an opportunity to practice my skills safely.
5) CP lets me learn from my mistakes better than other learning.
6) With CP I can practice my clinical skills on a variety of patients whenever it is convenient for me.
7) After using CP, I am more confident in my abilities to manage a patient independently.
8) The range of difficulty levels allow for a step-by-step progression in my learning.
9) CP allows me to learn about the continuum of care from start to end.
10) CP supports experiential learning (learning by doing).
11) CP supports competency-based learning.
12) CP supports problem-based/ case-based learning.
The third specific aim was to determine the opinion of students on the efficacy and benefit of CP. For this purpose, the following statements were posed to the students:
1) Arrive at relevant provisional diagnoses.
2) Conduct a relevant physical exam.
3) Determine suitable treatment options.
4) Identify critical conditions and their associated red flags.
5) Manage and document inpatient care.
6) Order relevant tests to narrow your differential diagnosis.
7) Take a relevant history.
In this section we also asked students to express their opinion whether CP benefited their specific clinical skills on the following parameters:
1) History taking
2) Physical Examination
3) Laboratory investigation
4) Diagnostic conclusion
5) Dailey management (monitoring of vital…
6) Consultations
7) Recommendations
8) Documentation
9) Feedback
The last questions in this section were about the impact of CP on their medical education at the end of their semester and how comfortable they were personally managing a patient. The opinion of students in these two questions was measured on a scale of 1 - 5, 1 being no impact/not comfortable to 5 being highly impacted/very comfortable.
In addition, for MG additional questions were designed to gather some general information on students who used the CP platform in order to gather some initial information for data analysis. To reach this aim the following information was requested from the students:
1) What year of medical school are you in?
2) Rate their clinical skills?
3) What level of CP platform do they use?
4) Did you use CP as self-directed study or as a mandatory part of the curriculum?
In this questionnaire a variety of answers were measured from a simple “yes” and “no” to using a scale of 1 - 5 pending on the specific aim of the question. The measurement mechanism of answers are depicted in the results section.
The questionnaire results were automatically gathered through the Google Forms and were downloaded to an excel sheet for data analysis and interpretation. Statistical analysis for Table 1 included Wilcoxon-Mann-Whitney permutation (to address ties) test with mid-p correction was performed to compare Likert scale answers between two groups. Statistical analysis for Table 1 included Wilcoxon permutation sign test with mid-p correction was performed to test null hypothesis of symmetry (it means that there is no shift in post-answers relative to pre-answers).
3. Results
Results of this survey are described according to the organization of the survey questionnaire. It was also deemed appropriate and useful to discuss each section of the results as the results are presented instead of having a discussion section separately.
3.1. Section A: General Questions about Distance Learning
The specific aims included:
· to investigate the use of CP as a freely accessed resource in comparison to the use of CP when it’s integrated into the curriculum as a mandatory resource.
· if the addition of the CP workload into the already heavy workload of students negatively affected the use of other online resources
Results of this survey (Table 1) show that majority of students (89.5%) in MG group used the CP platform (37% all the time, 18.6% often, 33.9% sometimes) and only a small minority (3.4%) of student used it rarely and (6.8%) never used
![]()
Table 1. Presents the mandatory use of CP in comparison to the voluntary use. It also shows the use of CP in comparison to other remote education tools.
the CP platform. In contrast to VG group where 41% of students used CP platform and some (7.6%) really used it and 51.5% did not use CP platform at all. The difference between these two groups in the usage of CP platform is highly significant (P < 0.0001). This indicates that when an online program is an integral part of the curriculum it would be used significantly (p < 0001) more than the freely accessed resources that are not included in the curriculum as a mandatory assignment (Ruiz et al., 2006).
In the meantime, this survey also shows that there is no difference between students in MG and VG groups attending the other online courses, such as: classes via video conferencing tools, completing tasks and taking tests on the educational portal of Kazan SMU, independent work with textbooks/self-study, or watching video lectures on the educational portal of Kazan SMU. This indicates that the addition of CP into the curriculum of students as an additional resource did not significantly affect the use of other educational resources and it can serve as a complementary online platform (Adamczyk et al., 2009).
In this section it was also important to have some general information related to the students that used CP, what level of CP they used and know if CP was assigned to them.
In Figure 1 the majority of students (61%) identified themselves as intermediate and (32.2%) advanced levels and only a small percentage (6.8%) of students identified themselves as beginners.
The intermediate opinion somewhat corresponds to the 4th year of medical with the majority of students (55.9%). It was expected that the year of medical school should correlate to the opinion of students about their level of competency. However, data in this survey shows that the majority of students have a better opinion about themselves ( Miles & Leinster, 2007).
Data in Figure 2 shows that the majority of students used Level III in the CP platform. Although students may have tried other levels, the data somewhat
![]()
Figure 1. Show results for two questions. On the right, how would you rate your clinical skills? On the left, please identify your year of medical school.
relates to the level of competency identified by students and year of education. Level III corresponds to intermediate confidence and 4th year of medical school. While Level I and II corresponds to the 3rd year medical school and beginners, Level 4 corresponds to the 5th year and advanced students. This may suggest that CP platform can be used by all medical students and that students may find their appropriate level to begin with or start from level I and archive competencies in a stepwise progression during their years of medical school.
Results presented in Figure 3 indicate that the majority of students used CP when it was assigned to them by faculty (69.5%) and when it is used as a self-directed study (57.6%). Some students used CP in a group learning environment such as classroom (28.8%) and private group discussions (22.0%). About 18.6% of students believed their progress was monitored. 16.9% of students used CP to receive feedback and support from the faculty after their use of CP.
This data provides evidence that CP can be used in a variety of educational environments such as self-directed study, or as an assignment from the faculty, and in a classroom as monitored or not monitored by the faculty. It can also be used in a private group discussion and in a classroom discussion similar to the blended online curriculum studied by Lindeman et al. (2015).
3.2. Specific Aims to Define Usability
Usability is defined by the Cambridge Dictionary as, “the degree to which something is easy to use”. More broadly in the software field, usability can also be understood as the capacity of a system to provide a condition for its users to
![]()
Figure 2. Show the level of CP platform used by students.
![]()
Figure 3. Depicts the implementation of CP platform through a variety of methods.
perform tasks safely, effectively and efficiently while enjoying the experience. The basic measurements of usability include quality metrics such as success rate (whether users can perform the task at all), the time a task requires, and users’ subjective satisfaction. The specific aim in this section of survey is to gather information on quality metrics for the following parameters.
The first parameter includes the degree of difficulty students experienced with the use of the CP platform. For this purpose, students were asked the question “How difficult it was to navigate and operate Cyberpatient?” Students were asked to express the degree of difficulty on a scale of 1 to 5, 1 being not confident in operating CP and 5 being highly confident.
Results shown in Figure 4 revealed that about 89.8% users believed that it was not difficult to operate the system (with a variety of confidence scaling between 40.7%—very confident, 39.0%—confident, and 6.8% somewhat confident and 10.2% highly confident). In this survey only 3.4% of the students had the feeling that it was very difficult for them to use the system and they were not confident navigating the system ( Taglieriet al., 2017).
To understand this high confidence level for the use of the platform, it was deemed useful to assess if training students on the use of the system assisted in overcoming any difficulties in navigation. To answer this we sought to know if students used the CP tutorial or any other instruction materials to learn how to navigate/operate CP before starting the cases.
Although it is strongly recommended to use CP’s tutorial before starting the cases, nevertheless data in this survey showed that 66.1% of students start using CyberPatientTM without the tutorial (Figure 5). This certainly shows that CP’s
![]()
Figure 4. Shows the opinion of the student on how difficult it was to navigate the system.
![]()
Figure 5. Shows how many students used CP Tutorial before starting the cases.
UI//UX is very intuitive. Results of the survey showed that about 33.9% of students used the tutorial provided on the Internet for navigation of CP and 66.1% did not use the tutorial and 50.8% received the other type of instruction for navigation of the CP platform (Figure 6). This indicates that overall 84.7% students received some type of instruction to navigate and the remainder of 15.3% used CP without instruction, for whatever reason. It also indicates that the vast majority of students have expressed interest in learning about the platform before knowing about the efficacy of the platform.
It is well known that most software, online or otherwise, when it is very complex to operate needs training on the navigation system. In fact, it is proven that the more complex the operation of the interface is, the more training is required to operate the software (McGrenere et al., 2002). Therefore, one of the questions in this survey was to see how important it is to prepare students for this online resource. Despite the availability of the tutorial and materials provided by the university, the number of students who did not use the material but showed confidence in their use of CP indicates that CP is intuitive (Gormley et al., 2009).
One of the measurable indicators of student interest in CP was to measure the number and variety of cases used, how many times students repeated a particular case and if repetition of the case were useful for them.
It was important to realize how many cases they finished and how much they studied with CP. As described elsewhere (Nouri, 2016), data shown in Figure 7 reveals that the majority of average achievers (64.4%) have completed what expected from them (between 1 - 5 cases) and about 33.9% over achievers completed up to 20 cases and 5.1% of over achievers have used more than 20 cases from the CP platform. About 1.7% of students that are most likely to be under achievers did not use any cases in the CP platform. While Figure 8 indicates that students repeated the cases multiple times (94.9% of students repeated a case between one and five times). Figure 9 shows the majority of students have the opinion that repetition improves their skills, leading to the conclusion that this repetition was done out of self interest.
![]()
Figure 6. Shows how many students used other materials such as seminars, lectures provided by the University for students on how to use CP.
![]()
Figure 7. Demonstrates the number of cases used by students during semester.
![]()
Figure 8. Shows that the majority (94.9%) of students repeated a case between one and five times.
As shown in Figure 9, the majority of students (83.4%) have the opinion that repetition helps them to improve their knowledge and among them 19.4% believe that repetition helped them a lot in acquiring knowledge and experience. However, 16.7% of students believe that repetition does not help them to improve their knowledge and experience (Barrett et al., 2004).
Among the users’ subjective satisfaction, identification of three favourite features in the CP platform was asked deliberately in order to distinguish benefits from favouritism. Students may have chosen, through superficial judgement, the section of CP with nice animations and easy-to-use interface over the benefits that they would get from more difficult sections. Therefore, the judgment about the value of each CP feature was designed separately and is presented in the next section.
This graph, Figure 10, shows that the three most favorite features of CP identified by students include History Taking 79.7%, Diagnostic Conclusion 67.8%, and Physical Examination 57.6%.
This phenomenon may relate to the experience and knowledge of students as well as the design of CP interface (Cho et al., 2009).
![]()
Figure 9. This figure demonstrates the opinion of students on the value of repetition in learning on the platform.
![]()
Figure 10.This Figureranks the favorite CP features as identified by students.
Perhaps the most important data on the satisfaction of students in this section was to identify students’ opinion and their satisfaction on the number of cases, quality of the cases and variety of the cases CP platform provides for students to learn.
Figure 11 shows that 93.3% of students in this survey are satisfied with the number, quality and variety of the cases in CP platform. However, there are students (6.7%) who have not been satisfied or are doubtful. On the other extreme side of this equation 8.5% of students were highly satisfied by the number of cases, with 15.3% highly satisfied with the quality of the cases and 10.2% extremely satisfied with the variety of the cases (Farahmand et al., 2020).
3.3. Specific Aims to Define Value Included
Value is defined as the quality of something being important or beneficial or useful, or as defined by Webster Dictionary: “is the rate or scale in usefulness, importance, or general worth”. In this survey the value of the CP platform was assessed by the 12 questions presented in Figure 12. These questions mostly
![]()
Figure 11. Presents the opinion of students on the number of cases, quality of cases and variety of cases stratified from 1 to 5 with one being not satisfied to five being highly satisfied.
![]()
Figure 12. Presents the opinion of students on learning or dealing with specific components of clinical competencies. Students were asked to express their opinion on the presence (yes) and absence (no) of the described value.
reflect the opinion of students on obtaining some aspects of clinical competency in the virtual environment of the CP platform.
This assessment revealed that the vast majority (86% - 91%) of students identified the value of CP in practically all the questions. The lowest score in this value assessment was the question of whether CyberPatientTM was more engaging than the textbook. Where 69.5% of students were in favor of CyberPatientTM and 30.5% of students felt that textbooks are as engaging as CyberPatientTM.
This may relate to the style of learning issues expressed by many Investigators. Research data has shown that 20% - 30% of students are extremely talented. They will be successful with any style of learning. However, about 70% will be able to succeed better when audio, visual, tactile memory is involved in the process of learning (Sobral, 1995).
3.4. Specific Aims to Define Efficacy and Benefit
One of the important issues in the assessment of an electronic platform is the capacity of a platform to determine if it is beneficial, effective, and has an impact on the user’s journey. Therefore, our most important aim in this survey was to define the opinion of students on the benefit, effectiveness and impact of CP platform. For this purpose, first we asked students to express their opinion and rank the nine questions related to clinical competencies by the extent of benefits it can provide for them.
Figure 13 describes the opinion of students and how CyberPatientTM benefited their specific clinical skills. As it is evident, the skills such as history taking, physical examination, laboratory investigations, and diagnostic criteria are rated the highest. Skills such as daily management, consultations and recommendations are rated second highest. Skills such as documentation and feedback are rated the lowest. This may be due to the level of students’ competency that relates to the year of medical school. Since CyberPatientTM was given to the years 3, 4, and 5 (which is equivalent to years 1 and 2 pre-clerkship level in North America), this cohort of students may not have the knowledge and experience for clinical skills such as daily management, consultation, recommendation,
![]()
Figure 13. Provides data on ranking of the benefit of the specific section of CP as related to competencies.
documentation and others. Their theoretical knowledge and background experience is on the level of history taking, physical examination, lab tests and diagnosis (Link & Marz, 2006).
The second important issue in this section of the survey was to define the opinion of students on the effectiveness of CP platform. For this purpose, we asked students to express their opinion on seven questions related to clinical competencies before and after the use of CP platform and to rate their practical knowledge (competencies) in the scale of 1 to 5 with one being not confident to five being highly confident (Table 2).
Comparing the pre and post values, students felt that after the use of CyberPatientTM, their level of confidence increased in history taking. The number of confident and very confident students increased from 40% to 48% and 15.3% - 20.3% subsequently, while the number of not confident or slightly confident students decreased or was unchanged.
The same pattern has applied to other sections of CP such as physical examination, identifying critical conditions, and others. The opinion of students depicted in Table 2 clearly demonstrates that the CP was highly effective (P < 0.001) in all aspects of clinical competencies expressed in this survey. They believe that they learned more and become more competent in those aspects of clinical encounters that they usually learn in the clinical setting, which due to COVID-19 they were not able to attend classes usually held in clinical environments. Since we stratified the students’ answers on a scale of 1 to 5, one being not effective and five being highly effective, the majority of students were in the scales of 2 somewhat better, 3 better, and 4 much better. Obviously, some of the
![]()
Table 2. Opinion of students before and after the use of CP platform in acquiring practical knowledge and clinical competencies.
students were in the extreme corners of this scale. Students in the lowest corner of the scale who did not believe that they improved their practical knowledge with CP platform ranged from 1 to 3 students, pending on the specific scales, and students who believed that CP platform was highly effective in improving their practical knowledge and competencies ranged from 6 to 12 students as described in Table 2. This indicates that about 3 in a group of 59 students had difficulty understanding, operating or recognizing the value of the CyberPatientTM platform in relation to acquiring practical knowledge (Bösner et al., 2015).
It was also interesting to know how the opinion of students shifted on this scale from negative to positive and vice versa. Student’s opinion shift is presented in Table 3. It demonstrates that there was a positive shift in the opinion of students and that shift was significant in certain competencies such as history taking (P < 0.0337) and identifying critical conditions and their associated red flags. In all other areas, such as the relevant provisional diagnoses, conducting a relevant physical exam, determining suitable treatment options, managing and documenting inpatient care, the shift was borderline (P<0.006 to 0.09). The only area where the shift was not significant when it pertained to ordering relevant tests and differential diagnosis (Ruiz et al., 2006).
In relation to the impacts we asked two questions from students. One was related to their opinion about their level of comfort personally managing a patient after having a virtual experience in CP and the second was a direct question on the impact of CP on their clinical knowledge (Figure 14).
The opinion of the student on the positive impact of CP on their education was very firm with 89.9% of students considering the CP to have a measurable impact on their knowledge and experience and about 6.8% expressed that it was somewhat impactful on their education (Figure 15). However, two students (2/59 - 3.4%) did not share the opinion of the vast majority and had the opinion that CP did not have any impact on their education.
In this section satisfaction of students toward the quality, number, and variety
![]()
Table 3. Shift of students’ opinion toward better or worse on the improvement of practical knowledge and clinical competencies after using the CP platform.
![]()
Figure 14. Demonstrates the data on the level of comfort students will have on managing the patients. A scale of 1 to 5, 1 being not confident at all, and 5 being very confident, was used to measure the level of comfort pre and post use of CP platform.
![]()
Figure 15. Reflects the opinion of the students on the impact of the platform on their clinical performance. A scale of 1 to 5, 1 being no impact, and 5 being high impact, was used to measure the level of impact using the CP platform.
of the case were also assessed. As depicted in Figure 11 the vast majority students were very satisfied with the quality, number, and variety cases presented by CP. In this cohort of students only 2 students (3.4%) expressed their dissatisfaction with quality, number, and variety of the cases.
4. Discussion
Results of this survey can be summarized with the following findings:
1) CyberPatientTM is used significantly more when it is integrated into the school’s curriculum (P < 0.0001) and CyberPatientTM does not distract the attention of students away from other online materials.
2) 96% of students believe that the CP platform was intuitive and the vast majority of students (93.3%) were satisfied with the number, quality and variety of cases presented in the CP platform.
3) Over 90% of students considered CyberPatientTM a valuable teaching tool.
4) Students perceived CyberPatientTM to be highly effective (P < 0.001).
5) 89.9% of students perceived CyberPatientTM platform to have a positive and measurable impact on their educational experience.
E-learning gained popularity in medical education long before COVID-19. However, this popularity was more for theoretical and basic science courses than for clinical environments where practical medical skills are required (Ruiz et al., 2006; Moberg & Whitcomb, 1999). Similar surveys and meta analysis have been completed by others in relation to this issue (Vallée et al., 2020; Kononowicz et al., 2019). Computer literacy and attitudes towards e-learning amongst first year students was studied by Link & Marz (2006). It was concluded that many students would benefit from basic introduction to computers and computer-based resources. This research revealed that 12% of students make little or no use of existing e-learning offerings. Results of this survey confirmed our findings (Table 1) and further alluded that if the course is mandatory, 3.4% of students will not use the software but if it is voluntary about 51.5% of students will refrain from using it. This difference is significant (P < 0.0001). This means that to maximize use by students an e-learning platform—including CP—should be integrated into the curriculum of medical schools. This study supports the perception that if students do not feel that they will be examined on a book they will rarely read the book just for their interest or personal development. There is also a perception that the addition of an online virtual platform will increase the workload of medical students and may reduce their time for using other e-learning tools provided by the university. Results of this study (Table 1) show that the addition of CP in the curriculum did not significantly change the attention of students from other online resources.
The usability score in this study may relate to many features such as the students’ familiarity with the use of computers, e-learning processes, and the platform/software interface. We believe these factors are the most common. It’s assumed that every young person at this time is already familiar with computers. Many universities are already switching into e-education and toward the use of learning management systems (LMS). The only familiarity they need is with the platform functionality and interface. Computer literacy and attitudes towards e-learning amongst first year students have been studied by others (Link & Marz, 2006). Results show that about 10% of students have never used computers as an online learning tool and about 4.4% do not regard any e-learning methods as helpful. Considering that this study was conducted in 2006, results of our survey are similar to this finding. In our study, 3.4% believe that it is difficult to use the CP platform and it did not benefit them. However, the vast majority of students (96.6%) believe otherwise (Figure 4).
The high score of CP on the interactivity and usability of CP’s interface (Figures 4-11) is due to two important factors. First, the CP interface is designed by a validated method as described by Vincent Cho et al. (Cho et al., 2009). According to their theory, user interface design (UID) is mediated by two variables, namely, perceived functionality (PF) and perceived system support (PSS) which influence perceived usefulness (PU) and perceived end of use (PEOU), respectively. The CP interface is built on these principals. The second reason for our success in UID is that we tested the UID of CP with medical students in three different places with three distinct groups of medical students. Feedback from these tests were applied to UID immediately.
Repetition—also called drilling technique—has always been used in educational circles (Figure 11). We believe that repetition is a powerful method to learn and retain memory. However, in synchronous delivery repetition is not always available, and when repetition is available it makes the learning process expensive, particularly when it pertains to the use of patients or standardized patients (SPs). Online solutions, on the other hand, provide better opportunities for students to drill their practical competencies with no additional cost, ethical or legal issues. Traditionally, students cannot see the same patient several times and the faculty may not be able to find similar patients for students to repeatedly practice their clinical skills. E-learning is able to provide the environment for students to repeat the same case over and over until the student is satisfied with their performance. The effect of repetition on acquiring clinical skills was studied by Barrett et al. (2004) and they provide evidence that the repetition of basic cardiac murmur identification significantly (P < 0.001) improves efficiency in recognizing basic cardiac murmur sounds by medical students.
Results of this study express the perceived value of CyberPatientTM identified by students. CP’s value and value proposition, as well as cost effectiveness, in comparison to SPs has been studied by Frahmand et al (2020). Results of this study reveal that CyberPatientTM is sixfold more cost effective and has a better cost value proposition in comparison to SPs. It also demonstrates that if we train SPs for the whole clinical part of the medical school curriculum (about 120 cases) and repeat the use of SPs for all levels of medical school many times, the cost will be astronomical and unaffordable for most medical schools. Therefore, virtual clinical simulations such as CP will be the low hanging fruit that is effective, economical and reliable.
The perceived efficacy and benefits of CP (Figure 13, Table 2, and Table 3) is also supported by two separate perspective randomized controlled research projects (Kononowicz et al., 2019; Qayumi et al., 2004). In the first study, CP is compared to textbook learning in relation to the acquisition of clinical skills (specifically, the physical examination of the abdomen). Results of this controlled, randomized study performed at two Universities in Japan revealed that CP is significantly more effective than traditional methods such as a textbook. Furthermore, the study shows that CP efficacy benefits lower achiever students at a higher degree than higher achievers (Qayumi et al., 2004). A second study by Frahmand et al. (2020) confirmed the efficacy of CP in a prospective randomized controlled trial with standardized patients. Results of this study demonstrated a significant improvement comparing pre and post values for both methods. However, the degree of significance and the satisfaction of students favoured CP. The authors concluded that CP is as good as SPs for the acquisition of clinical skills such as history taking. The efficacy of online methods of delivery on medical education have also been proven by others (Lindeman et al., 2015; Douglass et al., 2013; Battaglia et al., 2012).
Several studies on the impact of virtual simulation in terms of flow, relevance, interest, effectiveness and support of learners have been published previously (Taglieri et al., 2017; McCoy et al., 2016; Benedict et al., 2013; Botezatu et al., 2010). The publications support the results of our study (Figure 14 and Figure 15) for the use of virtual patients and support the integration of virtual clinical environments into the curriculum of medical schools regardless of the effect of COVID-19 in medical education. Perhaps the presence of COVID-19 and associated social distancing measures, to prevent patients, students and faculty from this dreadful disease, provides a condition which enforces a cultural change in medical schools and guides the traditional/conservative individuals and institutions to become lateral thinkers and accept the growing evidence—which has existed long before COVID-19—of the benefits of virtual learning and lead these individuals and institutions to provide a safe environment to accommodate the continuation of medical education during and post the COVID-19 period.
5. Conclusion
In conclusion it is important to highlight the following statements related to the objectives of the survey:
General:
1) CP is used significantly (P < 0.0001) more when it is an integral part of the curriculum.
2) The use of CP did not interfere with other duties of the students that were delivered through online education.
Usability:
1) 96.6% of students believed that it was not difficult to operate the system; however it is important and useful to have some type of training on the use of the CP platform.
2) Students appreciate the variety of the cases.
3) A majority of students (83.4%) find that it is useful to repeat the cases over and over.
4) The three favorite features of CP identified by this cohort of students include History Taking 79.7%, Diagnostic Conclusion 67.8%, and Physical Examination 57.6%.
5) The vast majority of students (93.3%) in this survey are satisfied with number, quality and variety of the cases in CP platform.
Value:
1) The vast majority (over 90%) of students identified CP as valuable (value questions listed in Figure 12).
2) However, about 69.5% of students were in favor of CP and 30.5% of students felt that textbooks are as engaging as CP.
Efficacy:
1) Comparing the opinion of students before and after the use of CP platform it can be concluded that CP platform was highly effective (P < 0.001) in all aspects of clinical competencies expressed in Table 2.
2) It can be concluded that in certain areas of clinical competencies such as history taking (P < 0.03), recognition of the critical condition (P < 0.0009) and management (P < 0.006), a significant number of students shifted from low level of confidence to higher level of confidence after using CP platform.
3) The CP virtual platform’s overall impact was measured and 89.9% of students consider the CP platform to have a measurable high impactful on their knowledge and experience.
6. Recommendations
The COVID-19 Pandemic and post COVID structural changes impose certain conditions on the society as a whole and on medical education in particular. Results of this survey demonstrate the use of virtual platforms is viable and effective for practical clinical knowledge gain. Therefore, it is recommended for medical educational and regulatory bodies to realize cultural shifts in medical education and use technology to the max for support of their students. Medical technology platforms are not simple to understand and operate. Therefore, it is very important for technology providers to have sophisticated and continuous support for students and faculty alike (such as tutorials and webinars, etc). On the other hand, medical education organizations have to adapt to the new environment by providing special opportunities for students to learn the use of technology and for the faculty to understand how the technology works and how they can take advantage of the technology to support and improve their educational mandate. The minimum recommendation from this survey is that virtual platforms begin to be integrated into the modern medical school curriculum and to make sure that the bar is sufficiently high for students to repeat cases as many times as they need in order to reach that bar.
The results of this study provide more evidence, in addition to the evidence provided by others, for the effective use of virtual environments in the training of clinical skills of medical students and other health professionals. Although the need for the use of digital solutions for the training of clinical skills was evident before the COVID-19 pandemic, it is more evident now than ever that these solutions will become a necessity during the continuing COVID pandemic and in the post-pandemic period. It is also obvious that the use of virtual environments in clinical training will require further research and additional surveys like this one for further clarification of the best practices and execution of clinical training in virtual spaces.