Eye Movement Modeling Examples as a Teaching Tool for Epidural Block ()
1. Introduction
Eye tracking is the procedure of determining an individual’s eye movements, using a device called an eye-tracker, to reveal his attentional behavior while performing a task. By recording the direction of a user’s gaze and relationship to an area of interest (AOI), eye-tracking technology provides an objective tool for measuring visual patterns [1] .
Current literature demonstrates the ability of eye tracking to provide reliable quantitative data as an objective assessment tool, with potential applications to medical and surgical training to improve performance [1] .
Eye metrics can consistently distinguish non-experts from experts, and many studies, in different medical fields, have reported that as proficiency in a task improves, there is an increase in focused attention and visual representation during its performance [2] [3] [4] [5] [6] .
Recently eye tracking has also been used for the development of new methods to improve learning processes. The reason for this is that we take most information in through our eyes when we learn, just as when we perform a professional task [7] .
Based on practical experience, we know that simply telling beginners to “look the way experts do” does work, even if it does not necessarily improve their performance [8] . Indeed, one very powerful way of learning practical tasks is imitation. Consequently, research on teaching and training has picked up this approach. Indeed, decades of research have shown that studying examples of a model successfully executing a task is more efficient for learning than learning by trial-and-error [9] .
It is possible nowadays to create videos of task demonstrations not only by recording an expert’s problem-solving behavior and verbal explanations, but also by recording and visualizing their eye movements overlaid on the information the expert was looking at [10] .
This can guide the learners’ attention to the information the expert is referring to at the right moment, so fostering their understanding. Video examples with task demonstrations by experts, with the expert’s eye movements superimposed on the task, are known as “eye movement modeling examples” (EMME) [11] .
We performed this study to evaluate the changes in the performance of anesthesia novice trainees when executing the epidural technique after observing an eye movement modelling example (EMME) of epidural block procedure.
2. Method
The study protocol was registered at Clinical.Trial.Gov (ID n. NCT04749186) and approved by the Ethics Committee of Fondazione Policlinico Gemelli IRCCS, Rome (approval n 0017199/21). The study protocol was performed in accordance with the Declaration of Helsinki.
Forty two PGY3 anesthesia trainees from the Catholic University Medical School-Fondazione Policlinico Gemelli IRCCS accepted to be enrolled in this prospective, blinded, observational study. The study was carried out in the period between January 2021 and April 2022. Each participant gave written informed consent, and privacy, confidentiality and anonymity were fully guaranteed.
Only trainees who had never previously performed an epidural block and were about to begin their obstetrics rotation were enrolled in this study.
Before starting the study, we developed an eye movement modeling example (EMME) from recordings made from 7 experienced anesthesiologists with more than 20 years of experience performing lumbar epidural blocks in obstetrics.
For this purpose, we used a commercially available Tobii Pro Glasses 50 Hz wearable wireless eye tracker. This system can measure eye movements using cameras integrated into the eyeglasses which record the corneal reflection of infrared lighting to track pupil position, mapping the subject’s focus of attention on video recordings of the subject’s field of vision (gaze). In addition to tracking gaze, this system enables the measurement of various eye metrics including fixation frequency and dwell time.
The expert’s eye tracking video-recordings were stored and analyzed by using a Tobii Pro Lab Software. We selected five areas of interest (AOI), to define regions of a displayed stimulus, and to extract metrics specifically for those regions. The areas were the following: 1) Point of the epidural needle at its insertion into the skin; 2) Shaft of the epidural needle; 3) Hub of the needle; 4) Barrel of the syringe; 5) Plunger of the syringe; 6) Other fields of view.
We also divided the epidural procedure into three phases: 1) Epidural needle insertion and advancement; 2) Loss of resistance procedure (LOR); 3) Insertion of the epidural catheter. More detailed information on this technique is reported and published elsewhere [12] .
Mean values of experts’ eye tracking recordings were summarized as gaze plots and embedded in the EMME video, to show the eye tracking of the focus points where the expert looks while performing the epidural block. The various stages of performing the block were depicted and at the same time a voiceover indicated and explained which points to focus on while performing the block. The final product was a video recording of the epidural technique with a superimposed visualization of the recorded eye movements accompanied by the voice over explaining the technique and indicating where to visually focus the attention.
Immediately before the beginning of their 6 month clinical practice rotation in the labor and delivery suites, the trainees were randomized to receive (study group, n = 21) or not receive (control group, n = 21) the EMME video during their institutional training, consisting of a front-lecture on epidural anatomy and technique followed by four hours practice on an epidural simulator.
Trainees of the study group were instructed to watch the EMME video, focusing their attention precisely and carefully on the points that depicted where the expert looks, while tuning in to listen to the voice-over that gave instructions for the execution of the epidural block. This is in the attempt to establish a state of so-called “joint attention” [13] [14] .
During their clinical practice in the labor and delivery ward, all the trainees were evaluated every 10 epidural blocks until the end of the rotation period, by an independent observer, blinded as to which group the trainee belonged, by using the Global Rating Scale for Epidural Anesthesia (GRS) [15] , a validated scale to evaluate proficiency and progress in the technical skill of trainees in epidural anesthesia (Appendix 1).
3. Statistical Analysis
Statistical software R version 4.0.2 was used for data processing and the generation of figures. Percentage frequency distributions were presented as descriptive statistics for the assessment of the questionnaire results. To compare the assessment between the groups and/or the number of attempts, composite indicators [16] [17] were calculated as an instrument used to evaluate the degree of agreement expressed by the Likert scale assessment (Appendix 2).
A composite indicator value equal to 0 corresponded to the maximum agreement towards the worst judgment, an indicator equal to 1 corresponded to a maximum agreement towards the best judgment. (Disagreement among evaluators produced an indicator equal to 0.5).
The nonparametric Mann Withney-Wilcoxon test and the Kruskal-Wallis test were applied to compare the assessment questionnaire results between groups and/or the number of attempts. A power analysis was carried out on the primary endpoint of the study which provided a sample size of 42 to ensure a level of significance of 95% and a test power of 80%.
4. Results
All the trainees involved in the study completed their period of training and had their proficiency evaluated by the GRS every ten epidural blocks.
In Figure 1, the composite indicator values for each item of the GRS are reported.
We noted a trend towards an improvement in the composite indicators in the EMME group especially during the initial phase of training (first 10 blocks).
Trainees who received the EMME training exhibited more respect for the patient’s tissues (P < 0.05), better interaction with the materials needed to perform the block (P < 0.05) and demonstrated a smoother epidural technique (P < 0.05).
5. Discussion
In this study we investigated the effects of the experts’ eye movements tracked and replayed, sustained by the verbal description explaining where to look and where to focus the attention while performing the epidural technique in its various phases. According to our results, trainees who received the EMME training exhibited more respect for the patient’s tissues, better interaction with the materials needed to perform the epidural block and demonstrated a smoother technique, compared to those without the EMME training.
EMME is an adequately tested and validated technique, as confirmed by a recent meta-analysis [18] reporting that it was beneficial to learners’ performance, encouraging for future research as well as practical application regarding videos displaying a model’s visualized eye gaze behavior.
![]()
Figure 1. Composite indicator values for each item of the GRS. *P < 0.05.
In the medical field, however, it has been applied, until now, only in radiology even if with promising results.
One paper [19] investigated the diagnostic performance by medical experts and novices in interpreting PET/CT visualizations before and after exposure to eye movement modeling examples. The findings of this study indicate that studying eye movement modeling examples had positive effects on performance, task-relevant fixations, and the use of cognitive and metacognitive comprehension strategies.
Similarly, in another study [20] , the ability of a novice to identify pulmonary lesions consistently improved when the expert’s search behavior was previously shown.
According to Haider et al. [21] changes in the speed and quality of performance are partially due to a reduction in the amount of task information that is processed. This information reduction hypothesis holds that people learn, with practice, to become selective in their use of information, that is, to distinguish between task-relevant and task-redundant information, and limit their processing to task-relevant information. Improvements in task performance, in this view, reflect an increased knowledge about which information has to, and which information does not have to, be processed.
This information-reduction hypothesis [22] emphasizes the selective perceptual processing superiority of experts when compared with novices.
Based on the results in their meta-analysis, Xie et al. [18] hypothesized that expert models’ perceptual processing superiority, at least in part, can be transferred to other learners through visualization of the model’s eye movements.
The results of our study are in line with these theories, which have been confirmed by other studies [7] [11] [18] [19] [23] [24] [25] .
However, by analyzing our results, we noticed significant differences between the two groups only in some of the items of the GRS. Although we performed the study on a statistically sufficient sample of subjects, it cannot be ruled out that video training with EMME may influence only some of the procedural aspects of the technique or some aspects more than others. We could also hypothesize that the effects of the EMME video are diluted over time so that our indicators, assessing overall score trends over the six months of training, failed to detect differences between some items of the scale because these differences might only be detectable early in the training period. Interestingly, we noted a trend towards an improvement in the composite indicators in the EMME group in the first ten epidural blocks. We administered the GRS every ten epidural blocks, and we can speculate that if we had administered the GRS after each individual block, we would most likely have observed significant changes for all the items of the scale.
To the best of our knowledge, our study is the first one that has used the EMME for a practical, clinical teaching purpose on real patients and certainly we are the first to use it as an aid in teaching epidural anesthesia.
For this reason, we are aware that our results must be considered essentially preliminary and equally the technique that determined them experimental and innovative. We are also aware and recognize that ours is a single center study and therefore we encourage the diffusion of the methodology in other Institutions to investigate its potential benefits on teaching and learning epidural block.
Nevertheless, we believe that such an innovative technique deserves attention because it may complement and may be added to the traditional teaching methods. At the present time EMME has been included in our Institutional teaching programme for obstetric anesthesia trainees and its supplementary evaluation is in progress. Further studies are required to investigate the impact and the role of EMME on clinical teaching. In the field of epidural teaching, we hope that the preliminary results of our research will stimulate new studies to assess whether using EMME can increase not only the proficiency but also of the epidural block quality of inexperienced trainees.
6. Conclusion
In conclusion, we have demonstrated that inexperienced trainees who received the EMME training improved their proficiency at administering epidural blocks, showing more respect for the patient’s tissues, better interaction with the materials needed to perform the block and exhibited a smoother performance of the epidural technique.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Ethical Approval
The study was registered at Clinical.Trial.Gov (ID n. NCT04749186) and was approved by the institutional ethics committee (approval n 0017199/21).
Consent
Every study participant signed a consent form.
Authors’ Contribution
EC, GC, MS and GLG were responsible for the conception and design of the study, writing the manuscript, and revising the manuscript. MS, GLG, PPG, and MC were responsible for clinical case handling and collecting the data. MV was responsible for the analysis of the data and revising the manuscript. GD revised the manuscript. All authors have read and approved the manuscript.
Appendix 1
The Global rating Scale (GRS).
Adapted after [15]
Appendix 2: The Composite Indicator Calculation
To compare the frequency distribution of the questionnaire’s items among groups and/or number of attempts, we calculated an assessment agreement composite indicator (CI) considering a transformation of the original data, which respected their ordinal nature and the variability of the distributions. The indicator was based on the distance between an observed distribution and theoretical distribution under optimal conditions, i.e. maximum concordance for the best judgement:
m = number of modes of the variable; FAi = observed distribution; Fm = theoretical distribution.
This indicator is sensitive to the values assumed by the dispersion of the different distributions and the average level:
The indicator standardizes in a single measure the average and the variability of the distribution, giving a result value: 0 ≤ CI ≤ 1, where 0 indicates maximum concordance towards the worst judgement and 1 indicates the maximum concordance towards the best judgement.