Starting to Think Like an Expert: An Analysis of Resident Cognitive Processes During Simulation-Based Resuscitation Examinations
Summary
This study sought to identify the cognitive processes of emergency medicine residents in a crisis-based simulation. The authors ultimately distinguished high-performers from low-performers by four key themes. High-performers demonstrated the following skills:
- anticipatory behaviors: ability to see multiple steps ahead of their current task
- selective attention: ability to discriminate between task-relevant and task-redundant information
- Management of their cognitive load when new issues arose: ability to be flexible with algorithms and incorporating new information
- Utilization of a concurrent rather than a linear approach: ability to fluidly move between therapeutic and diagnostic components of patient management
This study emphasized the importance of high-fidelity simulation to gain more information on learners’ performance, while incorporating simultaneous technologies such as eye-tracking devices to enhance recall during the encounter debrief. By gaining insight into the behaviors differentiating high and low-performers, educators may investigate the cognitive skills necessary for residents to cultivate in order to move from low-to-high performance in crisis management.
Interview
RW: This study highlights key differences between high and low performers during emergent resuscitation efforts. How do you think educators can cultivate these skills during residency training, particularly in those at risk of underperforming?
AS: I think it’s tough. The first thing is identification of low-performers. For example, if you have two rooms with identical resuscitations with a big team, ie. general surgery, trauma, etc. – one led by a low-performing resident and one led by senior faculty – where everything goes well. You would say both leaders did well, but this doesn’t tell the whole story. The senior leader will have had contingency plans, “If I can’t get intubate or this equipment doesn’t work, I’ve got back-up plans B and C.” All of these thoughts are going on under the surface. The low-performing resident wouldn’t necessarily have the same cognitive processes. What you really need to do is get under the hood, being aware that these cognitive processes may be different. How do we do that? The first step is to ask the low-performing learner what they were thinking along each step. Educators who are tuned into these possible differences between high and low-performers might choose to delve into it with a debrief after the scenario is completed. I try to do this in my own teaching and what I’ve found works for me is to ask about hypotheticals – “If this situation had been different, what would you have done?”
The eye-tracking glasses were important in our study because of the notion of the expert blindspot. Experts are bad at being able to explain why certain decisions were made and may not perceive that decisions were made at all – this is all system 1 processing. Framed interviewing with the aid of the eye-tracking glasses can start to probe these thought processes and explore what learners might have done differently.
RW: For experts in resuscitation efforts, anticipatory skills are necessary. At which point in training do you suspect that these skills develop or what have you found in talking to experts?
AS: I often get questions like this. Can we accelerate the process? Like many answers in medicine, it depends. This process depends on the individual and their maturity. We haven’t specifically looked at answering this question but we looked at high versus low-performers, not senior versus junior residents. Senior residents have developed some amount of expertise, but others take longer to develop their cognitive processes. It was around the time that I was a PGY-4 resident, I was studying cognitive work, where I could see my own thought processes in real-time as I was learning and take note of how they were changing. Simulation has played a huge role in this learning and was a big part of my development. There is a lot to be learned from these experiences, but learners do need desirable difficulty to learn more. A key component is making sure that the cases become more difficult so that learners are really challenged and for an expert audience, we can uncover hidden thought processes.
RW: Did learners have insight into their own level of performance? To follow this up, I am curious if these experiences impacted how learners approach their next simulation sessions?
AS: Some people do and others don’t, but this isn’t something we specifically measured. Insight isn’t correlated with performance. Attending doctors generally agree the most dangerous person is the overconfident resident and would rather work with unconsciously competent resident. Society is made up of everyone though and attendings must be able to recognize who is entrustable while making practical changes.
While the study didn’t look at whether residents approached their next simulation sessions any differently, I would like to think they were more metacognitively aware. Learners did describe the eye-tracking training and simulation as being very useful, but we would need a longitudinal study. This would be challenging from a resource perspective, not only because of the expense of equipment, but from educator time. A lot of this is done on the good will of educators, but with healthcare resources being increasingly stretched, there is less time to commit in a simulation lab.
RW: How has this study changed your utilization of simulation-based learning in your educational practice? And how has this study impacted how you teach those four skills demonstrated by the high-performers?
AS: I’ve always been involved in simulation-based learning, so this hasn’t changed my practice a lot. In fact, another study has grown out of this study. We took this technology to expand the scope to try and improve debriefing in a multi-site study. It’s now under peer-review, but we’ve certainly been able to explore new ideas.
I do use the terms differentiating low and high-performers in the clinical setting. I hear residents talking about their cognitive load and I’d like to think I’ve spread my teaching. Whether they’re listening or learning, these concepts are being discussed. And I talk about it routinely, especially for senior learners or fellows in the resuscitations. These skills really differentiate good from great.
I enjoy reading about metacognition and learners are interested. The reality is, when you’re all done training, you will be responsible for these things. Of course, there is always learning on the job, but I think we should be striving for something better.
RW: What are the next steps for you?
AS: Like above, we have a paper under review partnered with another university in Canada, using eye-tracking glasses focused specifically on the debrief. We use the video as a tool to play back during debriefing to see if watching one’s own performance allows for deeper insights and better debriefs. Another item worth mentioning is that, is that we published another paper in 2021 looking at the application of cognitive load theory and how it relates specifically to clinical performance. It’s important to understand these concepts for self-growth and trying to understand learners; making sure we can see from learners’ perspectives. It’s easy to forget what it’s like to be a learner. Being able to reflect helps educators to stay humble and recognize that people aren’t always going to see what you think is obvious. This takes training, understanding, and recognizing where you are and where your learner is. If we train educators on these topics, then I think there’s potential to improve education. We don’t want to stay at the status quo.
Blog Post Author
Rachel Wojcik, MD is a third-year pulmonary and critical care fellow at UPMC, pursuing a Masters in Medical Education. She is passionate about physiology, mechanical ventilation, adult-learning theory, and global health. She enjoys traveling, cooking, reading, and generally being outdoors.
Twitter: @RMarieW1
Adam Szulewski, MD, FRCPC, MHPE, PhD is Associate Professor in the Departments of Emergency Medicine and Psychology, and Program Director of the Fellowship in Resuscitation and Reanimation Medicine at Queen's University. Adam is interested in understanding how physicians think. His academic work focuses on the application of cognitive load theory to medicine as well as the development and assessment of medical expertise. Adam completed his emergency medicine training as well as a fellowship in resuscitation and reanimation medicine at Queen’s University. He completed a Master of Health Professions Education at Maastricht University where he also received the best dissertation award for his PhD thesis on physician expertise development in resuscitation medicine. Adam is currently the Educational Scholarship Lead for postgraduate medical education at Queen's and a Clinician Educator with the Royal College of Physicians and Surgeons of Canada.
Twitter: @ Adam_Szulewski
