TY - JOUR
T1 - Effect of Machine Learning on Dispatcher Recognition of Out-of-Hospital Cardiac Arrest During Calls to Emergency Medical Services
T2 - A Randomized Clinical Trial
AU - Blomberg, Stig Nikolaj
AU - Christensen, Helle Collatz
AU - Lippert, Freddy
AU - Ersbøll, Annette Kjær
AU - Torp-Petersen, Christian
AU - Sayre, Michael R.
AU - Kudenchuk, Peter J.
AU - Folke, Fredrik
PY - 2021
Y1 - 2021
N2 - Importance: Emergency medical dispatchers fail to identify approximately 25% of cases of out-of-hospital cardiac arrest (OHCA), resulting in lost opportunities to save lives by initiating cardiopulmonary resuscitation. Objective: To examine how a machine learning model trained to identify OHCA and alert dispatchers during emergency calls affected OHCA recognition and response. Design, Setting, and Participants: This double-masked, 2-group, randomized clinical trial analyzed all calls to emergency number 112 (equivalent to 911) in Denmark. Calls were processed by a machine learning model using speech recognition software. The machine learning model assessed ongoing calls, and calls in which the model identified OHCA were randomized. The trial was performed at Copenhagen Emergency Medical Services, Denmark, between September 1, 2018, and December 31, 2019. Intervention: Dispatchers in the intervention group were alerted when the machine learning model identified out-of-hospital cardiac arrest, and those in the control group followed normal protocols without alert. Main Outcomes and Measures: The primary end point was the rate of dispatcher recognition of subsequently confirmed OHCA. Results: A total of 169 049 emergency calls were examined, of which the machine learning model identified 5242 as suspected OHCA. Calls were randomized to control (2661 [50.8%]) or intervention (2581 [49.2%]) groups. Of these, 336 (12.6%) and 318 (12.3%), respectively, had confirmed OHCA. The mean (SD) age among of these 654 patients was 70 (16.1) years, and 419 of 627 patients (67.8%) with known gender were men. Dispatchers in the intervention group recognized 296 confirmed OHCA cases (93.1%) with machine learning assistance compared with 304 confirmed OHCA cases (90.5%) using standard protocols without machine learning assistance (P = .15). Machine learning alerts alone had a significantly higher sensitivity than dispatchers without alerts for confirmed OHCA (85.0% vs 77.5%; P < .001) but lower specificity (97.4% vs 99.6%; P < .001) and positive predictive value (17.8% vs 55.8%; P < .001). Conclusions and Relevance: This randomized clinical trial did not find any significant improvement in dispatchers' ability to recognize cardiac arrest when supported by machine learning even though artificial intelligence did surpass human recognition. Trial Registration: ClinicalTrials.gov Identifier: NCT04219306.
AB - Importance: Emergency medical dispatchers fail to identify approximately 25% of cases of out-of-hospital cardiac arrest (OHCA), resulting in lost opportunities to save lives by initiating cardiopulmonary resuscitation. Objective: To examine how a machine learning model trained to identify OHCA and alert dispatchers during emergency calls affected OHCA recognition and response. Design, Setting, and Participants: This double-masked, 2-group, randomized clinical trial analyzed all calls to emergency number 112 (equivalent to 911) in Denmark. Calls were processed by a machine learning model using speech recognition software. The machine learning model assessed ongoing calls, and calls in which the model identified OHCA were randomized. The trial was performed at Copenhagen Emergency Medical Services, Denmark, between September 1, 2018, and December 31, 2019. Intervention: Dispatchers in the intervention group were alerted when the machine learning model identified out-of-hospital cardiac arrest, and those in the control group followed normal protocols without alert. Main Outcomes and Measures: The primary end point was the rate of dispatcher recognition of subsequently confirmed OHCA. Results: A total of 169 049 emergency calls were examined, of which the machine learning model identified 5242 as suspected OHCA. Calls were randomized to control (2661 [50.8%]) or intervention (2581 [49.2%]) groups. Of these, 336 (12.6%) and 318 (12.3%), respectively, had confirmed OHCA. The mean (SD) age among of these 654 patients was 70 (16.1) years, and 419 of 627 patients (67.8%) with known gender were men. Dispatchers in the intervention group recognized 296 confirmed OHCA cases (93.1%) with machine learning assistance compared with 304 confirmed OHCA cases (90.5%) using standard protocols without machine learning assistance (P = .15). Machine learning alerts alone had a significantly higher sensitivity than dispatchers without alerts for confirmed OHCA (85.0% vs 77.5%; P < .001) but lower specificity (97.4% vs 99.6%; P < .001) and positive predictive value (17.8% vs 55.8%; P < .001). Conclusions and Relevance: This randomized clinical trial did not find any significant improvement in dispatchers' ability to recognize cardiac arrest when supported by machine learning even though artificial intelligence did surpass human recognition. Trial Registration: ClinicalTrials.gov Identifier: NCT04219306.
U2 - 10.1001/jamanetworkopen.2020.32320
DO - 10.1001/jamanetworkopen.2020.32320
M3 - Journal article
C2 - 33404620
AN - SCOPUS:85100280242
VL - 4
SP - e2032320
JO - JAMA Network Open
JF - JAMA Network Open
SN - 2574-3805
IS - 1
ER -