Usefulness of the large language model ChatGPT (GPT-4) as a diagnostic tool and information source in dermatology

Jacob P.S. Nielsen*, Christian Grønhøj, Lone Skov, Mette Gyldenløve

*Corresponding author af dette arbejde

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Abstract

Background
The field of artificial intelligence is rapidly evolving. As an easily accessible platform with vast user engagement, the Chat Generative Pre-Trained Transformer (ChatGPT) holds great promise in medicine, with the latest version, GPT-4, capable of analyzing clinical images.

Objectives
To evaluate ChatGPT as a diagnostic tool and information source in clinical dermatology.

Methods
A total of 15 clinical images were selected from the Danish web atlas, Danderm, depicting various common and rare skin conditions. The images were uploaded to ChatGPT version GPT-4, which was prompted with ‘Please provide a description, a potential diagnosis, and treatment options for the following dermatological condition’. The generated responses were assessed by senior registrars in dermatology and consultant dermatologists in terms of accuracy, relevance, and depth (scale 1–5), and in addition, the image quality was rated (scale 0–10). Demographic and professional information about the respondents was registered.

Results
A total of 23 physicians participated in the study. The majority of the respondents were consultant dermatologists (83%), and 48% had more than 10 years of training. The overall image quality had a median rating of 10 out of 10 [interquartile range (IQR): 9–10]. The overall median rating of the ChatGPT generated responses was 2 (IQR: 1–4), while overall median ratings in terms of relevance, accuracy, and depth were 2 (IQR: 1–4), 3 (IQR: 2–4) and 2 (IQR: 1–3), respectively.

Conclusions
Despite the advancements in ChatGPT, including newly added image processing capabilities, the chatbot demonstrated significant limitations in providing reliable and clinically useful responses to illustrative images of various dermatological conditions.
OriginalsprogEngelsk
TidsskriftJEADV Clinical Practice
ISSN2768-6566
DOI
StatusAccepteret/In press - 2024

Bibliografisk note

Publisher Copyright:
© 2024 The Authors. JEADV Clinical Practice published by John Wiley & Sons Ltd on behalf of European Academy of Dermatology and Venereology.

Citationsformater