Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study

Dylan Montoya-Lora; Andrés F. Giraldo-Forero; Ever A. Torres-Silva; Santiago Rua; Sara Arango-Valencia; Juan G. Barrientos-Gómez; Jose F. Florez-Arango; Andrés Orozco-Duque

doi:10.1007/978-3-031-89514-2_5

Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study

Dylan Montoya-Lora, Andrés F. Giraldo-Forero, Ever A. Torres-Silva, Santiago Rua, Sara Arango-Valencia, Juan G. Barrientos-Gómez, Jose F. Florez-Arango, Andrés Orozco-Duque

Grupo de Investigación en Salud Pública

Producción científica: Capítulo del libro/informe/acta de congreso › Ponencia publicada en las memorias del evento con ISBN › revisión exhaustiva

2 Citas (Scopus)

Resumen

Extracting Named Entity Recognition (NER) associated with signs and symptoms, diseases and syndromes, or health care activities from raw text in maternal electronic health records (EHR) can help in the management of high maternal risk. In addition, identify if the entity is present, absent, or is only a possibility is important to avoid misinterpretation. In this paper, we present an implementation of an automatic information extraction scheme from text in EHR oriented to detect entities and assertions in maternal electronic health records using GPT-3.5 turbo through a few shot architecture. Additionally, we developed a prompt-based approach to interact with the model and obtain NER responses in JSON format, enabling integration with existing healthcare systems and workflows. Our results show an F1-score of 0.75 for NER and 0.76 for assertions. These results show that large languaje models can be used to extract information from EHRs which can improve patient follow-up.

Idioma original	Inglés
Título de la publicación alojada	X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024
Editores	Fabiola M. Martinez-Licona, Virginia L. Ballarin, Ernesto A. Ibarra-Ramírez, Sandra M. Pérez-Buitrago, Luis R. Berriere
Editorial	Springer Science and Business Media Deutschland GmbH
Páginas	53-62
Número de páginas	10
ISBN (versión impresa)	9783031895135
DOI	https://doi.org/10.1007/978-3-031-89514-2_5
Estado	Publicada - 2025
Evento	10th Latin American Conference on Biomedical Engineering, CLAIB 2024 - Panama City, Panamá Duración: 2 oct. 2024 → 5 oct. 2024

Serie de la publicación

Nombre	IFMBE Proceedings
Volumen	121
ISSN (versión impresa)	1680-0737
ISSN (versión digital)	1433-9277

Conferencia

Conferencia	10th Latin American Conference on Biomedical Engineering, CLAIB 2024
País/Territorio	Panamá
Ciudad	Panama City
Período	2/10/24 → 5/10/24

Nota bibliográfica

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

Tipos de Productos Minciencias

Eventos científicos con componente de apropiación

Acceder al documento

10.1007/978-3-031-89514-2_5

Citar esto

Montoya-Lora, D., Giraldo-Forero, A. F., Torres-Silva, E. A., Rua, S., Arango-Valencia, S., Barrientos-Gómez, J. G., Florez-Arango, J. F., & Orozco-Duque, A. (2025). Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study. En F. M. Martinez-Licona, V. L. Ballarin, E. A. Ibarra-Ramírez, S. M. Pérez-Buitrago, & L. R. Berriere (Eds.), X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024 (pp. 53-62). (IFMBE Proceedings; Vol. 121). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-89514-2_5

Montoya-Lora, Dylan ; Giraldo-Forero, Andrés F. ; Torres-Silva, Ever A. et al. / Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records : A Pilot Study. X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024. editor / Fabiola M. Martinez-Licona ; Virginia L. Ballarin ; Ernesto A. Ibarra-Ramírez ; Sandra M. Pérez-Buitrago ; Luis R. Berriere. Springer Science and Business Media Deutschland GmbH, 2025. pp. 53-62 (IFMBE Proceedings).

@inproceedings{65ae97529a7a483baa76bbe4166acf26,

title = "Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study",

abstract = "Extracting Named Entity Recognition (NER) associated with signs and symptoms, diseases and syndromes, or health care activities from raw text in maternal electronic health records (EHR) can help in the management of high maternal risk. In addition, identify if the entity is present, absent, or is only a possibility is important to avoid misinterpretation. In this paper, we present an implementation of an automatic information extraction scheme from text in EHR oriented to detect entities and assertions in maternal electronic health records using GPT-3.5 turbo through a few shot architecture. Additionally, we developed a prompt-based approach to interact with the model and obtain NER responses in JSON format, enabling integration with existing healthcare systems and workflows. Our results show an F1-score of 0.75 for NER and 0.76 for assertions. These results show that large languaje models can be used to extract information from EHRs which can improve patient follow-up.",

keywords = "Electronic Health Records, Large Language Models, Named Entity Recognition, Retrieval Augmented Generation",

author = "Dylan Montoya-Lora and Giraldo-Forero, \{Andr{\'e}s F.\} and Torres-Silva, \{Ever A.\} and Santiago Rua and Sara Arango-Valencia and Barrientos-G{\'o}mez, \{Juan G.\} and Florez-Arango, \{Jose F.\} and Andr{\'e}s Orozco-Duque",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 10th Latin American Conference on Biomedical Engineering, CLAIB 2024 ; Conference date: 02-10-2024 Through 05-10-2024",

year = "2025",

doi = "10.1007/978-3-031-89514-2\_5",

language = "Ingl{\'e}s",

isbn = "9783031895135",

series = "IFMBE Proceedings",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "53--62",

editor = "Martinez-Licona, \{Fabiola M.\} and Ballarin, \{Virginia L.\} and Ibarra-Ram{\'i}rez, \{Ernesto A.\} and P{\'e}rez-Buitrago, \{Sandra M.\} and Berriere, \{Luis R.\}",

booktitle = "X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024",

}

Montoya-Lora, D, Giraldo-Forero, AF, Torres-Silva, EA, Rua, S, Arango-Valencia, S, Barrientos-Gómez, JG, Florez-Arango, JF & Orozco-Duque, A 2025, Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study. En FM Martinez-Licona, VL Ballarin, EA Ibarra-Ramírez, SM Pérez-Buitrago & LR Berriere (eds.), X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024. IFMBE Proceedings, vol. 121, Springer Science and Business Media Deutschland GmbH, pp. 53-62, 10th Latin American Conference on Biomedical Engineering, CLAIB 2024, Panama City, Panamá, 2/10/24. https://doi.org/10.1007/978-3-031-89514-2_5

Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study. / Montoya-Lora, Dylan; Giraldo-Forero, Andrés F.; Torres-Silva, Ever A. et al.
X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024. ed. / Fabiola M. Martinez-Licona; Virginia L. Ballarin; Ernesto A. Ibarra-Ramírez; Sandra M. Pérez-Buitrago; Luis R. Berriere. Springer Science and Business Media Deutschland GmbH, 2025. p. 53-62 (IFMBE Proceedings; Vol. 121).

Producción científica: Capítulo del libro/informe/acta de congreso › Ponencia publicada en las memorias del evento con ISBN › revisión exhaustiva

TY - GEN

T1 - Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records

T2 - 10th Latin American Conference on Biomedical Engineering, CLAIB 2024

AU - Montoya-Lora, Dylan

AU - Giraldo-Forero, Andrés F.

AU - Torres-Silva, Ever A.

AU - Rua, Santiago

AU - Arango-Valencia, Sara

AU - Barrientos-Gómez, Juan G.

AU - Florez-Arango, Jose F.

AU - Orozco-Duque, Andrés

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025

Y1 - 2025

N2 - Extracting Named Entity Recognition (NER) associated with signs and symptoms, diseases and syndromes, or health care activities from raw text in maternal electronic health records (EHR) can help in the management of high maternal risk. In addition, identify if the entity is present, absent, or is only a possibility is important to avoid misinterpretation. In this paper, we present an implementation of an automatic information extraction scheme from text in EHR oriented to detect entities and assertions in maternal electronic health records using GPT-3.5 turbo through a few shot architecture. Additionally, we developed a prompt-based approach to interact with the model and obtain NER responses in JSON format, enabling integration with existing healthcare systems and workflows. Our results show an F1-score of 0.75 for NER and 0.76 for assertions. These results show that large languaje models can be used to extract information from EHRs which can improve patient follow-up.

AB - Extracting Named Entity Recognition (NER) associated with signs and symptoms, diseases and syndromes, or health care activities from raw text in maternal electronic health records (EHR) can help in the management of high maternal risk. In addition, identify if the entity is present, absent, or is only a possibility is important to avoid misinterpretation. In this paper, we present an implementation of an automatic information extraction scheme from text in EHR oriented to detect entities and assertions in maternal electronic health records using GPT-3.5 turbo through a few shot architecture. Additionally, we developed a prompt-based approach to interact with the model and obtain NER responses in JSON format, enabling integration with existing healthcare systems and workflows. Our results show an F1-score of 0.75 for NER and 0.76 for assertions. These results show that large languaje models can be used to extract information from EHRs which can improve patient follow-up.

KW - Electronic Health Records

KW - Large Language Models

KW - Named Entity Recognition

KW - Retrieval Augmented Generation

UR - https://www.scopus.com/pages/publications/105004661625

U2 - 10.1007/978-3-031-89514-2_5

DO - 10.1007/978-3-031-89514-2_5

M3 - Ponencia publicada en las memorias del evento con ISBN

AN - SCOPUS:105004661625

SN - 9783031895135

T3 - IFMBE Proceedings

SP - 53

EP - 62

BT - X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024

A2 - Martinez-Licona, Fabiola M.

A2 - Ballarin, Virginia L.

A2 - Ibarra-Ramírez, Ernesto A.

A2 - Pérez-Buitrago, Sandra M.

A2 - Berriere, Luis R.

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 2 October 2024 through 5 October 2024

ER -

Montoya-Lora D, Giraldo-Forero AF, Torres-Silva EA, Rua S, Arango-Valencia S, Barrientos-Gómez JG et al. Prompt Engineering of GPT-3.5 for Extracting Signs and Symptoms and Their Assertions in Maternal Electronic Health Records: A Pilot Study. En Martinez-Licona FM, Ballarin VL, Ibarra-Ramírez EA, Pérez-Buitrago SM, Berriere LR, editores, X Latin American Conference on Biomedical Engineering - Proceedings of CLAIB 2024. Springer Science and Business Media Deutschland GmbH. 2025. p. 53-62. (IFMBE Proceedings). doi: 10.1007/978-3-031-89514-2_5