KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916295176355840 |
|---|---|
| author | Kim, Hajung Kim, Chanhwi Lee, Hoonick Jang, Kyochul Lee, Jiwoo Lee, Kyungjae Kim, Gangwoo Kang, Jaewoo |
| author_facet | Kim, Hajung Kim, Chanhwi Lee, Hoonick Jang, Kyochul Lee, Jiwoo Lee, Kyungjae Kim, Gangwoo Kang, Jaewoo |
| contents | Transforming natural language questions into SQL queries is crucial for precise data retrieval from electronic health record (EHR) databases. A significant challenge in this process is detecting and rejecting unanswerable questions that request information beyond the database's scope or exceed the system's capabilities. In this paper, we introduce a novel text-to-SQL framework that robustly handles out-of-domain questions and verifies the generated queries with query execution.Our framework begins by standardizing the structure of questions into a templated format. We use a powerful large language model (LLM), fine-tuned GPT-3.5 with detailed prompts involving the table schemas of the EHR database system. Our experimental results demonstrate the effectiveness of our framework on the EHRSQL-2024 benchmark benchmark, a shared task in the ClinicalNLP workshop. Although a straightforward fine-tuning of GPT shows promising results on the development set, it struggled with the out-of-domain questions in the test set. With our framework, we improve our system's adaptability and achieve competitive performances in the official leaderboard of the EHRSQL-2024 challenge. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_00014 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR Kim, Hajung Kim, Chanhwi Lee, Hoonick Jang, Kyochul Lee, Jiwoo Lee, Kyungjae Kim, Gangwoo Kang, Jaewoo Databases Artificial Intelligence Computation and Language Information Retrieval Transforming natural language questions into SQL queries is crucial for precise data retrieval from electronic health record (EHR) databases. A significant challenge in this process is detecting and rejecting unanswerable questions that request information beyond the database's scope or exceed the system's capabilities. In this paper, we introduce a novel text-to-SQL framework that robustly handles out-of-domain questions and verifies the generated queries with query execution.Our framework begins by standardizing the structure of questions into a templated format. We use a powerful large language model (LLM), fine-tuned GPT-3.5 with detailed prompts involving the table schemas of the EHR database system. Our experimental results demonstrate the effectiveness of our framework on the EHRSQL-2024 benchmark benchmark, a shared task in the ClinicalNLP workshop. Although a straightforward fine-tuning of GPT shows promising results on the development set, it struggled with the out-of-domain questions in the test set. With our framework, we improve our system's adaptability and achieve competitive performances in the official leaderboard of the EHRSQL-2024 challenge. |
| title | KU-DMIS at EHRSQL 2024:Generating SQL query via question templatization in EHR |
| topic | Databases Artificial Intelligence Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2406.00014 |