Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.09729 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917834295083008 |
|---|---|
| author | Mankash, Tavish Kota, V. S. Chaithanya De, Anish Prakash, Praveen Jadhav, Kshitij |
| author_facet | Mankash, Tavish Kota, V. S. Chaithanya De, Anish Prakash, Praveen Jadhav, Kshitij |
| contents | Hospitals in India still rely on handwritten medical records despite the availability of Electronic Medical Records (EMR), complicating statistical analysis and record retrieval. Handwritten records pose a unique challenge, requiring specialized data for training models to recognize medications and their recommendation patterns. While traditional handwriting recognition approaches employ 2-D LSTMs, recent studies have explored using Multimodal Large Language Models (MLLMs) for OCR tasks. Building on this approach, we focus on extracting medication names and dosages from simulated medical records. Our methodology MIRAGE (Multimodal Identification and Recognition of Annotations in indian GEneral prescriptions) involves fine-tuning the QWEN VL, LLaVA 1.6 and Idefics2 models on 743,118 high resolution simulated medical record images-fully annotated from 1,133 doctors across India. Our approach achieves 82% accuracy in extracting medication names and dosages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09729 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | MIRAGE: Multimodal Identification and Recognition of Annotations in Indian General Prescriptions Mankash, Tavish Kota, V. S. Chaithanya De, Anish Prakash, Praveen Jadhav, Kshitij Computer Vision and Pattern Recognition Artificial Intelligence Hospitals in India still rely on handwritten medical records despite the availability of Electronic Medical Records (EMR), complicating statistical analysis and record retrieval. Handwritten records pose a unique challenge, requiring specialized data for training models to recognize medications and their recommendation patterns. While traditional handwriting recognition approaches employ 2-D LSTMs, recent studies have explored using Multimodal Large Language Models (MLLMs) for OCR tasks. Building on this approach, we focus on extracting medication names and dosages from simulated medical records. Our methodology MIRAGE (Multimodal Identification and Recognition of Annotations in indian GEneral prescriptions) involves fine-tuning the QWEN VL, LLaVA 1.6 and Idefics2 models on 743,118 high resolution simulated medical record images-fully annotated from 1,133 doctors across India. Our approach achieves 82% accuracy in extracting medication names and dosages. |
| title | MIRAGE: Multimodal Identification and Recognition of Annotations in Indian General Prescriptions |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2410.09729 |