LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909259154849792 |
|---|---|
| author | Le-Duc, Khai Zhang, Ryan Nguyen, Ngoc Son Pham, Tan-Hanh Dao, Anh Ngo, Ba Hung Nguyen, Anh Totti Hy, Truong-Son |
| author_facet | Le-Duc, Khai Zhang, Ryan Nguyen, Ngoc Son Pham, Tan-Hanh Dao, Anh Ngo, Ba Hung Nguyen, Anh Totti Hy, Truong-Son |
| contents | Vision-language models have been extensively explored across a wide range of tasks, achieving satisfactory performance; however, their application in medical imaging remains underexplored. In this work, we propose a unified framework - LiteGPT - for the medical imaging. We leverage multiple pre-trained visual encoders to enrich information and enhance the performance of vision-language models. To the best of our knowledge, this is the first study to utilize vision-language models for the novel task of joint localization and classification in medical images. Besides, we are pioneers in providing baselines for disease localization in chest X-rays. Finally, we set new state-of-the-art performance in the image classification task on the well-benchmarked VinDr-CXR dataset. All code and models are publicly available online: https://github.com/leduckhai/LiteGPT |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12064 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task Le-Duc, Khai Zhang, Ryan Nguyen, Ngoc Son Pham, Tan-Hanh Dao, Anh Ngo, Ba Hung Nguyen, Anh Totti Hy, Truong-Son Image and Video Processing Computation and Language Computer Vision and Pattern Recognition Machine Learning Multimedia Vision-language models have been extensively explored across a wide range of tasks, achieving satisfactory performance; however, their application in medical imaging remains underexplored. In this work, we propose a unified framework - LiteGPT - for the medical imaging. We leverage multiple pre-trained visual encoders to enrich information and enhance the performance of vision-language models. To the best of our knowledge, this is the first study to utilize vision-language models for the novel task of joint localization and classification in medical images. Besides, we are pioneers in providing baselines for disease localization in chest X-rays. Finally, we set new state-of-the-art performance in the image classification task on the well-benchmarked VinDr-CXR dataset. All code and models are publicly available online: https://github.com/leduckhai/LiteGPT |
| title | LiteGPT: Large Vision-Language Model for Joint Chest X-ray Localization and Classification Task |
| topic | Image and Video Processing Computation and Language Computer Vision and Pattern Recognition Machine Learning Multimedia |
| url | https://arxiv.org/abs/2407.12064 |