Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.05577 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911255383506944 |
|---|---|
| author | Vuong, An Van, Minh-Hao Verma, Prateek Zhao, Chen Wu, Xintao |
| author_facet | Vuong, An Van, Minh-Hao Verma, Prateek Zhao, Chen Wu, Xintao |
| contents | Vision-Language Models (VLMs) have shown strong performance in tasks like visual question answering and multimodal text generation, but their effectiveness in scientific domains such as materials science remains limited. While some machine learning methods have addressed specific challenges in this field, there is still a lack of foundation models designed for broad tasks like polymer property prediction using multimodal data. In this work, we present a multimodal polymer dataset to fine-tune VLMs through instruction-tuning pairs and assess the impact of multimodality on prediction performance. Our fine-tuned models, using LoRA, outperform unimodal and baseline approaches, demonstrating the benefits of multimodal learning. Additionally, this approach reduces the need to train separate models for different properties, lowering deployment and maintenance costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_05577 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Fine-Tuning Vision-Language Models for Multimodal Polymer Property Prediction Vuong, An Van, Minh-Hao Verma, Prateek Zhao, Chen Wu, Xintao Machine Learning Materials Science Artificial Intelligence Computation and Language Vision-Language Models (VLMs) have shown strong performance in tasks like visual question answering and multimodal text generation, but their effectiveness in scientific domains such as materials science remains limited. While some machine learning methods have addressed specific challenges in this field, there is still a lack of foundation models designed for broad tasks like polymer property prediction using multimodal data. In this work, we present a multimodal polymer dataset to fine-tune VLMs through instruction-tuning pairs and assess the impact of multimodality on prediction performance. Our fine-tuned models, using LoRA, outperform unimodal and baseline approaches, demonstrating the benefits of multimodal learning. Additionally, this approach reduces the need to train separate models for different properties, lowering deployment and maintenance costs. |
| title | Fine-Tuning Vision-Language Models for Multimodal Polymer Property Prediction |
| topic | Machine Learning Materials Science Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2511.05577 |