Online-PVLM: Advancing Personalized VLMs with Online Concept Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912772373086208 |
|---|---|
| author | Bai, Huiyu Wang, Runze Du, Zhuoyun Zhao, Yiyang Zhang, Fengji Chen, Haoyu Zhu, Xiaoyong Zheng, Bo Zhao, Xuejiao |
| author_facet | Bai, Huiyu Wang, Runze Du, Zhuoyun Zhao, Yiyang Zhang, Fengji Chen, Haoyu Zhu, Xiaoyong Zheng, Bo Zhao, Xuejiao |
| contents | Personalized Visual Language Models (VLMs) are gaining increasing attention for their formidable ability in user-specific concepts aligned interactions (e.g., identifying a user's bike). Existing methods typically require the learning of separate embeddings for each new concept, which fails to support real-time adaptation during testing. This limitation becomes particularly pronounced in large-scale scenarios, where efficient retrieval of concept embeddings is not achievable. To alleviate this gap, we propose Online-PVLM, a framework for online concept learning by leveraging hyperbolic representations. Our approach makes a train-free paradigm for concept embeddings generation at test time, making the use of personalized VLMs both scalable and efficient. In addition, we develop OP-Eval, a comprehensive and large-scale benchmark comprising 1,292 concepts and over 30K high-quality instances with diverse question types, designed to rigorously assess online concept learning in realistic scenarios. Extensive experiments demonstrate the state-of-the-art performance of our proposed framework. Our source code and dataset will be made available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20056 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Online-PVLM: Advancing Personalized VLMs with Online Concept Learning Bai, Huiyu Wang, Runze Du, Zhuoyun Zhao, Yiyang Zhang, Fengji Chen, Haoyu Zhu, Xiaoyong Zheng, Bo Zhao, Xuejiao Computation and Language Personalized Visual Language Models (VLMs) are gaining increasing attention for their formidable ability in user-specific concepts aligned interactions (e.g., identifying a user's bike). Existing methods typically require the learning of separate embeddings for each new concept, which fails to support real-time adaptation during testing. This limitation becomes particularly pronounced in large-scale scenarios, where efficient retrieval of concept embeddings is not achievable. To alleviate this gap, we propose Online-PVLM, a framework for online concept learning by leveraging hyperbolic representations. Our approach makes a train-free paradigm for concept embeddings generation at test time, making the use of personalized VLMs both scalable and efficient. In addition, we develop OP-Eval, a comprehensive and large-scale benchmark comprising 1,292 concepts and over 30K high-quality instances with diverse question types, designed to rigorously assess online concept learning in realistic scenarios. Extensive experiments demonstrate the state-of-the-art performance of our proposed framework. Our source code and dataset will be made available. |
| title | Online-PVLM: Advancing Personalized VLMs with Online Concept Learning |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2511.20056 |