Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.16495 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908628844281856 |
|---|---|
| author | Wang, Lingfeng Lin, Hualing Chen, Senda Wang, Tao Cheng, Changxu Zhong, Yangyang Zheng, Dong Zhao, Wuyue |
| author_facet | Wang, Lingfeng Lin, Hualing Chen, Senda Wang, Tao Cheng, Changxu Zhong, Yangyang Zheng, Dong Zhao, Wuyue |
| contents | While humans effortlessly draw visual objects and shapes by adaptively allocating attention based on their complexity, existing multimodal large language models (MLLMs) remain constrained by rigid token representations. Bridging this gap, we propose ALTo, an adaptive length tokenizer for autoregressive mask generation. To achieve this, a novel token length predictor is designed, along with a length regularization term and a differentiable token chunking strategy. We further build ALToLLM that seamlessly integrates ALTo into MLLM. Preferences on the trade-offs between mask quality and efficiency is implemented by group relative policy optimization (GRPO). Experiments demonstrate that ALToLLM achieves state-of-the-art performance with adaptive token cost on popular segmentation benchmarks. Code and models are released at https://github.com/yayafengzi/ALToLLM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_16495 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | ALTo: Adaptive-Length Tokenizer for Autoregressive Mask Generation Wang, Lingfeng Lin, Hualing Chen, Senda Wang, Tao Cheng, Changxu Zhong, Yangyang Zheng, Dong Zhao, Wuyue Computer Vision and Pattern Recognition While humans effortlessly draw visual objects and shapes by adaptively allocating attention based on their complexity, existing multimodal large language models (MLLMs) remain constrained by rigid token representations. Bridging this gap, we propose ALTo, an adaptive length tokenizer for autoregressive mask generation. To achieve this, a novel token length predictor is designed, along with a length regularization term and a differentiable token chunking strategy. We further build ALToLLM that seamlessly integrates ALTo into MLLM. Preferences on the trade-offs between mask quality and efficiency is implemented by group relative policy optimization (GRPO). Experiments demonstrate that ALToLLM achieves state-of-the-art performance with adaptive token cost on popular segmentation benchmarks. Code and models are released at https://github.com/yayafengzi/ALToLLM. |
| title | ALTo: Adaptive-Length Tokenizer for Autoregressive Mask Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2505.16495 |