Cover Image

ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Jieyu, Xue, Le, Song, Linxin, Wang, Jun, Huang, Weikai, Shu, Manli, Yan, An, Ma, Zixian, Niebles, Juan Carlos, Savarese, Silvio, Xiong, Caiming, Chen, Zeyuan, Krishna, Ranjay, Xu, Ran
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LATTE: Learning to Think with Vision Specialists
by: Ma, Zixian, et al.
Published: (2024)

m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks
by: Ma, Zixian, et al.
Published: (2024)

CoAct-1: Computer-using Multi-Agent System with Coding Actions
by: Song, Linxin, et al.
Published: (2025)

xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs
by: Ryoo, Michael S., et al.
Published: (2024)

Strefer: Empowering Video LLMs with Space-Time Referring and Reasoning via Synthetic Instruction Data
by: Zhou, Honglu, et al.
Published: (2025)

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding
by: Xue, Le, et al.
Published: (2023)

MINT-1T: Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens
by: Awadalla, Anas, et al.
Published: (2024)

Hierarchical Point Attention for Indoor 3D Object Detection
by: Shu, Manli, et al.
Published: (2023)

Iterated Learning Improves Compositionality in Large Vision-Language Models
by: Zheng, Chenhao, et al.
Published: (2024)

Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training
by: Gao, Ziqi, et al.
Published: (2024)

You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass
by: Yang, Yinuo, et al.
Published: (2026)

Trust but Verify: Programmatic VLM Evaluation in the Wild
by: Prabhu, Viraj, et al.
Published: (2024)

X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning
by: Panagopoulou, Artemis, et al.
Published: (2023)

Future Optical Flow Prediction Improves Robot Control & Video Generation
by: Ranasinghe, Kanchana, et al.
Published: (2026)

Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models
by: Hu, Yushi, et al.
Published: (2023)

HIVE: Harnessing Human Feedback for Instructional Visual Editing
by: Zhang, Shu, et al.
Published: (2023)

INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness
by: Le, Hung, et al.
Published: (2024)

Enabling High Data Throughput Reinforcement Learning on GPUs: A Domain Agnostic Framework for Data-Driven Scientific Research
by: Lan, Tian, et al.
Published: (2024)

xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations
by: Qin, Can, et al.
Published: (2024)

ViUniT: Visual Unit Tests for More Robust Visual Programming
by: Panagopoulou, Artemis, et al.
Published: (2024)

BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions
by: Awadalla, Anas, et al.
Published: (2024)

DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs
by: Wang, Zhenhailong, et al.
Published: (2025)

Shared Imagination: LLMs Hallucinate Alike
by: Zhou, Yilun, et al.
Published: (2024)

Causal Layering via Conditional Entropy
by: Feigenbaum, Itai, et al.
Published: (2024)

Editing Arbitrary Propositions in LLMs without Subject Labels
by: Feigenbaum, Itai, et al.
Published: (2024)

Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
by: Yao, Weiran, et al.
Published: (2023)

REX: Rapid Exploration and eXploitation for AI Agents
by: Murthy, Rithesh, et al.
Published: (2023)

LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designer
by: Yu, Ning, et al.
Published: (2022)

Offline Training of Language Model Agents with Functions as Learnable Weights
by: Zhang, Shaokun, et al.
Published: (2024)

CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models
by: Li, Jierui, et al.
Published: (2024)

Video-Based Reward Modeling for Computer-Use Agents
by: Song, Linxin, et al.
Published: (2026)

Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
by: Pang, Bo, et al.
Published: (2025)

Task Me Anything
by: Zhang, Jieyu, et al.
Published: (2024)

Robotic VLA Benefits from Joint Learning with Motion Image Diffusion
by: Fang, Yu, et al.
Published: (2025)

Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
by: Wang, Ziyang, et al.
Published: (2025)

Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?
by: Zhang, Tianyi, et al.
Published: (2026)

VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
by: He, Qijia, et al.
Published: (2026)

UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG
by: Peng, Xiangyu, et al.
Published: (2025)

Efficient Inference of Vision Instruction-Following Models with Elastic Cache
by: Liu, Zuyan, et al.
Published: (2024)

xGen-MM (BLIP-3): A Family of Open Large Multimodal Models
by: Xue, Le, et al.
Published: (2024)