Cover Image

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Zikang, Zhou, Kun, Zhao, Wayne Xin, Gao, Dawei, Li, Yaliang, Wen, Ji-Rong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Computer Vision and Pattern Recognition
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Less is More: High-value Data Selection for Visual Instruction Tuning
by: Liu, Zikang, et al.
Published: (2024)

What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning
by: Du, Yifan, et al.
Published: (2023)

Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning
by: Min, Yingqian, et al.
Published: (2024)

PAL-UI: Planning with Active Look-back for Vision-Based GUI Agents
by: Liu, Zikang, et al.
Published: (2025)

COCO is "ALL'' You Need for Visual Instruction Fine-tuning
by: Han, Xiaotian, et al.
Published: (2024)

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment
by: Chen, Zhipeng, et al.
Published: (2024)

Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models
by: Li, Yifan, et al.
Published: (2024)

LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
by: You, Zebin, et al.
Published: (2025)

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph
by: Jiang, Jinhao, et al.
Published: (2023)

Do Vision-Language Models Really Understand Visual Language?
by: Hou, Yifan, et al.
Published: (2024)

Fine-tuning Large Language Models with Sequential Instructions
by: Hu, Hanxu, et al.
Published: (2024)

Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models
by: Li, Bin, et al.
Published: (2025)

Do We Really Need a Large Number of Visual Prompts?
by: Kim, Youngeun, et al.
Published: (2023)

Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers
by: Tang, Xinyu, et al.
Published: (2024)

Evian: Towards Explainable Visual Instruction-tuning Data Auditing
by: Jia, Zimu, et al.
Published: (2026)

Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach
by: Zhou, Meng, et al.
Published: (2024)

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint
by: Chen, Zhipeng, et al.
Published: (2024)

Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models
by: Sun, Yuchong, et al.
Published: (2023)

Demystifying Instruction Mixing for Fine-tuning Large Language Models
by: Wang, Renxi, et al.
Published: (2023)

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
by: Bai, Yuelin, et al.
Published: (2024)

ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild
by: Masry, Ahmed, et al.
Published: (2024)

Instruction-Free Tuning of Large Vision Language Models for Medical Instruction Following
by: Kang, Myeongkyun, et al.
Published: (2026)

Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models
by: Chen, Yushuo, et al.
Published: (2024)

Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning
by: Lu, Yuheng, et al.
Published: (2025)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)

Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions
by: Li, Juncheng, et al.
Published: (2023)

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
by: Bai, Jiamu, et al.
Published: (2024)

Phased Instruction Fine-Tuning for Large Language Models
by: Pang, Wei, et al.
Published: (2024)

Finer: Investigating and Enhancing Fine-Grained Visual Concept Recognition in Large Vision Language Models
by: Kim, Jeonghwan, et al.
Published: (2024)

B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructions
by: Zhang, Hao, et al.
Published: (2024)

MotIF: Motion Instruction Fine-tuning
by: Hwang, Minyoung, et al.
Published: (2024)

Do Clinical Question Answering Systems Really Need Specialised Medical Fine Tuning?
by: Ray, Sushant Kumar, et al.
Published: (2026)

TaskGalaxy: Scaling Multi-modal Instruction Fine-tuning with Tens of Thousands Vision Task Types
by: Chen, Jiankang, et al.
Published: (2025)

Instruction-Guided Fusion of Multi-Layer Visual Features in Large Vision-Language Models
by: Li, Xu, et al.
Published: (2024)

CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models
by: Zhou, Nan, et al.
Published: (2026)

Learning or Self-aligning? Rethinking Instruction Fine-tuning
by: Ren, Mengjie, et al.
Published: (2024)

Exploring the Design Space of Visual Context Representation in Video MLLMs
by: Du, Yifan, et al.
Published: (2024)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
by: Song, Chiyu, et al.
Published: (2023)

How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
by: Zhang, Huanyu, et al.
Published: (2026)

A Study of Backdoors in Instruction Fine-tuned Language Models
by: Raghuram, Jayaram, et al.
Published: (2024)