Cover Image

Phase Diagram of Vision Large Language Models Inference: A Perspective from Interaction across Image and Instruction

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wei, Houjing, Shi, Yuting, Inoue, Naoya
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Revisiting In-context Learning Inference Circuit in Large Language Models
by: Cho, Hakaze, et al.
Published: (2024)

Phased Instruction Fine-Tuning for Large Language Models
by: Pang, Wei, et al.
Published: (2024)

On Effects of Steering Latent Representation for Large Language Model Unlearning
by: Huu-Tien, Dang, et al.
Published: (2024)

The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMs
by: Tezuka, Hinata, et al.
Published: (2025)

Binary Autoencoder for Mechanistic Interpretability of Large Language Models
by: Cho, Hakaze, et al.
Published: (2025)

Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models
by: Yoshida, Haruto, et al.
Published: (2026)

Instruction-Following Evaluation of Large Vision-Language Models
by: Shiono, Daiki, et al.
Published: (2025)

From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models
by: Tong, Junlong, et al.
Published: (2026)

NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning
by: Zhao, Yufeng, et al.
Published: (2024)

A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
by: Park, Sihyeong, et al.
Published: (2025)

Evaluating Compliance with Visualization Guidelines in Diagrams for Scientific Publications Using Large Vision Language Models
by: Rückert, Johannes, et al.
Published: (2025)

Non-Interactive Symbolic-Aided Chain-of-Thought for Logical Reasoning
by: Nguyen, Phuong Minh, et al.
Published: (2025)

Resilience of Large Language Models for Noisy Instructions
by: Wang, Bin, et al.
Published: (2024)

GraphGPT: Graph Instruction Tuning for Large Language Models
by: Tang, Jiabin, et al.
Published: (2023)

Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?
by: Son, Guijin, et al.
Published: (2024)

StaICC: Standardized Evaluation for Classification Task in In-context Learning
by: Cho, Hakaze, et al.
Published: (2025)

Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
by: Zhang, Jun, et al.
Published: (2026)

Constraint-aware Path Planning from Natural Language Instructions Using Large Language Models
by: Shim, Dylan, et al.
Published: (2026)

Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking
by: Li, Jingru, et al.
Published: (2026)

Diving into Mitigating Hallucinations from a Vision Perspective for Large Vision-Language Models
by: Wang, Weihang, et al.
Published: (2025)

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models
by: Liu, Emmy, et al.
Published: (2024)

Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
by: Wang, Xing, et al.
Published: (2025)

mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model
by: Hu, Anwen, et al.
Published: (2023)

Detecting Knowledge Boundary of Vision Large Language Models by Sampling-Based Inference
by: Chen, Zhuo, et al.
Published: (2025)

Enhancing Large Vision Language Models with Self-Training on Image Comprehension
by: Deng, Yihe, et al.
Published: (2024)

Efficient Tuning and Inference for Large Language Models on Textual Graphs
by: Zhu, Yun, et al.
Published: (2024)

Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis
by: Yang, Haolin, et al.
Published: (2025)

Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models
by: Li, Bin, et al.
Published: (2025)

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models
by: Hayati, Shirley Anugrah, et al.
Published: (2024)

Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarization
by: Laskar, Md Tahmid Rahman, et al.
Published: (2024)

Can We Predict Performance of Large Models across Vision-Language Tasks?
by: Zhao, Qinyu, et al.
Published: (2024)

IAPT: Instruction-Aware Prompt Tuning for Large Language Models
by: Zhu, Wei, et al.
Published: (2024)

Toward Interactive Regional Understanding in Vision-Large Language Models
by: Lee, Jungbeom, et al.
Published: (2024)

LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios
by: Wu, Xiaodong, et al.
Published: (2024)

XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
by: Li, Zhenyu, et al.
Published: (2025)

Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models
by: Sasabuchi, Kazuhiro, et al.
Published: (2025)

AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Models
by: Wei, Yuting, et al.
Published: (2024)

A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Models
by: Chen, Bowen, et al.
Published: (2024)

Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency
by: Zhao, Hanyu, et al.
Published: (2024)

Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding
by: Wang, Xintong, et al.
Published: (2024)