Cover Image

OneLLM: One Framework to Align All Modalities with Language

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Jiaming, Gong, Kaixiong, Zhang, Yiyuan, Wang, Jiaqi, Zhang, Kaipeng, Lin, Dahua, Qiao, Yu, Gao, Peng, Yue, Xiangyu
Format:	Preprint
Published:	2023
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Computation and Language Machine Learning Multimedia
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multimodal Long Video Modeling Based on Temporal Dynamic Context
by: Hao, Haoran, et al.
Published: (2025)

One Framework to Rule Them All: Unifying Multimodal Tasks with LLM Neural-Tuning
by: Sun, Hao, et al.
Published: (2024)

Anisotropic Modality Align
by: Yu, Xiaomin, et al.
Published: (2026)

TurboTalk: Progressive Distillation for One-Step Audio-Driven Talking Avatar Generation
by: Liu, Xiangyu, et al.
Published: (2026)

Explore the Limits of Omni-modal Pretraining at Scale
by: Zhang, Yiyuan, et al.
Published: (2024)

Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations
by: Han, Jiaming, et al.
Published: (2025)

One Size Doesn't Fit All: Age-Aware Gamification Mechanics for Multimedia Learning Environments
by: Kaißer, Sarah, et al.
Published: (2025)

RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models
by: Hao, Haoran, et al.
Published: (2024)

RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving
by: Huang, Zhijian, et al.
Published: (2024)

AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
by: Gong, Kaixiong, et al.
Published: (2024)

InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructions
by: Zhang, Yiyuan, et al.
Published: (2024)

CalliReader: Contextualizing Chinese Calligraphy via an Embedding-Aligned Vision-Language Model
by: Luo, Yuxuan, et al.
Published: (2025)

MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
by: Fang, Xinyu, et al.
Published: (2024)

OneDiff: A Generalist Model for Image Difference Captioning
by: Hu, Erdong, et al.
Published: (2024)

Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
by: Liu, Che, et al.
Published: (2026)

VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understanding
by: Gao, Shibo, et al.
Published: (2025)

SonoWorld: From One Image to a 3D Audio-Visual Scene
by: Jin, Derong, et al.
Published: (2026)

ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding
by: Zhang, Zhenxing, et al.
Published: (2024)

OneHOI: Unifying Human-Object Interaction Generation and Editing
by: Hoe, Jiun Tian, et al.
Published: (2026)

One Size, Many Fits: Aligning Diverse Group-Wise Click Preferences in Large-Scale Advertising Image Generation
by: Lu, Shuo, et al.
Published: (2026)

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
by: Liu, Zhiyuan, et al.
Published: (2023)

LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
by: Zhang, Renrui, et al.
Published: (2023)

AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning
by: Gao, Jun, et al.
Published: (2024)

Robust Modality-incomplete Anomaly Detection: A Modality-instructive Framework with Benchmark
by: Miao, Bingchen, et al.
Published: (2024)

OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities
by: Chen, Lichang, et al.
Published: (2024)

Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM
by: Ji, Yatai, et al.
Published: (2024)

A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding
by: Lu, Jinghui, et al.
Published: (2024)

DreamArtist++: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter
by: Dong, Ziyi, et al.
Published: (2022)

SLAM-LLM: A Modular, Open-Source Multimodal Large Language Model Framework and Best Practice for Speech, Language, Audio and Music Processing
by: Ma, Ziyang, et al.
Published: (2026)

ALMol: Aligned Language-Molecule Translation LLMs through Offline Preference Contrastive Optimisation
by: Gkoumas, Dimitris
Published: (2024)

DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
by: Cai, Minghong, et al.
Published: (2024)

CLIP-PCQA: Exploring Subjective-Aligned Vision-Language Modeling for Point Cloud Quality Assessment
by: Liu, Yating, et al.
Published: (2025)

GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models
by: Zhu, Xingyu, et al.
Published: (2026)

Modality Gap-Driven Subspace Alignment Training Paradigm For Multimodal Large Language Models
by: Yu, Xiaomin, et al.
Published: (2026)

Evaluating Multimodal Large Language Models on Spoken Sarcasm Understanding
by: Li, Zhu, et al.
Published: (2025)

LongInsightBench: A Comprehensive Benchmark for Evaluating Omni-Modal Models on Human-Centric Long-Video Understanding
by: Han, ZhaoYang, et al.
Published: (2025)

Gradient-Guided Modality Decoupling for Missing-Modality Robustness
by: Wang, Hao, et al.
Published: (2024)

Tracking and Segmenting Anything in Any Modality
by: Zhang, Tianlu, et al.
Published: (2025)

ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Capability for Large Vision-Language Models
by: Liu, Shuo, et al.
Published: (2024)

Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis
by: Feng, Xinyu, et al.
Published: (2024)