Copertina

Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Yiyuan, Ding, Xiaohan, Gong, Kaixiong, Ge, Yixiao, Shan, Ying, Yue, Xiangyu
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition
di: Ding, Xiaohan, et al.
Pubblicazione: (2023)

Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations
di: Zhang, Yiyuan, et al.
Pubblicazione: (2024)

Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Engines
di: Zhang, Zhixin, et al.
Pubblicazione: (2024)

InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructions
di: Zhang, Yiyuan, et al.
Pubblicazione: (2024)

OneLLM: One Framework to Align All Modalities with Language
di: Han, Jiaming, et al.
Pubblicazione: (2023)

Online Vectorized HD Map Construction using Geometry
di: Zhang, Zhixin, et al.
Pubblicazione: (2023)

GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning
di: Chen, Yi, et al.
Pubblicazione: (2025)

Multimodal Long Video Modeling Based on Temporal Dynamic Context
di: Hao, Haoran, et al.
Pubblicazione: (2025)

Supervised Fine-tuning in turn Improves Visual Foundation Models
di: Jiang, Xiaohu, et al.
Pubblicazione: (2024)

EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios
di: Qiu, Lu, et al.
Pubblicazione: (2024)

HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation
di: Xiao, Yicheng, et al.
Pubblicazione: (2025)

Explore the Limits of Omni-modal Pretraining at Scale
di: Zhang, Yiyuan, et al.
Pubblicazione: (2024)

Source-Free Cross-Modal Knowledge Transfer by Unleashing the Potential of Task-Irrelevant Data
di: Zhu, Jinjing, et al.
Pubblicazione: (2024)

Relevant Irrelevance: Generating Alterfactual Explanations for Image Classifiers
di: Mertes, Silvan, et al.
Pubblicazione: (2024)

Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1
di: Chen, Yi, et al.
Pubblicazione: (2025)

Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos
di: Chen, Yi, et al.
Pubblicazione: (2024)

DiTCtrl: Exploring Attention Control in Multi-Modal Diffusion Transformer for Tuning-Free Multi-Prompt Longer Video Generation
di: Cai, Minghong, et al.
Pubblicazione: (2024)

DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA
di: Chen, Yi, et al.
Pubblicazione: (2026)

TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation
di: Lin, Haokun, et al.
Pubblicazione: (2025)

TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs
di: Zhang, Jun, et al.
Pubblicazione: (2025)

Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development
di: Chen, Daoyuan, et al.
Pubblicazione: (2024)

Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation
di: Luo, Zhuoyan, et al.
Pubblicazione: (2024)

Scalable Image Tokenization with Index Backpropagation Quantization
di: Shi, Fengyuan, et al.
Pubblicazione: (2024)

RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformer
di: Ni, Haotian, et al.
Pubblicazione: (2025)

CroMe: Multimodal Fake News Detection using Cross-Modal Tri-Transformer and Metric Learning
di: Choi, Eunjee, et al.
Pubblicazione: (2025)

RoHyDR: Robust Hybrid Diffusion Recovery for Incomplete Multimodal Emotion Recognition
di: Jin, Yuehan, et al.
Pubblicazione: (2025)

Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learning
di: Yaras, Can, et al.
Pubblicazione: (2024)

Deep Multimodal Learning with Missing Modality: A Survey
di: Wu, Renjie, et al.
Pubblicazione: (2024)

LOTUS: Improving Transformer Efficiency with Sparsity Pruning and Data Lottery Tickets
di: Upadhyay, Ojasw
Pubblicazione: (2024)

Towards Multimodal Video Paragraph Captioning Models Robust to Missing Modality
di: Chen, Sishuo, et al.
Pubblicazione: (2024)

Unlabeled Data Improves Fine-Grained Image Zero-shot Classification with Multimodal LLMs
di: Hong, Yunqi, et al.
Pubblicazione: (2025)

Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models
di: Cai, Rui, et al.
Pubblicazione: (2025)

Robust Multimodal Learning via Cross-Modal Proxy Tokens
di: Reza, Md Kaykobad, et al.
Pubblicazione: (2025)

ViT-Lens: Towards Omni-modal Representations
di: Lei, Weixian, et al.
Pubblicazione: (2023)

Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
di: Hu, Tao, et al.
Pubblicazione: (2026)

Multimodal Foundation Model for Cross-Modal Retrieval and Activity Recognition Tasks
di: Matsuishi, Koki, et al.
Pubblicazione: (2025)

Robult: Leveraging Redundancy and Modality Specific Features for Robust Multimodal Learning
di: Nguyen, Duy A., et al.
Pubblicazione: (2025)

NeuroPath: A Neural Pathway Transformer for Joining the Dots of Human Connectomes
di: Wei, Ziquan, et al.
Pubblicazione: (2024)

Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
di: Qiu, Haibo, et al.
Pubblicazione: (2025)

MIND: Modality-Informed Knowledge Distillation Framework for Multimodal Clinical Prediction Tasks
di: Guerra-Manzanares, Alejandro, et al.
Pubblicazione: (2025)