Cover Image

Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jin, Yijie, Peng, Junjie, Lin, Xuanchao, Yuan, Haochen, Wang, Lan, Zheng, Cangzhi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stage-wise Fine-tuning for Graph-to-Text Generation
by: Wang, Qingyun, et al.
Published: (2021)

Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis
by: Lin, Ronghao, et al.
Published: (2022)

GraphERE: Jointly Multiple Event-Event Relation Extraction via Graph-Enhanced Event Embeddings
by: Li, Haochen, et al.
Published: (2024)

Towards Efficient LLM-aware Heterogeneous Graph Learning
by: Li, Wenda, et al.
Published: (2025)

TriSPrompt: A Hierarchical Soft Prompt Model for Multimodal Rumor Detection with Incomplete Modalities
by: Chen, Jiajun, et al.
Published: (2025)

KG-CF: Knowledge Graph Completion with Context Filtering under the Guidance of Large Language Models
by: Zheng, Zaiyi, et al.
Published: (2025)

EasyJudge: an Easy-to-use Tool for Comprehensive Response Evaluation of LLMs
by: Li, Yijie, et al.
Published: (2024)

M3HG: Multimodal, Multi-scale, and Multi-type Node Heterogeneous Graph for Emotion Cause Triplet Extraction in Conversations
by: Liang, Qiao, et al.
Published: (2025)

SOD: Step-wise On-policy Distillation for Small Language Model Agents
by: Zhong, Qiyong, et al.
Published: (2026)

LLMBind: A Unified Modality-Task Integration Framework
by: Zhu, Bin, et al.
Published: (2024)

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs
by: Chen, Yurun, et al.
Published: (2025)

CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference
by: Liao, Ruqi, et al.
Published: (2024)

Multimodal Reasoning with Multimodal Knowledge Graph
by: Lee, Junlin, et al.
Published: (2024)

Knowledge Graph-Enhanced Large Language Models via Path Selection
by: Liu, Haochen, et al.
Published: (2024)

ContextRAG: Extraction-Free Hierarchical Graph Construction for Retrieval-Augmented Generation
by: Prosvirnin, Roman, et al.
Published: (2026)

Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention
by: Weng, Yuzhe, et al.
Published: (2024)

Multimodal Table Understanding
by: Zheng, Mingyu, et al.
Published: (2024)

HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs
by: Xiao, Yongkang, et al.
Published: (2025)

RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
by: Wang, Bowen, et al.
Published: (2025)

Enhancing Semantics in Multimodal Chain of Thought via Soft Negative Sampling
by: Zheng, Guangmin, et al.
Published: (2024)

Transformer-Based Multimodal Knowledge Graph Completion with Link-Aware Contexts
by: Ma, Haodi, et al.
Published: (2025)

GSIFN: A Graph-Structured and Interlaced-Masked Multimodal Transformer-based Fusion Network for Multimodal Sentiment Analysis
by: Jin, Yijie
Published: (2024)

2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion
by: Wang, Dongsheng, et al.
Published: (2024)

LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference
by: Kapadia, Shashank, et al.
Published: (2026)

LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings
by: Wang, Duo, et al.
Published: (2024)

Cross-Modal Consistency in Multimodal Large Language Models
by: Zhang, Xiang, et al.
Published: (2024)

Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables
by: Gong, Haisong, et al.
Published: (2024)

HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation
by: Liu, Pei, et al.
Published: (2025)

Heterogeneous Subgraph Transformer for Fake News Detection
by: Zhang, Yuchen, et al.
Published: (2024)

SentGraph: Hierarchical Sentence Graph for Multi-hop Retrieval-Augmented Question Answering
by: Liang, Junli, et al.
Published: (2026)

AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
by: Wang, Zhaowei, et al.
Published: (2023)

Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks
by: Liu, Haoyu, et al.
Published: (2026)

Knowledge-Guided Dynamic Modality Attention Fusion Framework for Multimodal Sentiment Analysis
by: Feng, Xinyu, et al.
Published: (2024)

NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generation
by: Wu, Weiming, et al.
Published: (2025)

Retrieval-Augmented Generation with Hierarchical Knowledge
by: Huang, Haoyu, et al.
Published: (2025)

DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text
by: Li, Shuaiyi, et al.
Published: (2023)

CritiPrefill: A Segment-wise Criticality-based Approach for Prefilling Acceleration in LLMs
by: Lv, Junlin, et al.
Published: (2024)

Diffusion-based Hierarchical Negative Sampling for Multimodal Knowledge Graph Completion
by: Niu, Guanglin, et al.
Published: (2025)

AIMDiT: Modality Augmentation and Interaction via Multimodal Dimension Transformation for Emotion Recognition in Conversations
by: Wu, Sheng, et al.
Published: (2024)

SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Critic
by: Chen, Jikai, et al.
Published: (2025)