Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignment
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914719400460288 |
|---|---|
| author | Wang, Yuanyi Sun, Haifeng Wang, Jiabo Wang, Jingyu Tang, Wei Qi, Qi Sun, Shaoling Liao, Jianxin |
| author_facet | Wang, Yuanyi Sun, Haifeng Wang, Jiabo Wang, Jingyu Tang, Wei Qi, Qi Sun, Shaoling Liao, Jianxin |
| contents | In Multi-Modal Knowledge Graphs (MMKGs), Multi-Modal Entity Alignment (MMEA) is crucial for identifying identical entities across diverse modal attributes. However, semantic inconsistency, mainly due to missing modal attributes, poses a significant challenge. Traditional approaches rely on attribute interpolation, but this often introduces modality noise, distorting the original semantics. Moreover, the lack of a universal theoretical framework limits advancements in achieving semantic consistency. This study introduces a novel approach, DESAlign, which addresses these issues by applying a theoretical framework based on Dirichlet energy to ensure semantic consistency. We discover that semantic inconsistency leads to model overfitting to modality noise, causing performance fluctuations, particularly when modalities are missing. DESAlign innovatively combats over-smoothing and interpolates absent semantics using existing modalities. Our approach includes a multi-modal knowledge graph learning strategy and a propagation technique that employs existing semantic features to compensate for missing ones, providing explicit Euler solutions. Comprehensive evaluations across 60 benchmark splits, including monolingual and bilingual scenarios, demonstrate that DESAlign surpasses existing methods, setting a new standard in performance. Further testing with high rates of missing modalities confirms its robustness, offering an effective solution to semantic inconsistency in real-world MMKGs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_17859 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignment Wang, Yuanyi Sun, Haifeng Wang, Jiabo Wang, Jingyu Tang, Wei Qi, Qi Sun, Shaoling Liao, Jianxin Information Retrieval In Multi-Modal Knowledge Graphs (MMKGs), Multi-Modal Entity Alignment (MMEA) is crucial for identifying identical entities across diverse modal attributes. However, semantic inconsistency, mainly due to missing modal attributes, poses a significant challenge. Traditional approaches rely on attribute interpolation, but this often introduces modality noise, distorting the original semantics. Moreover, the lack of a universal theoretical framework limits advancements in achieving semantic consistency. This study introduces a novel approach, DESAlign, which addresses these issues by applying a theoretical framework based on Dirichlet energy to ensure semantic consistency. We discover that semantic inconsistency leads to model overfitting to modality noise, causing performance fluctuations, particularly when modalities are missing. DESAlign innovatively combats over-smoothing and interpolates absent semantics using existing modalities. Our approach includes a multi-modal knowledge graph learning strategy and a propagation technique that employs existing semantic features to compensate for missing ones, providing explicit Euler solutions. Comprehensive evaluations across 60 benchmark splits, including monolingual and bilingual scenarios, demonstrate that DESAlign surpasses existing methods, setting a new standard in performance. Further testing with high rates of missing modalities confirms its robustness, offering an effective solution to semantic inconsistency in real-world MMKGs. |
| title | Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignment |
| topic | Information Retrieval |
| url | https://arxiv.org/abs/2401.17859 |