HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909150143840256 |
|---|---|
| author | Zhu, He Wu, Junran Liu, Ruomei Hou, Yue Yuan, Ze Li, Shangzhe Pan, Yicheng Xu, Ke |
| author_facet | Zhu, He Wu, Junran Liu, Ruomei Hou, Yue Yuan, Ze Li, Shangzhe Pan, Yicheng Xu, Ke |
| contents | Existing self-supervised methods in natural language processing (NLP), especially hierarchical text classification (HTC), mainly focus on self-supervised contrastive learning, extremely relying on human-designed augmentation rules to generate contrastive samples, which can potentially corrupt or distort the original information. In this paper, we tend to investigate the feasibility of a contrastive learning scheme in which the semantic and syntactic information inherent in the input sample is adequately reserved in the contrastive samples and fused during the learning process. Specifically, we propose an information lossless contrastive learning strategy for HTC, namely \textbf{H}ierarchy-aware \textbf{I}nformation \textbf{L}ossless contrastive \textbf{L}earning (HILL), which consists of a text encoder representing the input document, and a structure encoder directly generating the positive sample. The structure encoder takes the document embedding as input, extracts the essential syntactic information inherent in the label hierarchy with the principle of structural entropy minimization, and injects the syntactic information into the text representation via hierarchical representation learning. Experiments on three common datasets are conducted to verify the superiority of HILL. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_17307 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification Zhu, He Wu, Junran Liu, Ruomei Hou, Yue Yuan, Ze Li, Shangzhe Pan, Yicheng Xu, Ke Computation and Language Information Theory Existing self-supervised methods in natural language processing (NLP), especially hierarchical text classification (HTC), mainly focus on self-supervised contrastive learning, extremely relying on human-designed augmentation rules to generate contrastive samples, which can potentially corrupt or distort the original information. In this paper, we tend to investigate the feasibility of a contrastive learning scheme in which the semantic and syntactic information inherent in the input sample is adequately reserved in the contrastive samples and fused during the learning process. Specifically, we propose an information lossless contrastive learning strategy for HTC, namely \textbf{H}ierarchy-aware \textbf{I}nformation \textbf{L}ossless contrastive \textbf{L}earning (HILL), which consists of a text encoder representing the input document, and a structure encoder directly generating the positive sample. The structure encoder takes the document embedding as input, extracts the essential syntactic information inherent in the label hierarchy with the principle of structural entropy minimization, and injects the syntactic information into the text representation via hierarchical representation learning. Experiments on three common datasets are conducted to verify the superiority of HILL. |
| title | HILL: Hierarchy-aware Information Lossless Contrastive Learning for Hierarchical Text Classification |
| topic | Computation and Language Information Theory |
| url | https://arxiv.org/abs/2403.17307 |