Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929267107954688 |
|---|---|
| author | He, Jianfeng Su, Hang Cai, Jason Shalyminov, Igor Song, Hwanjun Mansour, Saab |
| author_facet | He, Jianfeng Su, Hang Cai, Jason Shalyminov, Igor Song, Hwanjun Mansour, Saab |
| contents | Semi-supervised dialogue summarization (SSDS) leverages model-generated summaries to reduce reliance on human-labeled data and improve the performance of summarization models. While addressing label noise, previous works on semi-supervised learning primarily focus on natural language understanding tasks, assuming each sample has a unique label. However, these methods are not directly applicable to SSDS, as it is a generative task, and each dialogue can be summarized in different ways. In this work, we propose a novel scoring approach, SiCF, which encapsulates three primary dimensions of summarization model quality: Semantic invariance (indicative of model confidence), Coverage (factual recall), and Faithfulness (factual precision). Using the SiCF score, we select unlabeled dialogues with high-quality generated summaries to train summarization models. Comprehensive experiments on three public datasets demonstrate the effectiveness of SiCF scores in uncertainty estimation and semi-supervised learning for dialogue summarization tasks. Our code is available at \url{https://github.com/amazon-science/summarization-sicf-score}. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_04073 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection He, Jianfeng Su, Hang Cai, Jason Shalyminov, Igor Song, Hwanjun Mansour, Saab Computation and Language Artificial Intelligence Semi-supervised dialogue summarization (SSDS) leverages model-generated summaries to reduce reliance on human-labeled data and improve the performance of summarization models. While addressing label noise, previous works on semi-supervised learning primarily focus on natural language understanding tasks, assuming each sample has a unique label. However, these methods are not directly applicable to SSDS, as it is a generative task, and each dialogue can be summarized in different ways. In this work, we propose a novel scoring approach, SiCF, which encapsulates three primary dimensions of summarization model quality: Semantic invariance (indicative of model confidence), Coverage (factual recall), and Faithfulness (factual precision). Using the SiCF score, we select unlabeled dialogues with high-quality generated summaries to train summarization models. Comprehensive experiments on three public datasets demonstrate the effectiveness of SiCF scores in uncertainty estimation and semi-supervised learning for dialogue summarization tasks. Our code is available at \url{https://github.com/amazon-science/summarization-sicf-score}. |
| title | Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2403.04073 |