Technical Report on Text Dataset Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912746721771520 |
|---|---|
| author | Ogawa, Keith Ando Yamamoto, Bruno Lopes de Alcantara, Lucas Lauton Zacarias, Victor Bollis, Edson Pellicer, Lucas Costa, Rosimeire Pereira Costa, Anna Helena Reali Jordao, Artur |
| author_facet | Ogawa, Keith Ando Yamamoto, Bruno Lopes de Alcantara, Lucas Lauton Zacarias, Victor Bollis, Edson Pellicer, Lucas Costa, Rosimeire Pereira Costa, Anna Helena Reali Jordao, Artur |
| contents | In the vision domain, dataset distillation arises as a technique to condense a large dataset into a smaller synthetic one that exhibits a similar result in the training process. While image data presents an extensive literature of distillation methods, text dataset distillation has fewer works in comparison. Text dataset distillation initially grew as an adaptation of efforts from the vision universe, as the particularities of the modality became clear obstacles, it rose into a separate branch of research. Several milestones mark the development of this area, such as the introduction of methods that use transformer models, the generation of discrete synthetic text, and the scaling to decoder-only models with over 1B parameters. Despite major advances in modern approaches, the field remains in a maturing phase, with room for improvement on benchmarking standardization, approaches to overcome the discrete nature of text, handling complex tasks, and providing explicit examples of real-world applications. In this report, we review past and recent advances in dataset distillation for text, highlighting different distillation strategies, key contributions, and general challenges. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_03967 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Technical Report on Text Dataset Distillation Ogawa, Keith Ando Yamamoto, Bruno Lopes de Alcantara, Lucas Lauton Zacarias, Victor Bollis, Edson Pellicer, Lucas Costa, Rosimeire Pereira Costa, Anna Helena Reali Jordao, Artur Machine Learning In the vision domain, dataset distillation arises as a technique to condense a large dataset into a smaller synthetic one that exhibits a similar result in the training process. While image data presents an extensive literature of distillation methods, text dataset distillation has fewer works in comparison. Text dataset distillation initially grew as an adaptation of efforts from the vision universe, as the particularities of the modality became clear obstacles, it rose into a separate branch of research. Several milestones mark the development of this area, such as the introduction of methods that use transformer models, the generation of discrete synthetic text, and the scaling to decoder-only models with over 1B parameters. Despite major advances in modern approaches, the field remains in a maturing phase, with room for improvement on benchmarking standardization, approaches to overcome the discrete nature of text, handling complex tasks, and providing explicit examples of real-world applications. In this report, we review past and recent advances in dataset distillation for text, highlighting different distillation strategies, key contributions, and general challenges. |
| title | Technical Report on Text Dataset Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2512.03967 |