Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2310.16897 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917586716852224 |
|---|---|
| author | Helland, Solveig Gavagnin, Elena de Spindler, Alexandre |
| author_facet | Helland, Solveig Gavagnin, Elena de Spindler, Alexandre |
| contents | The growing capabilities of transformer models pave the way for solving increasingly complex NLP tasks. A key to supporting application-specific requirements is the ability to fine-tune. However, compiling a fine-tuning dataset tailored to complex tasks is tedious and results in large datasets, limiting the ability to control transformer output. We present an approach in which complex tasks are divided into simpler subtasks. Multiple transformer models are fine-tuned to one subtask each, and lined up to accomplish the complex task. This simplifies the compilation of fine-tuning datasets and increases overall controllability. Using the example of reducing gender bias as a complex task, we demonstrate our approach and show that it performs better than using a single model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_16897 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Divide et Impera: Multi-Transformer Architectures for Complex NLP-Tasks Helland, Solveig Gavagnin, Elena de Spindler, Alexandre Computation and Language The growing capabilities of transformer models pave the way for solving increasingly complex NLP tasks. A key to supporting application-specific requirements is the ability to fine-tune. However, compiling a fine-tuning dataset tailored to complex tasks is tedious and results in large datasets, limiting the ability to control transformer output. We present an approach in which complex tasks are divided into simpler subtasks. Multiple transformer models are fine-tuned to one subtask each, and lined up to accomplish the complex task. This simplifies the compilation of fine-tuning datasets and increases overall controllability. Using the example of reducing gender bias as a complex task, we demonstrate our approach and show that it performs better than using a single model. |
| title | Divide et Impera: Multi-Transformer Architectures for Complex NLP-Tasks |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2310.16897 |