Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909456528310272 |
|---|---|
| author | Siddiqui, Saad Mashkoor Sheikh, Mohammad Ali Aleem, Muhammad Singh, Kajol R |
| author_facet | Siddiqui, Saad Mashkoor Sheikh, Mohammad Ali Aleem, Muhammad Singh, Kajol R |
| contents | In this work, we investigate the efficacy of various adapter architectures on supervised binary classification tasks from the SuperGLUE benchmark as well as a supervised multi-class news category classification task from Kaggle. Specifically, we compare classification performance and time complexity of three transformer models, namely DistilBERT, ELECTRA, and BART, using conventional fine-tuning as well as nine state-of-the-art (SoTA) adapter architectures. Our analysis reveals performance differences across adapter architectures, highlighting their ability to achieve comparable or better performance relative to fine-tuning at a fraction of the training time. Similar results are observed on the new classification task, further supporting our findings and demonstrating adapters as efficient and flexible alternatives to fine-tuning. This study provides valuable insights and guidelines for selecting and implementing adapters in diverse natural language processing (NLP) applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_08271 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models Siddiqui, Saad Mashkoor Sheikh, Mohammad Ali Aleem, Muhammad Singh, Kajol R Computation and Language Artificial Intelligence In this work, we investigate the efficacy of various adapter architectures on supervised binary classification tasks from the SuperGLUE benchmark as well as a supervised multi-class news category classification task from Kaggle. Specifically, we compare classification performance and time complexity of three transformer models, namely DistilBERT, ELECTRA, and BART, using conventional fine-tuning as well as nine state-of-the-art (SoTA) adapter architectures. Our analysis reveals performance differences across adapter architectures, highlighting their ability to achieve comparable or better performance relative to fine-tuning at a fraction of the training time. Similar results are observed on the new classification task, further supporting our findings and demonstrating adapters as efficient and flexible alternatives to fine-tuning. This study provides valuable insights and guidelines for selecting and implementing adapters in diverse natural language processing (NLP) applications. |
| title | Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2501.08271 |