Generative or Discriminative? Revisiting Text Classification in the Era of Transformers
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915717680463872 |
|---|---|
| author | Kasa, Siva Rajesh Gupta, Karan Roychowdhury, Sumegh Kumar, Ashutosh Biruduraju, Yaswanth Kasa, Santhosh Kumar Pattisapu, Nikhil Priyatam Bhattacharya, Arindam Agarwal, Shailendra huddar, Vijay |
| author_facet | Kasa, Siva Rajesh Gupta, Karan Roychowdhury, Sumegh Kumar, Ashutosh Biruduraju, Yaswanth Kasa, Santhosh Kumar Pattisapu, Nikhil Priyatam Bhattacharya, Arindam Agarwal, Shailendra huddar, Vijay |
| contents | The comparison between discriminative and generative classifiers has intrigued researchers since Efron's seminal analysis of logistic regression versus discriminant analysis. While early theoretical work established that generative classifiers exhibit lower sample complexity but higher asymptotic error in simple linear settings, these trade-offs remain unexplored in the transformer era. We present the first comprehensive evaluation of modern generative and discriminative architectures - Auto-regressive modeling, Masked Language Modeling, Discrete Diffusion, and Encoders for text classification. Our study reveals that the classical 'two regimes' phenomenon manifests distinctly across different architectures and training paradigms. Beyond accuracy, we analyze sample efficiency, calibration, noise robustness, and ordinality across diverse scenarios. Our findings offer practical guidance for selecting the most suitable modeling approach based on real-world constraints such as latency and data limitations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12181 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generative or Discriminative? Revisiting Text Classification in the Era of Transformers Kasa, Siva Rajesh Gupta, Karan Roychowdhury, Sumegh Kumar, Ashutosh Biruduraju, Yaswanth Kasa, Santhosh Kumar Pattisapu, Nikhil Priyatam Bhattacharya, Arindam Agarwal, Shailendra huddar, Vijay Machine Learning Computation and Language The comparison between discriminative and generative classifiers has intrigued researchers since Efron's seminal analysis of logistic regression versus discriminant analysis. While early theoretical work established that generative classifiers exhibit lower sample complexity but higher asymptotic error in simple linear settings, these trade-offs remain unexplored in the transformer era. We present the first comprehensive evaluation of modern generative and discriminative architectures - Auto-regressive modeling, Masked Language Modeling, Discrete Diffusion, and Encoders for text classification. Our study reveals that the classical 'two regimes' phenomenon manifests distinctly across different architectures and training paradigms. Beyond accuracy, we analyze sample efficiency, calibration, noise robustness, and ordinality across diverse scenarios. Our findings offer practical guidance for selecting the most suitable modeling approach based on real-world constraints such as latency and data limitations. |
| title | Generative or Discriminative? Revisiting Text Classification in the Era of Transformers |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2506.12181 |