Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2023
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929246154260480 |
|---|---|
| author | Wang, Zengzhi Xie, Qiming Feng, Yi Ding, Zixiang Yang, Zinong Xia, Rui |
| author_facet | Wang, Zengzhi Xie, Qiming Feng, Yi Ding, Zixiang Yang, Zinong Xia, Rui |
| contents | Recently, ChatGPT has drawn great attention from both the research community and the public. We are particularly interested in whether it can serve as a universal sentiment analyzer. To this end, in this work, we provide a preliminary evaluation of ChatGPT on the understanding of \emph{opinions}, \emph{sentiments}, and \emph{emotions} contained in the text. Specifically, we evaluate it in three settings, including \emph{standard} evaluation, \emph{polarity shift} evaluation and \emph{open-domain} evaluation. We conduct an evaluation on 7 representative sentiment analysis tasks covering 17 benchmark datasets and compare ChatGPT with fine-tuned BERT and corresponding state-of-the-art (SOTA) models on them. We also attempt several popular prompting techniques to elicit the ability further. Moreover, we conduct human evaluation and present some qualitative case studies to gain a deep comprehension of its sentiment analysis capabilities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2304_04339 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study Wang, Zengzhi Xie, Qiming Feng, Yi Ding, Zixiang Yang, Zinong Xia, Rui Computation and Language Artificial Intelligence Recently, ChatGPT has drawn great attention from both the research community and the public. We are particularly interested in whether it can serve as a universal sentiment analyzer. To this end, in this work, we provide a preliminary evaluation of ChatGPT on the understanding of \emph{opinions}, \emph{sentiments}, and \emph{emotions} contained in the text. Specifically, we evaluate it in three settings, including \emph{standard} evaluation, \emph{polarity shift} evaluation and \emph{open-domain} evaluation. We conduct an evaluation on 7 representative sentiment analysis tasks covering 17 benchmark datasets and compare ChatGPT with fine-tuned BERT and corresponding state-of-the-art (SOTA) models on them. We also attempt several popular prompting techniques to elicit the ability further. Moreover, we conduct human evaluation and present some qualitative case studies to gain a deep comprehension of its sentiment analysis capabilities. |
| title | Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2304.04339 |