Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913196621692928 |
|---|---|
| author | Ouyang, Tinghui MaungMaung, AprilPyone Konishi, Koichi Seo, Yoshiki Echizen, Isao |
| author_facet | Ouyang, Tinghui MaungMaung, AprilPyone Konishi, Koichi Seo, Yoshiki Echizen, Isao |
| contents | In the era of large AI models, the complex architecture and vast parameters present substantial challenges for effective AI quality management (AIQM), e.g. large language model (LLM). This paper focuses on investigating the quality assurance of a specific LLM-based AI product--a ChatGPT-based sentiment analysis system. The study delves into stability issues related to both the operation and robustness of the expansive AI model on which ChatGPT is based. Experimental analysis is conducted using benchmark datasets for sentiment analysis. The results reveal that the constructed ChatGPT-based sentiment analysis system exhibits uncertainty, which is attributed to various operational factors. It demonstrated that the system also exhibits stability issues in handling conventional small text attacks involving robustness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_07441 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance Ouyang, Tinghui MaungMaung, AprilPyone Konishi, Koichi Seo, Yoshiki Echizen, Isao Computation and Language In the era of large AI models, the complex architecture and vast parameters present substantial challenges for effective AI quality management (AIQM), e.g. large language model (LLM). This paper focuses on investigating the quality assurance of a specific LLM-based AI product--a ChatGPT-based sentiment analysis system. The study delves into stability issues related to both the operation and robustness of the expansive AI model on which ChatGPT is based. Experimental analysis is conducted using benchmark datasets for sentiment analysis. The results reveal that the constructed ChatGPT-based sentiment analysis system exhibits uncertainty, which is attributed to various operational factors. It demonstrated that the system also exhibits stability issues in handling conventional small text attacks involving robustness. |
| title | Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2401.07441 |