Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete?
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913388097961984 |
|---|---|
| author | Delile, Zack Radel, Sean Godinez, Joe Engstrom, Garrett Brucker, Theo Young, Kenzie Ghanavati, Sepideh |
| author_facet | Delile, Zack Radel, Sean Godinez, Joe Engstrom, Garrett Brucker, Theo Young, Kenzie Ghanavati, Sepideh |
| contents | Stack Overflow and other similar forums are used commonly by developers to seek answers for their software development as well as privacy-related concerns. Recently, ChatGPT has been used as an alternative to generate code or produce responses to developers' questions. In this paper, we aim to understand developers' privacy challenges by evaluating the types of privacy-related questions asked on Stack Overflow. We then conduct a comparative analysis between the accepted responses given by Stack Overflow users and the responses produced by ChatGPT for those extracted questions to identify if ChatGPT could serve as a viable alternative. Our results show that most privacy-related questions are related to choice/consent, aggregation, and identification. Furthermore, our findings illustrate that ChatGPT generates similarly correct responses for about 56% of questions, while for the rest of the responses, the answers from Stack Overflow are slightly more accurate than ChatGPT. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2306_11174 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete? Delile, Zack Radel, Sean Godinez, Joe Engstrom, Garrett Brucker, Theo Young, Kenzie Ghanavati, Sepideh Software Engineering Artificial Intelligence Computation and Language Stack Overflow and other similar forums are used commonly by developers to seek answers for their software development as well as privacy-related concerns. Recently, ChatGPT has been used as an alternative to generate code or produce responses to developers' questions. In this paper, we aim to understand developers' privacy challenges by evaluating the types of privacy-related questions asked on Stack Overflow. We then conduct a comparative analysis between the accepted responses given by Stack Overflow users and the responses produced by ChatGPT for those extracted questions to identify if ChatGPT could serve as a viable alternative. Our results show that most privacy-related questions are related to choice/consent, aggregation, and identification. Furthermore, our findings illustrate that ChatGPT generates similarly correct responses for about 56% of questions, while for the rest of the responses, the answers from Stack Overflow are slightly more accurate than ChatGPT. |
| title | Evaluating Privacy Questions From Stack Overflow: Can ChatGPT Compete? |
| topic | Software Engineering Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2306.11174 |