Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866917725411999744 |
|---|---|
| author | Kenthapadi, Krishnaram Sameki, Mehrnoosh Taly, Ankur |
| author_facet | Kenthapadi, Krishnaram Sameki, Mehrnoosh Taly, Ankur |
| contents | With the ongoing rapid adoption of Artificial Intelligence (AI)-based systems in high-stakes domains, ensuring the trustworthiness, safety, and observability of these systems has become crucial. It is essential to evaluate and monitor AI systems not only for accuracy and quality-related metrics but also for robustness, bias, security, interpretability, and other responsible AI dimensions. We focus on large language models (LLMs) and other generative AI models, which present additional challenges such as hallucinations, harmful and manipulative content, and copyright infringement. In this survey article accompanying our KDD 2024 tutorial, we highlight a wide range of harms associated with generative AI systems, and survey state of the art approaches (along with open challenges) to address these harms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_12858 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) Kenthapadi, Krishnaram Sameki, Mehrnoosh Taly, Ankur Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning With the ongoing rapid adoption of Artificial Intelligence (AI)-based systems in high-stakes domains, ensuring the trustworthiness, safety, and observability of these systems has become crucial. It is essential to evaluate and monitor AI systems not only for accuracy and quality-related metrics but also for robustness, bias, security, interpretability, and other responsible AI dimensions. We focus on large language models (LLMs) and other generative AI models, which present additional challenges such as hallucinations, harmful and manipulative content, and copyright infringement. In this survey article accompanying our KDD 2024 tutorial, we highlight a wide range of harms associated with generative AI systems, and survey state of the art approaches (along with open challenges) to address these harms. |
| title | Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2407.12858 |