Understanding and Evaluating Trust in Generative AI and Large Language Models for Spreadsheets
Fuente:
arXiv
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916569340182528 |
|---|---|
| author | Thorne, Simon |
| author_facet | Thorne, Simon |
| contents | Generative AI and Large Language Models (LLMs) hold promise for automating spreadsheet formula creation. However, due to hallucinations, bias and variable user skill, outputs obtained from generative AI cannot be assumed to be accurate or trustworthy. To address these challenges, a trustworthiness framework is proposed based on evaluating the transparency and dependability of the formula. The transparency of the formula is explored through explainability (understanding the formula's reasoning) and visibility (inspecting the underlying algorithms). The dependability of the generated formula is evaluated in terms of reliability (consistency and accuracy) and ethical considerations (bias and fairness). The paper also examines the drivers to these metrics in the form of hallucinations, training data bias and poorly constructed prompts. Finally, examples of mistrust in technology are considered and the consequences explored. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_14062 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Understanding and Evaluating Trust in Generative AI and Large Language Models for Spreadsheets Thorne, Simon Human-Computer Interaction Computers and Society Generative AI and Large Language Models (LLMs) hold promise for automating spreadsheet formula creation. However, due to hallucinations, bias and variable user skill, outputs obtained from generative AI cannot be assumed to be accurate or trustworthy. To address these challenges, a trustworthiness framework is proposed based on evaluating the transparency and dependability of the formula. The transparency of the formula is explored through explainability (understanding the formula's reasoning) and visibility (inspecting the underlying algorithms). The dependability of the generated formula is evaluated in terms of reliability (consistency and accuracy) and ethical considerations (bias and fairness). The paper also examines the drivers to these metrics in the form of hallucinations, training data bias and poorly constructed prompts. Finally, examples of mistrust in technology are considered and the consequences explored. |
| title | Understanding and Evaluating Trust in Generative AI and Large Language Models for Spreadsheets |
| topic | Human-Computer Interaction Computers and Society |
| url | https://arxiv.org/abs/2412.14062 |