Understanding and Mitigating Risks of Generative AI in Financial Services
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908342484467712 |
|---|---|
| author | Gehrmann, Sebastian Huang, Claire Teng, Xian Yurovski, Sergei Shode, Iyanuoluwa Patel, Chirag S. Bhorkar, Arjun Thomas, Naveen Doucette, John Rosenberg, David Dredze, Mark Rabinowitz, David |
| author_facet | Gehrmann, Sebastian Huang, Claire Teng, Xian Yurovski, Sergei Shode, Iyanuoluwa Patel, Chirag S. Bhorkar, Arjun Thomas, Naveen Doucette, John Rosenberg, David Dredze, Mark Rabinowitz, David |
| contents | To responsibly develop Generative AI (GenAI) products, it is critical to define the scope of acceptable inputs and outputs. What constitutes a "safe" response is an actively debated question. Academic work puts an outsized focus on evaluating models by themselves for general purpose aspects such as toxicity, bias, and fairness, especially in conversational applications being used by a broad audience. In contrast, less focus is put on considering sociotechnical systems in specialized domains. Yet, those specialized systems can be subject to extensive and well-understood legal and regulatory scrutiny. These product-specific considerations need to be set in industry-specific laws, regulations, and corporate governance requirements. In this paper, we aim to highlight AI content safety considerations specific to the financial services domain and outline an associated AI content risk taxonomy. We compare this taxonomy to existing work in this space and discuss implications of risk category violations on various stakeholders. We evaluate how existing open-source technical guardrail solutions cover this taxonomy by assessing them on data collected via red-teaming activities. Our results demonstrate that these guardrails fail to detect most of the content risks we discuss. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20086 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Understanding and Mitigating Risks of Generative AI in Financial Services Gehrmann, Sebastian Huang, Claire Teng, Xian Yurovski, Sergei Shode, Iyanuoluwa Patel, Chirag S. Bhorkar, Arjun Thomas, Naveen Doucette, John Rosenberg, David Dredze, Mark Rabinowitz, David Computation and Language Artificial Intelligence Computers and Society Machine Learning To responsibly develop Generative AI (GenAI) products, it is critical to define the scope of acceptable inputs and outputs. What constitutes a "safe" response is an actively debated question. Academic work puts an outsized focus on evaluating models by themselves for general purpose aspects such as toxicity, bias, and fairness, especially in conversational applications being used by a broad audience. In contrast, less focus is put on considering sociotechnical systems in specialized domains. Yet, those specialized systems can be subject to extensive and well-understood legal and regulatory scrutiny. These product-specific considerations need to be set in industry-specific laws, regulations, and corporate governance requirements. In this paper, we aim to highlight AI content safety considerations specific to the financial services domain and outline an associated AI content risk taxonomy. We compare this taxonomy to existing work in this space and discuss implications of risk category violations on various stakeholders. We evaluate how existing open-source technical guardrail solutions cover this taxonomy by assessing them on data collected via red-teaming activities. Our results demonstrate that these guardrails fail to detect most of the content risks we discuss. |
| title | Understanding and Mitigating Risks of Generative AI in Financial Services |
| topic | Computation and Language Artificial Intelligence Computers and Society Machine Learning |
| url | https://arxiv.org/abs/2504.20086 |