Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913929273278464 |
|---|---|
| author | Chawla, Divij Bhutada, Ashita Anh, Do Duc Raghunathan, Abhinav SP, Vinod Guo, Cathy Liew, Dar Win Gupta, Prannaya Bhardwaj, Rishabh Bhardwaj, Rajat Poria, Soujanya |
| author_facet | Chawla, Divij Bhutada, Ashita Anh, Do Duc Raghunathan, Abhinav SP, Vinod Guo, Cathy Liew, Dar Win Gupta, Prannaya Bhardwaj, Rishabh Bhardwaj, Rajat Poria, Soujanya |
| contents | We assess whether AI systems can credibly evaluate investment risk appetite-a task that must be thoroughly validated before automation. Our analysis was conducted on proprietary systems (GPT, Claude, Gemini) and open-weight models (LLaMA, DeepSeek, Mistral), using carefully curated user profiles that reflect real users with varying attributes such as country and gender. As a result, the models exhibit significant variance in score distributions when user attributes-such as country or gender-that should not influence risk computation are changed. For example, GPT-4o assigns higher risk scores to Nigerian and Indonesian profiles. While some models align closely with expected scores in the Low- and Mid-risk ranges, none maintain consistent scores across regions and demographics, thereby violating AI and finance regulations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_18953 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk? Chawla, Divij Bhutada, Ashita Anh, Do Duc Raghunathan, Abhinav SP, Vinod Guo, Cathy Liew, Dar Win Gupta, Prannaya Bhardwaj, Rishabh Bhardwaj, Rajat Poria, Soujanya Computation and Language We assess whether AI systems can credibly evaluate investment risk appetite-a task that must be thoroughly validated before automation. Our analysis was conducted on proprietary systems (GPT, Claude, Gemini) and open-weight models (LLaMA, DeepSeek, Mistral), using carefully curated user profiles that reflect real users with varying attributes such as country and gender. As a result, the models exhibit significant variance in score distributions when user attributes-such as country or gender-that should not influence risk computation are changed. For example, GPT-4o assigns higher risk scores to Nigerian and Indonesian profiles. While some models align closely with expected scores in the Low- and Mid-risk ranges, none maintain consistent scores across regions and demographics, thereby violating AI and finance regulations. |
| title | Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk? |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.18953 |