Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2510.06093 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909829804589056 |
|---|---|
| author | Mainali, Mallika Sureshbabu, Harsha Sen, Anik Rauch, Christopher B. Reifsnyder, Noah D. Meyer, John Turner, J. T. Floyd, Michael W. Molineaux, Matthew Weber, Rosina O. |
| author_facet | Mainali, Mallika Sureshbabu, Harsha Sen, Anik Rauch, Christopher B. Reifsnyder, Noah D. Meyer, John Turner, J. T. Floyd, Michael W. Molineaux, Matthew Weber, Rosina O. |
| contents | As algorithmic decision-makers are increasingly applied to high-stakes domains, AI alignment research has evolved from a focus on universal value alignment to context-specific approaches that account for decision-maker attributes. Prior work on Decision-Maker Alignment (DMA) has explored two primary strategies: (1) classical AI methods integrating case-based reasoning, Bayesian reasoning, and naturalistic decision-making, and (2) large language model (LLM)-based methods leveraging prompt engineering. While both approaches have shown promise in limited domains such as medical triage, their generalizability to novel contexts remains underexplored. In this work, we implement a prior classical AI model and develop an LLM-based algorithmic decision-maker evaluated using a large reasoning model (GPT-5) and a non-reasoning model (GPT-4) with weighted self-consistency under a zero-shot prompting framework, as proposed in recent literature. We evaluate both approaches on a health insurance decision-making dataset annotated for three target decision-makers with varying levels of risk tolerance (0.0, 0.5, 1.0). In the experiments reported herein, classical AI and LLM-based models achieved comparable alignment with attribute-based targets, with classical AI exhibiting slightly better alignment for a moderate risk profile. The dataset and open-source implementation are publicly available at: https://github.com/TeX-Base/ClassicalAIvsLLMsforDMAlignment and https://github.com/Parallax-Advanced-Research/ITM/tree/feature_insurance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_06093 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices Mainali, Mallika Sureshbabu, Harsha Sen, Anik Rauch, Christopher B. Reifsnyder, Noah D. Meyer, John Turner, J. T. Floyd, Michael W. Molineaux, Matthew Weber, Rosina O. Artificial Intelligence As algorithmic decision-makers are increasingly applied to high-stakes domains, AI alignment research has evolved from a focus on universal value alignment to context-specific approaches that account for decision-maker attributes. Prior work on Decision-Maker Alignment (DMA) has explored two primary strategies: (1) classical AI methods integrating case-based reasoning, Bayesian reasoning, and naturalistic decision-making, and (2) large language model (LLM)-based methods leveraging prompt engineering. While both approaches have shown promise in limited domains such as medical triage, their generalizability to novel contexts remains underexplored. In this work, we implement a prior classical AI model and develop an LLM-based algorithmic decision-maker evaluated using a large reasoning model (GPT-5) and a non-reasoning model (GPT-4) with weighted self-consistency under a zero-shot prompting framework, as proposed in recent literature. We evaluate both approaches on a health insurance decision-making dataset annotated for three target decision-makers with varying levels of risk tolerance (0.0, 0.5, 1.0). In the experiments reported herein, classical AI and LLM-based models achieved comparable alignment with attribute-based targets, with classical AI exhibiting slightly better alignment for a moderate risk profile. The dataset and open-source implementation are publicly available at: https://github.com/TeX-Base/ClassicalAIvsLLMsforDMAlignment and https://github.com/Parallax-Advanced-Research/ITM/tree/feature_insurance. |
| title | Classical AI vs. LLMs for Decision-Maker Alignment in Health Insurance Choices |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.06093 |