Improving LLM Group Fairness on Tabular Data via In-Context Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916509981343744 |
|---|---|
| author | Cherepanova, Valeriia Lee, Chia-Jung Akpinar, Nil-Jana Fogliato, Riccardo Bertran, Martin Andres Kearns, Michael Zou, James |
| author_facet | Cherepanova, Valeriia Lee, Chia-Jung Akpinar, Nil-Jana Fogliato, Riccardo Bertran, Martin Andres Kearns, Michael Zou, James |
| contents | Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_04642 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Improving LLM Group Fairness on Tabular Data via In-Context Learning Cherepanova, Valeriia Lee, Chia-Jung Akpinar, Nil-Jana Fogliato, Riccardo Bertran, Martin Andres Kearns, Michael Zou, James Machine Learning Artificial Intelligence Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints. |
| title | Improving LLM Group Fairness on Tabular Data via In-Context Learning |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2412.04642 |