SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918129956814848 |
|---|---|
| author | Chhikara, Garima Ghosh, Kripabandhu Chakraborty, Abhijnan |
| author_facet | Chhikara, Garima Ghosh, Kripabandhu Chakraborty, Abhijnan |
| contents | Large Language Models (LLMs) are widely used for downstream tasks such as tabular classification, where ensuring fairness in their outputs is critical for inclusivity, equal representation, and responsible AI deployment. This study introduces a novel approach to enhancing LLM performance and fairness through the concept of a dynamic validation set, which evolves alongside the test set, replacing the traditional static validation approach. We also propose an iterative algorithm, SMITE, to select optimal in-context examples, with each example set validated against its corresponding dynamic validation set. The in-context set with the lowest total error is used as the final demonstration set. Our experiments across four different LLMs show that our proposed techniques significantly improve both predictive accuracy and fairness compared to baseline methods. To our knowledge, this is the first study to apply dynamic validation in the context of in-context learning for LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_17735 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation Chhikara, Garima Ghosh, Kripabandhu Chakraborty, Abhijnan Computation and Language Large Language Models (LLMs) are widely used for downstream tasks such as tabular classification, where ensuring fairness in their outputs is critical for inclusivity, equal representation, and responsible AI deployment. This study introduces a novel approach to enhancing LLM performance and fairness through the concept of a dynamic validation set, which evolves alongside the test set, replacing the traditional static validation approach. We also propose an iterative algorithm, SMITE, to select optimal in-context examples, with each example set validated against its corresponding dynamic validation set. The in-context set with the lowest total error is used as the final demonstration set. Our experiments across four different LLMs show that our proposed techniques significantly improve both predictive accuracy and fairness compared to baseline methods. To our knowledge, this is the first study to apply dynamic validation in the context of in-context learning for LLMs. |
| title | SMITE: Enhancing Fairness in LLMs through Optimal In-Context Example Selection via Dynamic Validation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2508.17735 |