Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.06979 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908640680607744 |
|---|---|
| author | Lv, Xinpeng Mao, Yunxin Li, Haoxuan Liang, Ke Yang, Jinxuan Huang, Wanrong Chi, Haoang Chen, Huan Lan, Long Chen, Yuanlong Yang, Wenjing Wang, Haotian |
| author_facet | Lv, Xinpeng Mao, Yunxin Li, Haoxuan Liang, Ke Yang, Jinxuan Huang, Wanrong Chi, Haoang Chen, Huan Lan, Long Chen, Yuanlong Yang, Wenjing Wang, Haotian |
| contents | Strategic classification~(SC) explores how individuals or entities modify their features strategically to achieve favorable classification outcomes. However, existing SC methods, which are largely based on linear models or shallow neural networks, face significant limitations in terms of scalability and capacity when applied to real-world datasets with significantly increasing scale, especially in financial services and the internet sector. In this paper, we investigate how to leverage large language models to design a more scalable and efficient SC framework, especially in the case of growing individuals engaged with decision-making processes. Specifically, we introduce GLIM, a gradient-free SC method grounded in in-context learning. During the feed-forward process of self-attention, GLIM implicitly simulates the typical bi-level optimization process of SC, including both the feature manipulation and decision rule optimization. Without fine-tuning the LLMs, our proposed GLIM enjoys the advantage of cost-effective adaptation in dynamic strategic environments. Theoretically, we prove GLIM can support pre-trained LLMs to adapt to a broad range of strategic manipulations. We validate our approach through experiments with a collection of pre-trained LLMs on real-world and synthetic datasets in financial and internet domains, demonstrating that our GLIM exhibits both robustness and efficiency, and offering an effective solution for large-scale SC tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_06979 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Breaking the Gradient Barrier: Unveiling Large Language Models for Strategic Classification Lv, Xinpeng Mao, Yunxin Li, Haoxuan Liang, Ke Yang, Jinxuan Huang, Wanrong Chi, Haoang Chen, Huan Lan, Long Chen, Yuanlong Yang, Wenjing Wang, Haotian Machine Learning Computer Science and Game Theory Strategic classification~(SC) explores how individuals or entities modify their features strategically to achieve favorable classification outcomes. However, existing SC methods, which are largely based on linear models or shallow neural networks, face significant limitations in terms of scalability and capacity when applied to real-world datasets with significantly increasing scale, especially in financial services and the internet sector. In this paper, we investigate how to leverage large language models to design a more scalable and efficient SC framework, especially in the case of growing individuals engaged with decision-making processes. Specifically, we introduce GLIM, a gradient-free SC method grounded in in-context learning. During the feed-forward process of self-attention, GLIM implicitly simulates the typical bi-level optimization process of SC, including both the feature manipulation and decision rule optimization. Without fine-tuning the LLMs, our proposed GLIM enjoys the advantage of cost-effective adaptation in dynamic strategic environments. Theoretically, we prove GLIM can support pre-trained LLMs to adapt to a broad range of strategic manipulations. We validate our approach through experiments with a collection of pre-trained LLMs on real-world and synthetic datasets in financial and internet domains, demonstrating that our GLIM exhibits both robustness and efficiency, and offering an effective solution for large-scale SC tasks. |
| title | Breaking the Gradient Barrier: Unveiling Large Language Models for Strategic Classification |
| topic | Machine Learning Computer Science and Game Theory |
| url | https://arxiv.org/abs/2511.06979 |