Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.18882 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914579062194176 |
|---|---|
| author | Shi, Wei Peng, Ziheng Li, Sihang Wang, Xiting Wang, Xiang Du, Mengnan Zou, Na |
| author_facet | Shi, Wei Peng, Ziheng Li, Sihang Wang, Xiting Wang, Xiang Du, Mengnan Zou, Na |
| contents | LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%-70% range. We trace this to an Intrinsic Bias Hypothesis (IBH): the call/no-call decision mapping carries an activation-independent call offset, so the model favors call even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the call/no_call decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when no_call activation outweighs call activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering (AMCS), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. Code is available at https://github.com/SKURA502/agent-sae/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18882 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents Shi, Wei Peng, Ziheng Li, Sihang Wang, Xiting Wang, Xiang Du, Mengnan Zou, Na Machine Learning Artificial Intelligence LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%-70% range. We trace this to an Intrinsic Bias Hypothesis (IBH): the call/no-call decision mapping carries an activation-independent call offset, so the model favors call even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the call/no_call decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when no_call activation outweighs call activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering (AMCS), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. Code is available at https://github.com/SKURA502/agent-sae/. |
| title | To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.18882 |