Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.04188 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916672427786240 |
|---|---|
| author | Xian, R. Patrick Cui, Qiming Bauer, Stefan Abbasi-Asl, Reza |
| author_facet | Xian, R. Patrick Cui, Qiming Bauer, Stefan Abbasi-Asl, Reza |
| contents | Temporal progression is an integral part of knowledge accumulation and update. Web search is frequently adopted as grounding for agent knowledge, yet an improper configuration affects the quality of the agent's responses. Here, we assess the agent behavior using distinct date-controlled tools (DCTs) as stress test to measure the knowledge variability of large language model (LLM) agents. We demonstrate the temporal effects of an LLM agent as a writing assistant, which uses web search to complete scientific publication abstracts. We show that the temporality of search engine translates into tool-dependent agent performance but can be alleviated with base model choice and explicit reasoning instructions such as chain-of-thought prompting. Our results indicate that agent design and evaluations should take a dynamical view and implement measures to account for the temporal influence of external resources to ensure reliability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_04188 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Measuring temporal effects of agent knowledge by date-controlled tool use Xian, R. Patrick Cui, Qiming Bauer, Stefan Abbasi-Asl, Reza Computation and Language Information Retrieval Temporal progression is an integral part of knowledge accumulation and update. Web search is frequently adopted as grounding for agent knowledge, yet an improper configuration affects the quality of the agent's responses. Here, we assess the agent behavior using distinct date-controlled tools (DCTs) as stress test to measure the knowledge variability of large language model (LLM) agents. We demonstrate the temporal effects of an LLM agent as a writing assistant, which uses web search to complete scientific publication abstracts. We show that the temporality of search engine translates into tool-dependent agent performance but can be alleviated with base model choice and explicit reasoning instructions such as chain-of-thought prompting. Our results indicate that agent design and evaluations should take a dynamical view and implement measures to account for the temporal influence of external resources to ensure reliability. |
| title | Measuring temporal effects of agent knowledge by date-controlled tool use |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2503.04188 |