AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910204562505728 |
|---|---|
| author | Mazumder, Aritra Dipta, Shubhashis Roy Lia, Nusrat Jahan Khan, Tanzila Hossain, Kainat Raisa Shri, Nehaa Debsarkar, Shubhrangshu Tasnim, Humayra Shawon, Gour Gupal Talukder Mitra, Debjoty Rani, Sumaiya Ahmed Anik, Al Jami Islam Khan, Al Nafeu |
| author_facet | Mazumder, Aritra Dipta, Shubhashis Roy Lia, Nusrat Jahan Khan, Tanzila Hossain, Kainat Raisa Shri, Nehaa Debsarkar, Shubhrangshu Tasnim, Humayra Shawon, Gour Gupal Talukder Mitra, Debjoty Rani, Sumaiya Ahmed Anik, Al Jami Islam Khan, Al Nafeu |
| contents | Multi-agent systems achieve state-of-the-art outcomes through peer collaboration. However, when an agent in the pipeline silently drops a constraint, the system's final output may look correct even though the reasoning chain was quietly corrupted, and existing outcome-based evaluations are blind to such multi-hop process failures. To make these vulnerabilities measurable before deployment, we introduce AgentCollabBench, a diagnostic benchmark of 900 human-validated tasks spanning software engineering, DevOps, and data engineering. Each task isolates one of four behavioral risks: instruction decay (does a constraint survive peer pressure?), false-belief contagion (does a falsehood spread through consensus?), context leakage (does information bleed between tasks?), and tracer durability (does marked data reach the final agent?). Evaluating four modern LLMs (GPT 4.1 mini, Gemini 2.5 Flash Lite, Qwen-3.5-35B-A3B, and Llama 3.1 8B Instruct), we expose model-specific vulnerability profiles invisible to outcome-only evaluation; Qwen-3.5-35B-A3B, for example, leads on tracer durability and instruction stability, while GPT 4.1 mini leads on leakage containment and false-belief resistance. Beyond per-model differences, communication topology emerges as a primary risk factor that explains 7-40% of the variance in multi-hop information survival. The effect traces to a synthesis bottleneck specific to converging-DAG nodes: an agent weighing competing parent inputs discards constraints carried by a minority branch, a bottleneck structurally absent from linear chains. AgentCollabBench demonstrates that suboptimal topology can silently erase the safeguards of highly capable models, arguing that multi-agent reliability is fundamentally a structural problem and that scaling model intelligence alone is no substitute for architecture. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_08647 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators Mazumder, Aritra Dipta, Shubhashis Roy Lia, Nusrat Jahan Khan, Tanzila Hossain, Kainat Raisa Shri, Nehaa Debsarkar, Shubhrangshu Tasnim, Humayra Shawon, Gour Gupal Talukder Mitra, Debjoty Rani, Sumaiya Ahmed Anik, Al Jami Islam Khan, Al Nafeu Computation and Language Artificial Intelligence Machine Learning Multi-agent systems achieve state-of-the-art outcomes through peer collaboration. However, when an agent in the pipeline silently drops a constraint, the system's final output may look correct even though the reasoning chain was quietly corrupted, and existing outcome-based evaluations are blind to such multi-hop process failures. To make these vulnerabilities measurable before deployment, we introduce AgentCollabBench, a diagnostic benchmark of 900 human-validated tasks spanning software engineering, DevOps, and data engineering. Each task isolates one of four behavioral risks: instruction decay (does a constraint survive peer pressure?), false-belief contagion (does a falsehood spread through consensus?), context leakage (does information bleed between tasks?), and tracer durability (does marked data reach the final agent?). Evaluating four modern LLMs (GPT 4.1 mini, Gemini 2.5 Flash Lite, Qwen-3.5-35B-A3B, and Llama 3.1 8B Instruct), we expose model-specific vulnerability profiles invisible to outcome-only evaluation; Qwen-3.5-35B-A3B, for example, leads on tracer durability and instruction stability, while GPT 4.1 mini leads on leakage containment and false-belief resistance. Beyond per-model differences, communication topology emerges as a primary risk factor that explains 7-40% of the variance in multi-hop information survival. The effect traces to a synthesis bottleneck specific to converging-DAG nodes: an agent weighing competing parent inputs discards constraints carried by a minority branch, a bottleneck structurally absent from linear chains. AgentCollabBench demonstrates that suboptimal topology can silently erase the safeguards of highly capable models, arguing that multi-agent reliability is fundamentally a structural problem and that scaling model intelligence alone is no substitute for architecture. |
| title | AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2605.08647 |