Cover Image

OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ding, Deming, Liu, Shichun, Yang, Enhui, Lin, Jiahang, Chen, Ziying, Dou, Shihan, Guo, Honglin, Cheng, Weiyu, Zhao, Pengyu, Xiao, Chengjun, Zeng, Qunhong, Zhang, Qi, Huang, Xuanjing, Xu, Qidi, Gui, Tao
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
by: Lin, Jiahang, et al.
Published: (2026)

EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
by: Pan, Chengjun, et al.
Published: (2026)

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development
by: Yang, Jie, et al.
Published: (2026)

OctoPack: Instruction Tuning Code Large Language Models
by: Muennighoff, Niklas, et al.
Published: (2023)

MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
by: Lin, Jiahang, et al.
Published: (2026)

OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning
by: Lu, Pan, et al.
Published: (2025)

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models
by: Lv, Huijie, et al.
Published: (2024)

A-OctoMap: An Adaptive OctoMap for Online Path Planning
by: Mao, Yihui, et al.
Published: (2024)

Improving RL Exploration for LLM Reasoning through Retrospective Replay
by: Dou, Shihan, et al.
Published: (2025)

Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric
by: Yang, Yuming, et al.
Published: (2025)

AInsteinBench: Benchmarking Coding Agents on Scientific Repositories
by: Duston, Titouan, et al.
Published: (2025)

TransferTOD: A Generalizable Chinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities
by: Zhang, Ming, et al.
Published: (2024)

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees
by: Wang, Yuhui, et al.
Published: (2026)

Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization
by: Liu, Boyang, et al.
Published: (2025)

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation
by: Wang, Peiding, et al.
Published: (2025)

RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions
by: Zhang, Yuansen, et al.
Published: (2024)

Octo: An Open-Source Generalist Robot Policy
by: Octo Model Team, et al.
Published: (2024)

OctoNav: Towards Generalist Embodied Navigation
by: Gao, Chen, et al.
Published: (2025)

EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
by: Li, Jia, et al.
Published: (2024)

AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems
by: Wang, Weiyi, et al.
Published: (2026)

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories
by: Shen, Chihao, et al.
Published: (2025)

CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks
by: Fu, Lingyue, et al.
Published: (2025)

MHRC-Bench: A Multilingual Hardware Repository-Level Code Completion benchmark
by: Zou, Qingyun, et al.
Published: (2026)

MigrationBench: Repository-Level Code Migration Benchmark from Java 8
by: Liu, Linbo, et al.
Published: (2025)

AACR-Bench: Evaluating Automatic Code Review with Holistic Repository-Level Context
by: Zhang, Lei, et al.
Published: (2026)

MRG-Bench: Evaluating and Exploring the Requirements of Context for Repository-Level Code Generation
by: Li, Haiyang
Published: (2025)

Detecting Essence Code Clones via Information Theoretic Analysis
by: Zhao, Lida, et al.
Published: (2025)

Training with Pseudo-Code for Instruction Following
by: Kumar, Prince, et al.
Published: (2025)

CIFE: Code Instruction-Following Evaluation
by: Gunnu, Sravani, et al.
Published: (2025)

OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling
by: Wang, Zengzhi, et al.
Published: (2025)

PyOcto: A high-throughput seismic phase associator
by: Münchmeyer, Jannes
Published: (2023)

Octo-planner: On-device Language Model for Planner-Action Agents
by: Chen, Wei, et al.
Published: (2024)

Compression Hacking: A Supplementary Perspective on Informatics Properties of Language Models from Geometric Distortion
by: Zang, Jianxiang, et al.
Published: (2025)

FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation
by: Li, Wei, et al.
Published: (2025)

LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation
by: Zhang, Ming, et al.
Published: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
by: Wen, Bosi, et al.
Published: (2026)

From Laboratory to Real-World Applications: Benchmarking Agentic Code Reasoning at the Repository Level
by: Li, Jia, et al.
Published: (2026)

DecIF: Improving Instruction-Following through Meta-Decomposition
by: Hui, Tingfeng, et al.
Published: (2025)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
by: Xi, Zhiheng, et al.
Published: (2024)

The Octo-Rail Lattice: a four-dimensional cluster state design
by: Østergaard, Emil E. B., et al.
Published: (2025)