Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.18423 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911694290157568 |
|---|---|
| author | Diller, Jonathan Barnes, David Bogdanoff, Rebekah Collier, Rhett Collins, Roddy Fieldhouse, Keith Gefen, Yonatan Johnson, Cameron Kodali, Anuriha Kriel, Brad Murali, Varun Niehaus, James Sukharev, Mish VanPelt, Joseph Hoogs, Anthony Kumar, Vijay Basharat, Arslan |
| author_facet | Diller, Jonathan Barnes, David Bogdanoff, Rebekah Collier, Rhett Collins, Roddy Fieldhouse, Keith Gefen, Yonatan Johnson, Cameron Kodali, Anuriha Kriel, Brad Murali, Varun Niehaus, James Sukharev, Mish VanPelt, Joseph Hoogs, Anthony Kumar, Vijay Basharat, Arslan |
| contents | As autonomous systems grow more advanced, objective metrics to evaluate their ethical and legal compliance are critical for informing end users of their limitations and ensuring accountability of those who misuse them. Current ethical embodied AI frameworks remain mostly qualitative, focusing on system design (through safety guardrails or targeted red teaming), and the realized guardrails often directly disallow unsafe behavior without providing the user with an override or interpretable reason. Instead, there is a need for computable metrics through rigorous testing that allow a user to determine the applicability of the system to the task. To address this gap, we introduce the Reference Ethical Benchmark for Autonomy Readiness (REBAR), a quantitative test and evaluation framework for autonomous systems. REBAR maps operating metrics into a computable Autonomy Readiness Level (ARL) rubric that can quantify ethical performance. Key innovations of the framework include a neuro-symbolic Large Language Model (LLM) approach to calculate and explain the ethical difficulty of scenarios, LLM-driven at-scale generation of test instances, and a versatile, photorealistic simulation environment. By evaluating white-box autonomy solutions through this rigorous testing pipeline, REBAR delivers an objective and repeatable benchmark score, bridging the gap between abstract principles and verifiable, accountable autonomy. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_18423 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | REBAR: Reference Ethical Benchmark for Autonomy Readiness Diller, Jonathan Barnes, David Bogdanoff, Rebekah Collier, Rhett Collins, Roddy Fieldhouse, Keith Gefen, Yonatan Johnson, Cameron Kodali, Anuriha Kriel, Brad Murali, Varun Niehaus, James Sukharev, Mish VanPelt, Joseph Hoogs, Anthony Kumar, Vijay Basharat, Arslan Robotics Computers and Society As autonomous systems grow more advanced, objective metrics to evaluate their ethical and legal compliance are critical for informing end users of their limitations and ensuring accountability of those who misuse them. Current ethical embodied AI frameworks remain mostly qualitative, focusing on system design (through safety guardrails or targeted red teaming), and the realized guardrails often directly disallow unsafe behavior without providing the user with an override or interpretable reason. Instead, there is a need for computable metrics through rigorous testing that allow a user to determine the applicability of the system to the task. To address this gap, we introduce the Reference Ethical Benchmark for Autonomy Readiness (REBAR), a quantitative test and evaluation framework for autonomous systems. REBAR maps operating metrics into a computable Autonomy Readiness Level (ARL) rubric that can quantify ethical performance. Key innovations of the framework include a neuro-symbolic Large Language Model (LLM) approach to calculate and explain the ethical difficulty of scenarios, LLM-driven at-scale generation of test instances, and a versatile, photorealistic simulation environment. By evaluating white-box autonomy solutions through this rigorous testing pipeline, REBAR delivers an objective and repeatable benchmark score, bridging the gap between abstract principles and verifiable, accountable autonomy. |
| title | REBAR: Reference Ethical Benchmark for Autonomy Readiness |
| topic | Robotics Computers and Society |
| url | https://arxiv.org/abs/2605.18423 |