Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909736591425536 |
|---|---|
| author | Sahai, Sattvik Goyal, Prasoon Johnston, Michael Gottardi, Anna Lu, Yao Hu, Lucy Dai, Luke Liu, Shaohua Sagi, Samyuth Shi, Hangjie Zhang, Desheng Vaz, Lavina Ball, Leslie Murray, Maureen Gupta, Rahul Ananthakrishna, Shankar |
| author_facet | Sahai, Sattvik Goyal, Prasoon Johnston, Michael Gottardi, Anna Lu, Yao Hu, Lucy Dai, Luke Liu, Shaohua Sagi, Samyuth Shi, Hangjie Zhang, Desheng Vaz, Lavina Ball, Leslie Murray, Maureen Gupta, Rahul Ananthakrishna, Shankar |
| contents | AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In the challenge, five teams focus on developing automated red teaming bots, while the other five create safe AI assistants. This challenge provides teams with a unique platform to evaluate automated red-teaming and safety alignment methods through head-to-head adversarial tournaments where red teams have multi-turn conversations with the competing AI coding assistants to test their safety alignment. Along with this, the challenge provides teams with a feed of high quality annotated data to fuel iterative improvement. Throughout the challenge, teams developed state-of-the-art techniques, introducing novel approaches in reasoning-based safety alignment, robust model guardrails, multi-turn jail-breaking, and efficient probing of large language models (LLMs). To support these efforts, the Amazon Nova AI Challenge team made substantial scientific and engineering investments, including building a custom baseline coding specialist model for the challenge from scratch, developing a tournament orchestration service, and creating an evaluation harness. This paper outlines the advancements made by university teams and the Amazon Nova AI Challenge team in addressing the safety challenges of AI for software development, highlighting this collaborative effort to raise the bar for AI safety. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_10108 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development Sahai, Sattvik Goyal, Prasoon Johnston, Michael Gottardi, Anna Lu, Yao Hu, Lucy Dai, Luke Liu, Shaohua Sagi, Samyuth Shi, Hangjie Zhang, Desheng Vaz, Lavina Ball, Leslie Murray, Maureen Gupta, Rahul Ananthakrishna, Shankar Artificial Intelligence Computation and Language I.2.7; I.2.6; E.0 AI systems for software development are rapidly gaining prominence, yet significant challenges remain in ensuring their safety. To address this, Amazon launched the Trusted AI track of the Amazon Nova AI Challenge, a global competition among 10 university teams to drive advances in secure AI. In the challenge, five teams focus on developing automated red teaming bots, while the other five create safe AI assistants. This challenge provides teams with a unique platform to evaluate automated red-teaming and safety alignment methods through head-to-head adversarial tournaments where red teams have multi-turn conversations with the competing AI coding assistants to test their safety alignment. Along with this, the challenge provides teams with a feed of high quality annotated data to fuel iterative improvement. Throughout the challenge, teams developed state-of-the-art techniques, introducing novel approaches in reasoning-based safety alignment, robust model guardrails, multi-turn jail-breaking, and efficient probing of large language models (LLMs). To support these efforts, the Amazon Nova AI Challenge team made substantial scientific and engineering investments, including building a custom baseline coding specialist model for the challenge from scratch, developing a tournament orchestration service, and creating an evaluation harness. This paper outlines the advancements made by university teams and the Amazon Nova AI Challenge team in addressing the safety challenges of AI for software development, highlighting this collaborative effort to raise the bar for AI safety. |
| title | Amazon Nova AI Challenge -- Trusted AI: Advancing secure, AI-assisted software development |
| topic | Artificial Intelligence Computation and Language I.2.7; I.2.6; E.0 |
| url | https://arxiv.org/abs/2508.10108 |