Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2601.15679 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912840538914816 |
|---|---|
| author | Seah, Ee Wei Zheng, Yongsen Nikshith, Naga Morsidi, Mahran Matienzo, Gabriel Waikin Loh Gay, Nigel Vij, Akriti Chua, Benjamin Ng, En Qi Johnson, Sharmini Wilfred, Vanessa Lee, Wan Sie Davidson, Anna Devine, Catherine Zorer, Erin Holvey, Gareth Coppock, Harry Walpole, James Wynee, Jerome Dubois, Magda Schmatz, Michael Keane, Patrick Deverett, Sam Black, Bill Yan, Bo Sabir, Bushra Sun, Frank Zhang, Hao Farlow, Harriet Zhou, Helen Dong, Lingming Lu, Qinghua Jang, Seung Abuadbba, Sharif O'Callaghan, Simon Ma, Suyu Howroyd, Tom Fung, Cyrus Azadi, Fatemeh Nejadgholi, Isar Vishnubhotla, Krishnapriya Xiong, Pulei Lohrasbi, Saeedeh Buffett, Scott Iqbal, Shahrear Vajjala, Sowmya Safont-Andreu, Anna Massarelli, Luca van der Wal, Oskar Möller, Simon Delaborde, Agnes Duguépéroux, Joris Rolin, Nicolas Gallienne, Romane Behanzin, Sarah Seimandi, Tom Murakami, Akiko Semitsu, Takayuki Tsukiji, Teresa Kinuthia, Angela Michie, Michael Kasaon, Stephanie Wangari, Jean Baek, Hankyul Noh, Jaewon Nam, Kihyuk Seo, Sang Shin, Sungpil Lee, Taewhi Kim, Yongsu |
| author_facet | Seah, Ee Wei Zheng, Yongsen Nikshith, Naga Morsidi, Mahran Matienzo, Gabriel Waikin Loh Gay, Nigel Vij, Akriti Chua, Benjamin Ng, En Qi Johnson, Sharmini Wilfred, Vanessa Lee, Wan Sie Davidson, Anna Devine, Catherine Zorer, Erin Holvey, Gareth Coppock, Harry Walpole, James Wynee, Jerome Dubois, Magda Schmatz, Michael Keane, Patrick Deverett, Sam Black, Bill Yan, Bo Sabir, Bushra Sun, Frank Zhang, Hao Farlow, Harriet Zhou, Helen Dong, Lingming Lu, Qinghua Jang, Seung Abuadbba, Sharif O'Callaghan, Simon Ma, Suyu Howroyd, Tom Fung, Cyrus Azadi, Fatemeh Nejadgholi, Isar Vishnubhotla, Krishnapriya Xiong, Pulei Lohrasbi, Saeedeh Buffett, Scott Iqbal, Shahrear Vajjala, Sowmya Safont-Andreu, Anna Massarelli, Luca van der Wal, Oskar Möller, Simon Delaborde, Agnes Duguépéroux, Joris Rolin, Nicolas Gallienne, Romane Behanzin, Sarah Seimandi, Tom Murakami, Akiko Semitsu, Takayuki Tsukiji, Teresa Kinuthia, Angela Michie, Michael Kasaon, Stephanie Wangari, Jean Baek, Hankyul Noh, Jaewon Nam, Kihyuk Seo, Sang Shin, Sungpil Lee, Taewhi Kim, Yongsu |
| contents | The rapid rise of autonomous AI systems and advancements in agent capabilities are introducing new risks due to reduced oversight of real-world interactions. Yet agent testing remains nascent and is still a developing science. As AI agents begin to be deployed globally, it is important that they handle different languages and cultures accurately and securely.
To address this, participants from The International Network for Advanced AI Measurement, Evaluation and Science, including representatives from Singapore, Japan, Australia, Canada, the European Commission, France, Kenya, South Korea, and the United Kingdom have come together to align approaches to agentic evaluations.
This is the third exercise, building on insights from two earlier joint testing exercises conducted by the Network in November 2024 and February 2025. The objective is to further refine best practices for testing advanced AI systems.
The exercise was split into two strands: (1) common risks, including leakage of sensitive information and fraud, led by Singapore AISI; and (2) cybersecurity, led by UK AISI. A mix of open and closed-weight models were evaluated against tasks from various public agentic benchmarks. Given the nascency of agentic testing, our primary focus was on understanding methodological issues in conducting such tests, rather than examining test results or model capabilities. This collaboration marks an important step forward as participants work together to advance the science of agentic evaluations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_15679 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Improving Methodologies for Agentic Evaluations Across Domains: Leakage of Sensitive Information, Fraud and Cybersecurity Threats Seah, Ee Wei Zheng, Yongsen Nikshith, Naga Morsidi, Mahran Matienzo, Gabriel Waikin Loh Gay, Nigel Vij, Akriti Chua, Benjamin Ng, En Qi Johnson, Sharmini Wilfred, Vanessa Lee, Wan Sie Davidson, Anna Devine, Catherine Zorer, Erin Holvey, Gareth Coppock, Harry Walpole, James Wynee, Jerome Dubois, Magda Schmatz, Michael Keane, Patrick Deverett, Sam Black, Bill Yan, Bo Sabir, Bushra Sun, Frank Zhang, Hao Farlow, Harriet Zhou, Helen Dong, Lingming Lu, Qinghua Jang, Seung Abuadbba, Sharif O'Callaghan, Simon Ma, Suyu Howroyd, Tom Fung, Cyrus Azadi, Fatemeh Nejadgholi, Isar Vishnubhotla, Krishnapriya Xiong, Pulei Lohrasbi, Saeedeh Buffett, Scott Iqbal, Shahrear Vajjala, Sowmya Safont-Andreu, Anna Massarelli, Luca van der Wal, Oskar Möller, Simon Delaborde, Agnes Duguépéroux, Joris Rolin, Nicolas Gallienne, Romane Behanzin, Sarah Seimandi, Tom Murakami, Akiko Semitsu, Takayuki Tsukiji, Teresa Kinuthia, Angela Michie, Michael Kasaon, Stephanie Wangari, Jean Baek, Hankyul Noh, Jaewon Nam, Kihyuk Seo, Sang Shin, Sungpil Lee, Taewhi Kim, Yongsu Artificial Intelligence The rapid rise of autonomous AI systems and advancements in agent capabilities are introducing new risks due to reduced oversight of real-world interactions. Yet agent testing remains nascent and is still a developing science. As AI agents begin to be deployed globally, it is important that they handle different languages and cultures accurately and securely. To address this, participants from The International Network for Advanced AI Measurement, Evaluation and Science, including representatives from Singapore, Japan, Australia, Canada, the European Commission, France, Kenya, South Korea, and the United Kingdom have come together to align approaches to agentic evaluations. This is the third exercise, building on insights from two earlier joint testing exercises conducted by the Network in November 2024 and February 2025. The objective is to further refine best practices for testing advanced AI systems. The exercise was split into two strands: (1) common risks, including leakage of sensitive information and fraud, led by Singapore AISI; and (2) cybersecurity, led by UK AISI. A mix of open and closed-weight models were evaluated against tasks from various public agentic benchmarks. Given the nascency of agentic testing, our primary focus was on understanding methodological issues in conducting such tests, rather than examining test results or model capabilities. This collaboration marks an important step forward as participants work together to advance the science of agentic evaluations. |
| title | Improving Methodologies for Agentic Evaluations Across Domains: Leakage of Sensitive Information, Fraud and Cybersecurity Threats |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2601.15679 |