Copertina

Developing and Maintaining an Open-Source Repository of AI Evaluations: Challenges and Insights

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Abbas, Alexandra, Waggoner, Celia, Olive, Justin
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

ViSoLex: An Open-Source Repository for Vietnamese Social Media Lexical Normalization
di: Nguyen, Anh Thi-Hoang, et al.
Pubblicazione: (2025)

Lynx: An Open Source Hallucination Evaluation Model
di: Ravi, Selvan Sunitha, et al.
Pubblicazione: (2024)

OpenFActScore: Open-Source Atomic Evaluation of Factuality in Text Generation
di: Lage, Lucas Fonseca, et al.
Pubblicazione: (2025)

Narrative Context Protocol: An Open-Source Storytelling Framework for Generative AI
di: Gerba, Hank
Pubblicazione: (2025)

Detecting AI-Generated Sentences in Human-AI Collaborative Hybrid Texts: Challenges, Strategies, and Insights
di: Zeng, Zijie, et al.
Pubblicazione: (2024)

Veracity: An Open-Source AI Fact-Checking System
di: Curtis, Taylor Lynn, et al.
Pubblicazione: (2025)

A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions
di: Shorinwa, Ola, et al.
Pubblicazione: (2024)

MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data
di: Han, Tianyu, et al.
Pubblicazione: (2023)

SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositories
di: Wang, Lilin, et al.
Pubblicazione: (2025)

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
di: Amjad, Husnain, et al.
Pubblicazione: (2026)

Thematic Analysis with Open-Source Generative AI and Machine Learning: A New Method for Inductive Qualitative Codebook Development
di: Katz, Andrew, et al.
Pubblicazione: (2024)

Promises, Outlooks and Challenges of Diffusion Language Modeling
di: Deschenaux, Justin, et al.
Pubblicazione: (2024)

Performance Evaluation of Open-Source Large Language Models for Assisting Pathology Report Writing in Japanese
di: Kawai, Masataka, et al.
Pubblicazione: (2026)

Understanding LLM Development Through Longitudinal Study: Insights from the Open Ko-LLM Leaderboard
di: Park, Chanjun, et al.
Pubblicazione: (2024)

Is Open Source the Future of AI? A Data-Driven Approach
di: Vake, Domen, et al.
Pubblicazione: (2025)

Analyzing Feedback Mechanisms in AI-Generated MCQs: Insights into Readability, Lexical Properties, and Levels of Challenge
di: Yaacoub, Antoun, et al.
Pubblicazione: (2025)

Maintaining Journalistic Integrity in the Digital Age: A Comprehensive NLP Framework for Evaluating Online News Content
di: Bojic, Ljubisa, et al.
Pubblicazione: (2024)

OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
di: Toshniwal, Shubham, et al.
Pubblicazione: (2024)

OpenHands: An Open Platform for AI Software Developers as Generalist Agents
di: Wang, Xingyao, et al.
Pubblicazione: (2024)

Orchard: An Open-Source Agentic Modeling Framework
di: Peng, Baolin, et al.
Pubblicazione: (2026)

Qabas: An Open-Source Arabic Lexicographic Database
di: Jarrar, Mustafa, et al.
Pubblicazione: (2024)

Is Open-Source There Yet? A Comparative Study on Commercial and Open-Source LLMs in Their Ability to Label Chest X-Ray Reports
di: Dorfner, Felix J., et al.
Pubblicazione: (2024)

Localizing AI: Evaluating Open-Weight Language Models for Languages of Baltic States
di: Kapočiūtė-Dzikienė, Jurgita, et al.
Pubblicazione: (2025)

ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code
di: Tang, Xiangru, et al.
Pubblicazione: (2023)

OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models
di: Wang, Jun, et al.
Pubblicazione: (2024)

RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation
di: Luo, Qinyu, et al.
Pubblicazione: (2024)

OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data
di: Du, Yuwen, et al.
Pubblicazione: (2026)

Benchmarking Open-Source Safety Guard Models: A Comprehensive Evaluation
di: Harsh, Reetu Raj, et al.
Pubblicazione: (2026)

TinyLlama: An Open-Source Small Language Model
di: Zhang, Peiyuan, et al.
Pubblicazione: (2024)

Design of an Open-Source Architecture for Neural Machine Translation
di: Lankford, Séamus, et al.
Pubblicazione: (2024)

CT Open: An Open-Access, Uncontaminated, Live Platform for the Open Challenge of Clinical Trial Outcome Prediction
di: Wang, Jianyou, et al.
Pubblicazione: (2026)

Use of AI Tools: Guidelines to Maintain Academic Integrity in Computing Colleges
di: El-boghdadi, Hatem M., et al.
Pubblicazione: (2026)

Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
di: Zou, Andy, et al.
Pubblicazione: (2025)

SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integration
di: Chen, Jialong, et al.
Pubblicazione: (2026)

An Open Source Data Contamination Report for Large Language Models
di: Li, Yucheng, et al.
Pubblicazione: (2023)

Tucano 2 Cool: Better Open Source LLMs for Portuguese
di: Corrêa, Nicholas Kluge, et al.
Pubblicazione: (2026)

RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph
di: Ouyang, Siru, et al.
Pubblicazione: (2024)

AI-Assisted Systematization for Evaluating GenAI Systems
di: Agarwal, Dhruv, et al.
Pubblicazione: (2026)

Creativity in AI: Progresses and Challenges
di: Ismayilzada, Mete, et al.
Pubblicazione: (2024)

VerifAI: A Verifiable Open-Source Search Engine for Biomedical Question Answering
di: Košprdić, Miloš, et al.
Pubblicazione: (2026)