Cover Image

OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shuting, Tan, Jiejun, Dou, Zhicheng, Wen, Ji-Rong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OmniEval: A Benchmark for Evaluating Omni-modal Models with Visual, Auditory, and Textual Inputs
by: Zhang, Yiman, et al.
Published: (2025)

DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation
by: Wang, Shuting, et al.
Published: (2024)

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs
by: Tan, Jiejun, et al.
Published: (2024)

HierSearch: A Hierarchical Enterprise Deep Search Framework Integrating Local and Web Searches
by: Tan, Jiejun, et al.
Published: (2025)

RichRAG: Crafting Rich Responses for Multi-faceted Queries in Retrieval-Augmented Generation
by: Wang, Shuting, et al.
Published: (2024)

HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems
by: Tan, Jiejun, et al.
Published: (2024)

FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models
by: Guo, Xin, et al.
Published: (2023)

Memory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning
by: Hu, Yuyang, et al.
Published: (2026)

OmniGAIA: Towards Native Omni-Modal AI Agents
by: Li, Xiaoxi, et al.
Published: (2026)

FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning
by: Dou, Shaoyu, et al.
Published: (2025)

QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
by: Kang, Zhaolu, et al.
Published: (2026)

One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
by: Zhu, Yutao, et al.
Published: (2024)

Respond Beyond Language: A Benchmark for Video Generation in Response to Realistic User Intents
by: Wang, Shuting, et al.
Published: (2025)

UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
by: Huang, Zhaoheng, et al.
Published: (2024)

Query-oriented Data Augmentation for Session Search
by: Chen, Haonan, et al.
Published: (2024)

FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain
by: Zhao, Suifeng, et al.
Published: (2025)

ProRAG: Process-Supervised Reinforcement Learning for Retrieval-Augmented Generation
by: Wang, Zhao, et al.
Published: (2026)

FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research
by: Jin, Jiajie, et al.
Published: (2024)

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
by: Shao, Wenqi, et al.
Published: (2023)

SMARTFinRAG: Interactive Modularized Financial RAG Benchmark
by: Zha, Yiwei
Published: (2025)

AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant
by: Zhou, Yujia, et al.
Published: (2024)

Large Language Models for Information Retrieval: A Survey
by: Zhu, Yutao, et al.
Published: (2023)

AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation
by: Zhang, Xiechi, et al.
Published: (2025)

HumanRankEval: Automatic Evaluation of LMs as Conversational Assistants
by: Gritta, Milan, et al.
Published: (2024)

CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation
by: Cheng, Yiruo, et al.
Published: (2024)

e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings
by: Chen, Haonan, et al.
Published: (2026)

FinS-Pilot: A Benchmark for Online Financial RAG System
by: Wang, Feng, et al.
Published: (2025)

OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models
by: Kim, Seunghee, et al.
Published: (2026)

TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains
by: Wang, Wanying, et al.
Published: (2024)

Single LLM, Multiple Roles: A Unified Retrieval-Augmented Generation Framework Using Role-Specific Token Optimization
by: Zhu, Yutao, et al.
Published: (2025)

Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
by: Dong, Guanting, et al.
Published: (2024)

OmniEvalKit: A Modular, Lightweight Toolbox for Evaluating Large Language Model and its Omni-Extensions
by: Zhang, Yi-Kai, et al.
Published: (2024)

Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMs
by: Mendonça, John, et al.
Published: (2024)

Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis
by: Kong, Zicheng, et al.
Published: (2026)

SGSimEval: A Comprehensive Multifaceted and Similarity-Enhanced Benchmark for Automatic Survey Generation Systems
by: Guo, Beichen, et al.
Published: (2025)

ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests
by: Xu, Shiyi, et al.
Published: (2025)

AILS-NTUA at SemEval-2026 Task 8: Evaluating Multi-Turn RAG Conversations
by: Athanasiou, Dimosthenis, et al.
Published: (2026)

EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis
by: Song, Xiaoshuai, et al.
Published: (2026)

EvalSense: A Framework for Domain-Specific LLM (Meta-)Evaluation
by: Dejl, Adam, et al.
Published: (2026)

From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning
by: Deng, Zhirui, et al.
Published: (2024)