Cover Image

A Methodology for Evaluating RAG Systems: A Case Study On Configuration Dependency Validation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Simon, Sebastian, Mailach, Alina, Dorn, Johannes, Siegmund, Norbert
Format:	Preprint
Published:	2024
Subjects:	Software Engineering Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Themes of Building LLM-based Applications for Production: A Practitioner's View
by: Mailach, Alina, et al.
Published: (2024)

Balanced Knowledge Distribution among Software Development Teams -- Observations from Open-Source and Closed-Source Software Development
by: Shafiq, Saad, et al.
Published: (2022)

Towards AI Evaluation in Domain-Specific RAG Systems: The AgriHubi Case Study
by: Hasan, Md. Toufique, et al.
Published: (2026)

An Empirical Study of Multi-Agent RAG for Real-World University Admissions Counseling
by: Nguyen-Duc, Anh, et al.
Published: (2025)

Leveraging Graph-RAG and Prompt Engineering to Enhance LLM-Based Automated Requirement Traceability and Compliance Checks
by: Masoudifard, Arsalan, et al.
Published: (2024)

Evaluating LLM-Based Mobile App Recommendations: An Empirical Study
by: Motger, Quim, et al.
Published: (2025)

Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report
by: Khan, Ayman Asad, et al.
Published: (2024)

Detecting Performance-Relevant Changes in Configurable Software Systems
by: Böhm, Sebastian, et al.
Published: (2025)

A Survey on Query-based API Recommendation
by: Wei, Moshi, et al.
Published: (2023)

CodeRAG: Finding Relevant and Necessary Knowledge for Retrieval-Augmented Repository-Level Code Completion
by: Zhang, Sheng, et al.
Published: (2025)

The Impact Of Bug Localization Based on Crash Report Mining: A Developers' Perspective
by: Medeiros, Marcos, et al.
Published: (2024)

MGS3: A Multi-Granularity Self-Supervised Code Search Framework
by: Li, Rui, et al.
Published: (2025)

Unveiling Bias in Fairness Evaluations of Large Language Models: A Critical Literature Review of Music and Movie Recommendation Systems
by: Sah, Chandan Kumar, et al.
Published: (2024)

Use as Directed? A Comparison of Software Tools Intended to Check Rigor and Transparency of Published Work
by: Eckmann, Peter, et al.
Published: (2025)

SBAN: A Framework & Multi-Dimensional Dataset for Large Language Model Pre-Training and Software Code Mining
by: Jelodar, Hamed, et al.
Published: (2025)

Deep Code Search with Naming-Agnostic Contrastive Multi-View Learning
by: Feng, Jiadong, et al.
Published: (2024)

Multi-View Adaptive Contrastive Learning for Information Retrieval Based Fault Localization
by: Zhou, Chunying, et al.
Published: (2024)

Supporting Cross-language Cross-project Bug Localization Using Pre-trained Language Models
by: Chandramohan, Mahinthan, et al.
Published: (2024)

Nirjas: An open source framework for extracting metadata from the source code
by: Bhardwaj, Ayush, et al.
Published: (2024)

Source Code Clone Detection Using Unsupervised Similarity Measures
by: Martinez-Gil, Jorge
Published: (2024)

What About Emotions? Guiding Fine-Grained Emotion Extraction from Mobile App Reviews
by: Motger, Quim, et al.
Published: (2025)

Incremental Analysis of Legacy Applications Using Knowledge Graphs for Application Modernization
by: Krishnan, Saravanan, et al.
Published: (2025)

FLOWER: Flow-Oriented Entity-Relationship Tool
by: Moskalev, Dmitry
Published: (2025)

GNN-Coder: Boosting Semantic Code Retrieval with Combined GNNs and Transformer
by: Ye, Yufan, et al.
Published: (2025)

Optimizing Retrieval Augmented Generation for Object Constraint Language
by: Li, Kevin Chenhao, et al.
Published: (2025)

From Chaos to Automation: Enabling the Use of Unstructured Data for Robotic Process Automation
by: Kurowski, Kelly, et al.
Published: (2025)

Using GPT to build a Project Management assistant for Jira environments
by: Garcia-Escribano, Joel, et al.
Published: (2025)

In-Context Learning as an Effective Estimator of Functional Correctness of LLM-Generated Code
by: Das, Susmita, et al.
Published: (2025)

Modular Layout Synthesis (MLS): Front-end Code via Structure Normalization and Constrained Generation
by: Liu, Chong, et al.
Published: (2025)

Towards Analysing Invoices and Receipts with Amazon Textract
by: Oommen, Sneha, et al.
Published: (2025)

Development of an Automated Web Application for Efficient Web Scraping: Design and Implementation
by: Dutta, Alok, et al.
Published: (2025)

Code-Craft: Hierarchical Graph-Based Code Summarization for Enhanced Context Retrieval
by: Sounthiraraj, David, et al.
Published: (2025)

yProv4DV: Reproducible Data Visualization Scripts Out of the Box
by: Padovani, Gabriele, et al.
Published: (2026)

Less is More: On the Importance of Data Quality for Unit Test Generation
by: Zhang, Junwei, et al.
Published: (2025)

ODataX: A Progressive Evolution of the Open Data Protocol
by: Ganesh, Anirudh, et al.
Published: (2025)

Can Code Evaluation Metrics Detect Code Plagiarism?
by: Ebrahim, Fahad, et al.
Published: (2026)

Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms
by: Shah, Deep, et al.
Published: (2026)

Development and Evaluation of Dental Image Exchange and Management System: A User-Centered Perspective
by: Rahimi, B, et al.
Published: (2022)

AI-assisted Coding with Cody: Lessons from Context Retrieval and Evaluation for Code Recommendations
by: Hartman, Jan, et al.
Published: (2024)

TreeRanker: Fast and Model-agnostic Ranking System for Code Suggestions in IDEs
by: Cipollone, Daniele, et al.
Published: (2025)