Buchumschlag

AI Observability for Large Language Model Systems: A Multi-Layer Analysis of Monitoring Approaches from Confidence Calibration to Infrastructure Tracing

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
1. Verfasser:	Sisodia, Twinkll
Format:	Preprint
Veröffentlicht:	2026
Schlagworte:	Software Engineering
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

From Natural Language to PromQL: A Catalog-Driven Framework with Dynamic Temporal Resolution for Cloud-Native Observability
von: Sisodia, Twinkll
Veröffentlicht: (2026)

AI Observability for Developer Productivity Tools: Bridging Cost Awareness and Code Quality
von: Bhati, Happy, et al.
Veröffentlicht: (2026)

Monitoring and Observability of Machine Learning Systems: Current Practices and Gaps
von: Leest, Joran, et al.
Veröffentlicht: (2025)

AgentTrace: A Structured Logging Framework for Agent System Observability
von: AlSayyad, Adam, et al.
Veröffentlicht: (2026)

Fine-grained Approaches for Confidence Calibration of LLMs in Automated Code Revision
von: Lin, Hong Yi, et al.
Veröffentlicht: (2026)

Can Large Language Models Serve as Data Analysts? A Multi-Agent Assisted Approach for Qualitative Data Analysis
von: Rasheed, Zeeshan, et al.
Veröffentlicht: (2024)

Statistical Confidence in Functional Correctness: An Approach for AI Product Functional Correctness Evaluation
von: Albertini, Wallace, et al.
Veröffentlicht: (2026)

Generative AI in Systems Engineering: A Framework for Risk Assessment of Large Language Models
von: Otten, Stefan, et al.
Veröffentlicht: (2026)

Large Language Model Evaluation Via Multi AI Agents: Preliminary results
von: Rasheed, Zeeshan, et al.
Veröffentlicht: (2024)

CARGO: A Framework for Confidence-Aware Routing of Large Language Models
von: Barrak, Amine, et al.
Veröffentlicht: (2025)

Enhancing COBOL Code Explanations: A Multi-Agents Approach Using Large Language Models
von: Lei, Fangjian, et al.
Veröffentlicht: (2025)

A Survey of using Large Language Models for Generating Infrastructure as Code
von: Srivatsa, Kalahasti Ganesh, et al.
Veröffentlicht: (2024)

Calibration of Large Language Models on Code Summarization
von: Virk, Yuvraj, et al.
Veröffentlicht: (2024)

TraceLLM: Leveraging Large Language Models with Prompt Engineering for Enhanced Requirements Traceability
von: Alturayeif, Nouf, et al.
Veröffentlicht: (2026)

Tracing the Lifecycle of Architecture Technical Debt in Software Systems: A Dependency Approach
von: Sutoyo, Edi, et al.
Veröffentlicht: (2025)

Sound Concurrent Traces for Online Monitoring Technical Report
von: Soueidi, Chukri, et al.
Veröffentlicht: (2024)

Victor Calibration (VC): Multi-Pass Confidence Calibration and CP4.3 Governance Stress Test under Round-Table Orchestration
von: Stasiuc, Victor
Veröffentlicht: (2025)

A Large Language Model Approach to Identify Flakiness in C++ Projects
von: Sun, Xin, et al.
Veröffentlicht: (2024)

Tracing and Metrics Design Patterns for Monitoring Cloud-native Applications
von: Albuquerque, Carlos, et al.
Veröffentlicht: (2025)

A Multi-Language Object-Oriented Programming Benchmark for Large Language Models
von: Wang, Shuai, et al.
Veröffentlicht: (2025)

LeGEND: A Top-Down Approach to Scenario Generation of Autonomous Driving Systems Assisted by Large Language Models
von: Tang, Shuncheng, et al.
Veröffentlicht: (2024)

Test Case Generation from Bug Reports via Large Language Models: A Cognitive Layered Evaluation Framework
von: Qureshi, Irtaza Sajid, et al.
Veröffentlicht: (2025)

Does In-IDE Calibration of Large Language Models work at Scale?
von: Koohestani, Roham, et al.
Veröffentlicht: (2025)

A Trace-based Approach for Code Safety Analysis
von: Xu, Hui
Veröffentlicht: (2025)

Benchmarking Large Language Models for Multi-Language Software Vulnerability Detection
von: Zhang, Ting, et al.
Veröffentlicht: (2025)

AgentSight: System-Level Observability for AI Agents Using eBPF
von: Zheng, Yusheng, et al.
Veröffentlicht: (2025)

Codified Context: Infrastructure for AI Agents in a Complex Codebase
von: Vasilopoulos, Aristidis
Veröffentlicht: (2026)

Can Small GenAI Language Models Rival Large Language Models in Understanding Application Behavior?
von: Meymani, Mohammad, et al.
Veröffentlicht: (2025)

A Learning Method for Symbolic Systems Using Large Language Models
von: Fang, Jian, et al.
Veröffentlicht: (2026)

ESG Reporting Lifecycle Management with Large Language Models and AI Agents
von: Hoang, Thong, et al.
Veröffentlicht: (2026)

Meta-Fair: AI-Assisted Fairness Testing of Large Language Models
von: Romero-Arjona, Miguel, et al.
Veröffentlicht: (2025)

Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach
von: Muttillo, Vittoriano, et al.
Veröffentlicht: (2024)

A Structured Approach to Safety Case Construction for AI Systems
von: Lee, Sung Une, et al.
Veröffentlicht: (2026)

Sema Code: Decoupling AI Coding Agents into Programmable, Embeddable Infrastructure
von: Wang, Huacan, et al.
Veröffentlicht: (2026)

Semantic-Enhanced Indirect Call Analysis with Large Language Models
von: Cheng, Baijun, et al.
Veröffentlicht: (2024)

BinMetric: A Comprehensive Binary Analysis Benchmark for Large Language Models
von: Shang, Xiuwei, et al.
Veröffentlicht: (2025)

A Tool for In-depth Analysis of Code Execution Reasoning of Large Language Models
von: Liu, Changshu, et al.
Veröffentlicht: (2025)

Code Vulnerability Detection: A Comparative Analysis of Emerging Large Language Models
von: Sultana, Shaznin, et al.
Veröffentlicht: (2024)

Code Digital Twin: A Knowledge Infrastructure for AI-Assisted Complex Software Development
von: Peng, Xin, et al.
Veröffentlicht: (2025)

GenSIaC: Toward Security-Aware Infrastructure-as-Code Generation with Large Language Models
von: Li, Yikun, et al.
Veröffentlicht: (2025)