Cover Image

ALIEN: Aligned Entropy Head for Improving Uncertainty Estimation of LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zabolotnyi, Artem, Makarov, Roman, Mitrovic, Mile, Proskura, Polina, Travkin, Oleg, Alferov, Roman, Zaytsev, Alexey
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Beyond Simple Averaging: Improving NLP Ensemble Performance with Topological-Data-Analysis-Based Weighting
by: Proskura, Polina, et al.
Published: (2024)

Never Skip a Batch: Continuous Training of Temporal GNNs via Adaptive Pseudo-Supervision
by: Panyshev, Alexander, et al.
Published: (2025)

GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms
by: Khrulkov, Valentin, et al.
Published: (2025)

Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search
by: Fadeeva, Ekaterina, et al.
Published: (2025)

Language steering in latent space to mitigate unintended code-switching
by: Goncharov, Andrey, et al.
Published: (2025)

Entropy, Disagreement, and the Limits of Foundation Models in Genomics
by: Rochkoulets, Maxime, et al.
Published: (2026)

Holistic Uncertainty Estimation For Open-Set Recognition
by: Erlygin, Leonid, et al.
Published: (2024)

Data-Driven Stochastic AC-OPF using Gaussian Processes
by: Mitrovic, Mile
Published: (2024)

The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimation
by: Liu, Mingyi
Published: (2026)

Uncertainty Estimation for the Open-Set Text Classification systems
by: Erlygin, Leonid, et al.
Published: (2026)

When an LLM is apprehensive about its answers -- and when its uncertainty is justified
by: Sychev, Petr, et al.
Published: (2025)

Complexity-aware fine-tuning
by: Goncharov, Andrey, et al.
Published: (2025)

ReDAct: Uncertainty-Aware Deferral for LLM Agents
by: Piatrashyn, Dzianis, et al.
Published: (2026)

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
by: Vashurin, Roman, et al.
Published: (2024)

Topic Modelling Black Box Optimization
by: Akramov, Roman, et al.
Published: (2025)

Shape: A Self-Supervised 3D Geometry Foundation Model for Industrial CAD Analysis
by: Mounmo, Bayangmbe, et al.
Published: (2026)

MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs
by: Bakman, Yavuz Faruk, et al.
Published: (2024)

Entropy-Aligned Decoding of LMs for Better Writing and Reasoning
by: Ahmed, Kareem, et al.
Published: (2026)

Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting
by: Dai, Hui, et al.
Published: (2026)

Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices
by: Kostenok, Elizaveta, et al.
Published: (2023)

CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought
by: Zhang, Boxuan, et al.
Published: (2025)

Aligning (Medical) LLMs for (Counterfactual) Fairness
by: Poulain, Raphael, et al.
Published: (2024)

Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
by: Nikitin, Alexander, et al.
Published: (2024)

Intrinsic Entropy of Context Length Scaling in LLMs
by: Shi, Jingzhe, et al.
Published: (2025)

Interpreting and Mitigating Unwanted Uncertainty in LLMs
by: Roy, Tiasa Singha, et al.
Published: (2025)

Confidence Estimation for Error Detection in Text-to-SQL Systems
by: Somov, Oleg, et al.
Published: (2025)

Aligning LLMs by Predicting Preferences from User Writing Samples
by: Aroca-Ouellette, Stéphane, et al.
Published: (2025)

Improving Transformers with Dynamically Composable Multi-Head Attention
by: Xiao, Da, et al.
Published: (2024)

Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering
by: Bakman, Yavuz, et al.
Published: (2025)

Krony-PT: GPT2 compressed with Kronecker Products
by: Ayad, Mohamed Ayoub Ben, et al.
Published: (2024)

Align to Structure: Aligning Large Language Models with Structural Information
by: Kim, Zae Myung, et al.
Published: (2025)

Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond
by: Wang, Zhiyuan, et al.
Published: (2024)

Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents
by: Wang, Jiawei, et al.
Published: (2025)

Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization
by: Zhang, Zheyuan, et al.
Published: (2026)

Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity
by: Nguyen, Dang, et al.
Published: (2025)

Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
by: Hashimoto, Wataru, et al.
Published: (2025)

TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs
by: Gu, Yuxuan, et al.
Published: (2025)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
by: Choi, Minsik, et al.
Published: (2025)

Improving Instruction Following in Language Models through Proxy-Based Uncertainty Estimation
by: Lee, JoonHo, et al.
Published: (2024)

The benefits of query-based KGQA systems for complex and temporal questions in LLM era
by: Alekseev, Artem, et al.
Published: (2025)