Copertina

Reliability Under Randomness: An Empirical Analysis of Sparse and Dense Language Models Across Decoding Temperatures

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autore principale:	Grover, Kabir
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Speculative Decoding Across Languages
di: Paudel, Nirajan, et al.
Pubblicazione: (2026)

Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism
di: Li, Guanchen, et al.
Pubblicazione: (2024)

Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language Models
di: O'Neill, Charles, et al.
Pubblicazione: (2024)

SqueezeLLM: Dense-and-Sparse Quantization
di: Kim, Sehoon, et al.
Pubblicazione: (2023)

$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts
di: Chen, Guanjie, et al.
Pubblicazione: (2024)

Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
di: Pan, Bowen, et al.
Pubblicazione: (2024)

Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
di: Tao, Leitian, et al.
Pubblicazione: (2025)

DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models
di: He, Wei, et al.
Pubblicazione: (2024)

Learn from the Past: Fast Sparse Indexing for Large Language Model Decoding
di: Yao, Feiyu, et al.
Pubblicazione: (2025)

Sparse and Dense Retrievers Learn Better Together: Joint Sparse-Dense Optimization for Text-Image Retrieval
di: Song, Jonghyun, et al.
Pubblicazione: (2025)

d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning
di: Zhao, Siyan, et al.
Pubblicazione: (2025)

PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding
di: Koromilas, Panagiotis, et al.
Pubblicazione: (2026)

Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoders
di: Lan, Michael, et al.
Pubblicazione: (2024)

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
di: Doering, Nigel, et al.
Pubblicazione: (2024)

DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices
di: Song, Chenyang, et al.
Pubblicazione: (2026)

Probing the Decision Boundaries of In-context Learning in Large Language Models
di: Zhao, Siyan, et al.
Pubblicazione: (2024)

SparseEval: Efficient Evaluation of Large Language Models by Sparse Optimization
di: Zhang, Taolin, et al.
Pubblicazione: (2026)

Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models
di: Hashimoto, Wataru, et al.
Pubblicazione: (2025)

Group Preference Optimization: Few-Shot Alignment of Large Language Models
di: Zhao, Siyan, et al.
Pubblicazione: (2023)

Sparse Attention Remapping with Clustering for Efficient LLM Decoding on PIM
di: Fan, Zehao, et al.
Pubblicazione: (2025)

How Reliable is Language Model Micro-Benchmarking?
di: Yauney, Gregory, et al.
Pubblicazione: (2025)

An Empirical Study of Mamba-based Language Models
di: Waleffe, Roger, et al.
Pubblicazione: (2024)

Stability-Weighted Decoding for Diffusion Language Models
di: Wu, Yue, et al.
Pubblicazione: (2026)

Learning to Decode Collaboratively with Multiple Language Models
di: Shen, Shannon Zejiang, et al.
Pubblicazione: (2024)

Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models
di: Bansal, Hritik, et al.
Pubblicazione: (2023)

Introducing Background Temperature to Characterise Hidden Randomness in Large Language Models
di: Messina, Alberto, et al.
Pubblicazione: (2026)

Accelerating Diffusion LLMs via Adaptive Parallel Decoding
di: Israel, Daniel, et al.
Pubblicazione: (2025)

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
di: Wang, Hongyu, et al.
Pubblicazione: (2024)

Do Large Language Model Benchmarks Test Reliability?
di: Vendrow, Joshua, et al.
Pubblicazione: (2025)

Transferring Linear Features Across Language Models With Model Stitching
di: Chen, Alan, et al.
Pubblicazione: (2025)

Dense vs Sparse Pretraining at Tiny Scale: Active-Parameter vs Total-Parameter Matching
di: Wael, Abdalrahman
Pubblicazione: (2026)

Layer-wise Representation Dynamics: An Empirical Investigation Across Embedders and Base LLMs
di: Jiang, Jingzhou, et al.
Pubblicazione: (2026)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
di: Zhao, Siyan, et al.
Pubblicazione: (2026)

Sparse Layers are Critical to Scaling Looped Language Models
di: Lee, Ryan, et al.
Pubblicazione: (2026)

An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
di: Mundra, Nandini, et al.
Pubblicazione: (2024)

Assessing Adversarial Robustness of Large Language Models: An Empirical Study
di: Yang, Zeyu, et al.
Pubblicazione: (2024)

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws
di: Jin, Tian, et al.
Pubblicazione: (2025)

FlashDecoding++: Faster Large Language Model Inference on GPUs
di: Hong, Ke, et al.
Pubblicazione: (2023)

Recurrent Drafter for Fast Speculative Decoding in Large Language Models
di: Cheng, Yunfei, et al.
Pubblicazione: (2024)

Decoding Rarity: Large Language Models in the Diagnosis of Rare Diseases
di: Carbonari, Valentina, et al.
Pubblicazione: (2025)