Cover Image

"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kurtic, Eldar, Marques, Alexandre, Pandit, Shubhra, Kurtz, Mark, Alistarh, Dan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models
by: Kurtic, Eldar, et al.
Published: (2024)

Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
by: Egiazarian, Vage, et al.
Published: (2025)

Statistically-Lossless Quantization of Large Language Models
by: Helcig, Michael, et al.
Published: (2026)

Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment
by: Agarwalla, Abhinav, et al.
Published: (2024)

EvoPress: Accurate Dynamic Model Compression via Evolutionary Search
by: Sieberling, Oliver, et al.
Published: (2024)

Fairness-Accuracy Trade-Offs: A Causal Perspective
by: Plecko, Drago, et al.
Published: (2024)

"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models
by: Knauer, Ricardo, et al.
Published: (2024)

Does "Do Differentiable Simulators Give Better Policy Gradients?'' Give Better Policy Gradients?
by: Onoda, Ku, et al.
Published: (2026)

Panza: Design and Analysis of a Fully-Local Personalized Text Writing Assistant
by: Nicolicioiu, Armand, et al.
Published: (2024)

DarwinLM: Evolutionary Structured Pruning of Large Language Models
by: Tang, Shengkun, et al.
Published: (2025)

Do Transformer World Models Give Better Policy Gradients?
by: Ma, Michel, et al.
Published: (2024)

ECO: Quantized Training without Full-Precision Master Weights
by: Nikdan, Mahdi, et al.
Published: (2026)

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling
by: Dadgarnia, Alireza, et al.
Published: (2026)

See Me and Believe Me: Causality and Intersectionality in Testimonial Injustice in Healthcare
by: Andrews, Kenya S., et al.
Published: (2024)

"Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations
by: Hou, Muhan, et al.
Published: (2024)

The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM
by: Lee, Kwanhee, et al.
Published: (2025)

First-See-Then-Design: A Multi-Stakeholder View for Optimal Performance-Fairness Trade-Offs
by: Gupta, Kavya, et al.
Published: (2026)

MatryoshkaLoRA: Learning Accurate Hierarchical Low-Rank Representations for LLM Fine-Tuning
by: Modoranu, Ionut-Vlad, et al.
Published: (2026)

Error Feedback Can Accurately Compress Preconditioners
by: Modoranu, Ionut-Vlad, et al.
Published: (2023)

Beyond Behavioural Trade-Offs: Mechanistic Tracing of Pain-Pleasure Decisions in an LLM
by: Bianco, Francesca, et al.
Published: (2026)

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
by: Knupp, Jonas, et al.
Published: (2026)

GiveMeLabeledIssues: An Open Source Issue Recommendation System
by: Vargovich, Joseph, et al.
Published: (2023)

Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning
by: Liu, Yong, et al.
Published: (2024)

Accuracy-Efficiency Trade-Offs in Spiking Neural Networks: A Lempel-Ziv Complexity Perspective on Learning Rules
by: Rudnicka, Zofia, et al.
Published: (2025)

Textual Unlearning Gives a False Sense of Unlearning
by: Du, Jiacheng, et al.
Published: (2024)

Pruning Extensions and Efficiency Trade-Offs for Sustainable Time Series Classification
by: Fischer, Raphael, et al.
Published: (2026)

MeSH: Memory-as-State-Highways for Recursive Transformers
by: Yu, Chengting, et al.
Published: (2025)

First-Explore, then Exploit: Meta-Learning to Solve Hard Exploration-Exploitation Trade-Offs
by: Norman, Ben, et al.
Published: (2023)

MeKi: Memory-based Expert Knowledge Injection for Efficient LLM Scaling
by: Ding, Ning, et al.
Published: (2026)

AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
by: Cai, Zhijie, et al.
Published: (2026)

Pool Me Wisely: On the Effect of Pooling in Transformer-Based Models
by: Ennadir, Sofiane, et al.
Published: (2025)

MeMo: Memory as a Model
by: Quek, Ryan Wei Heng, et al.
Published: (2026)

Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models
by: Shutova, Alina, et al.
Published: (2025)

Stop Listening to Me! How Multi-turn Conversations Can Degrade LLM Reliability
by: Guo, Kevin H., et al.
Published: (2026)

Teach Me to Trick: Exploring Adversarial Transferability via Knowledge Distillation
by: Pradhan, Siddhartha, et al.
Published: (2025)

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
by: Zhao, Runsong, et al.
Published: (2026)

MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
by: Duanmu, Haojie, et al.
Published: (2025)

Wasserstein Distances, Neuronal Entanglement, and Sparsity
by: Sawmya, Shashata, et al.
Published: (2024)

PatternPortrait: Draw Me Like One of Your Scribbles
by: Wieluch, Sabine, et al.
Published: (2024)

Differential Diffusion: Giving Each Pixel Its Strength
by: Levin, Eran, et al.
Published: (2023)