Image de couverture de livre

Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Cheng, Cheng, Jianyi, Shumailov, Ilia, Constantinides, George A., Zhao, Yiren
Format:	Preprint
Publié:	2023
Sujets:	Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LQER: Low-Rank Quantization Error Reconstruction for LLMs
par: Zhang, Cheng, et autres
Publié: (2024)

Hardware and Software Platform Inference
par: Zhang, Cheng, et autres
Publié: (2024)

AMPLE: Event-Driven Accelerator for Mixed-Precision Inference of Graph Neural Networks
par: Gimenes, Pedro, et autres
Publié: (2025)

Quantamination: Dynamic Quantization Leaks Your Data Across the Batch
par: Foerster, Hanna, et autres
Publié: (2026)

NeuraLUT-Assemble: Hardware-aware Assembling of Sub-Neural Networks for Efficient LUT Inference
par: Andronic, Marta, et autres
Publié: (2025)

QERA: an Analytical Framework for Quantization Error Reconstruction
par: Zhang, Cheng, et autres
Publié: (2024)

ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks
par: Clifford, Eleanor, et autres
Publié: (2022)

A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats
par: Cheng, Jianyi, et autres
Publié: (2023)

Optimised Grouped-Query Attention Mechanism for Transformers
par: Chen, Yuang, et autres
Publié: (2024)

Unlocking the Global Synergies in Low-Rank Adapters
par: Zhang, Zixi, et autres
Publié: (2024)

Architectural Neural Backdoors from First Principles
par: Langford, Harry, et autres
Publié: (2024)

Locking Machine Learning Models into Hardware
par: Clifford, Eleanor, et autres
Publié: (2024)

PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
par: Andronic, Marta, et autres
Publié: (2023)

SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks
par: Gao, Yue, et autres
Publié: (2023)

Beyond Labeling Oracles: What does it mean to steal ML models?
par: Shafran, Avital, et autres
Publié: (2023)

Fairness Feedback Loops: Training on Synthetic Data Amplifies Bias
par: Wyllie, Sierra, et autres
Publié: (2024)

Machine Learning needs Better Randomness Standards: Randomised Smoothing and PRNG-based attacks
par: Dahiya, Pranav, et autres
Publié: (2023)

Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation
par: Küchler, Nicolas, et autres
Publié: (2025)

Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated
par: Foerster, Hanna, et autres
Publié: (2025)

Ultra-Quantisation: Efficient Embedding Search via 1.58-bit Encodings
par: Connor, Richard, et autres
Publié: (2025)

Buffer Overflow in Mixture of Experts
par: Hayes, Jamie, et autres
Publié: (2024)

The Curse of Recursion: Training on Generated Data Makes Models Forget
par: Shumailov, Ilia, et autres
Publié: (2023)

LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation
par: Zhang, Zixi, et autres
Publié: (2023)

LO-BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inference
par: Elangovan, Reena, et autres
Publié: (2025)

PolyLUT: Ultra-low Latency Polynomial Inference with Hardware-Aware Structured Pruning
par: Andronic, Marta, et autres
Publié: (2025)

Scaling Laws For Mixed Quantization
par: Cao, Zeyu, et autres
Publié: (2024)

A3 : an Analytical Low-Rank Approximation Framework for Attention
par: Wong, Jeffrey T. H., et autres
Publié: (2025)

ceLLMate: Sandboxing Browser AI Agents
par: Meng, Luoxi, et autres
Publié: (2025)

NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions
par: Andronic, Marta, et autres
Publié: (2024)

Watermarking Needs Input Repetition Masking
par: Khachaturov, David, et autres
Publié: (2025)

Beyond Slow Signs in High-fidelity Model Extraction
par: Foerster, Hanna, et autres
Publié: (2024)

Measuring memorization in RLHF for code completion
par: Pappu, Aneesh, et autres
Publié: (2024)

HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator
par: Yu, Zhewen, et autres
Publié: (2024)

Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography
par: Shumailov, Ilia, et autres
Publié: (2025)

Tight Non-asymptotic Inference via Sub-Gaussian Intrinsic Moment Norm
par: Zhang, Huiming, et autres
Publié: (2023)

Towards Low-bit Communication for Tensor Parallel LLM Inference
par: Dong, Harry, et autres
Publié: (2024)

Optimal Formats for Weight Quantisation
par: Orr, Douglas, et autres
Publié: (2025)

When Vision Fails: Text Attacks Against ViT and OCR
par: Boucher, Nicholas, et autres
Publié: (2023)

Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning
par: Rinberg, Roy, et autres
Publié: (2025)

Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
par: Hayes, Jamie, et autres
Publié: (2024)