Cover Image

FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yi, Yang, Fei, Peng, Shuang, Wang, Fangyu, Pan, Aimin
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
by: Xiao, Guangxuan, et al.
Published: (2022)

End-to-End Graph Flattening Method for Large Language Models
by: Hong, Bin, et al.
Published: (2024)

Psychological Counseling Ability of Large Language Models
by: Peng, Fangyu, et al.
Published: (2025)

Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
by: Shen, Xuan, et al.
Published: (2023)

Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendation
by: Zhan, Zaifu, et al.
Published: (2025)

Evaluating Quantized Large Language Models
by: Li, Shiyao, et al.
Published: (2024)

Squat: Quant Small Language Models on the Edge
by: Shen, Xuan, et al.
Published: (2024)

GS-Quant: Granular Semantic and Generative Structural Quantization for Knowledge Graph Completion
by: Xie, Qizhuo, et al.
Published: (2026)

DLLMQuant: Quantizing Diffusion-based Large Language Models
by: Xu, Chen, et al.
Published: (2025)

MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods
by: Xu, Zukang, et al.
Published: (2025)

Interpretable Differential Diagnosis with Dual-Inference Large Language Models
by: Zhou, Shuang, et al.
Published: (2024)

SASQ: Static Activation Scaling for Quantization-Aware Training in Large Language Models
by: Mao, Shizhuo, et al.
Published: (2025)

Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
by: Wang, Dongwei, et al.
Published: (2026)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
by: Hu, Xing, et al.
Published: (2024)

QuantMoE-Bench: Examining Post-Training Quantization for Mixture-of-Experts
by: Li, Pingzhi, et al.
Published: (2024)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
by: Chen, Mengzhao, et al.
Published: (2024)

PerMedCQA: Benchmarking Large Language Models on Medical Consumer Question Answering in Persian Language
by: Jamali, Naghmeh, et al.
Published: (2025)

When Quantization Affects Confidence of Large Language Models?
by: Proskurina, Irina, et al.
Published: (2024)

CrossQuant: A Post-Training Quantization Method with Smaller Quantization Kernel for Precise Large Language Model Compression
by: Liu, Wenyuan, et al.
Published: (2024)

Self-supervised Quantized Representation for Seamlessly Integrating Knowledge Graphs with Large Language Models
by: Lin, Qika, et al.
Published: (2025)

Break the Chain: Large Language Models Can be Shortcut Reasoners
by: Ding, Mengru, et al.
Published: (2024)

On the Compressibility of Quantized Large Language Models
by: Mao, Yu, et al.
Published: (2024)

PersLLM: A Personified Training Approach for Large Language Models
by: Zeng, Zheni, et al.
Published: (2024)

A Comprehensive Evaluation of Quantization Strategies for Large Language Models
by: Jin, Renren, et al.
Published: (2024)

Channel-Wise Mixed-Precision Quantization for Large Language Models
by: Chen, Zihan, et al.
Published: (2024)

InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models
by: Yan, Yuchen, et al.
Published: (2025)

LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior Accuracy Preservation
by: Chen, Han, et al.
Published: (2025)

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
by: Xu, Bingxin, et al.
Published: (2025)

Enhance Reasoning for Large Language Models in the Game Werewolf
by: Wu, Shuang, et al.
Published: (2024)

Synthetic Data Generation for Phrase Break Prediction with Large Language Model
by: Lee, Hoyeon, et al.
Published: (2025)

Argument Collapse: LLMs Flatten Long-Form Public Debate
by: Kim, Yekyung, et al.
Published: (2026)

Mis-prompt: Benchmarking Large Language Models for Proactive Error Handling
by: Zeng, Jiayi, et al.
Published: (2025)

FinDABench: Benchmarking Financial Data Analysis Ability of Large Language Models
by: Liu, Shu, et al.
Published: (2024)

LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting
by: Li, Yewen, et al.
Published: (2026)

QuantLRM: Quantization of Large Reasoning Models via Fine-Tuning Signals
by: Zhang, Nan, et al.
Published: (2026)

Hey, That's My Data! Token-Only Dataset Inference in Large Language Models
by: Xiong, Chen, et al.
Published: (2025)

DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization
by: Lin, Haokun, et al.
Published: (2026)

CataLM: Empowering Catalyst Design Through Large Language Models
by: Wang, Ludi, et al.
Published: (2024)

Large Language Models and Causal Inference in Collaboration: A Survey
by: Liu, Xiaoyu, et al.
Published: (2024)