Image de couverture de livre

Characterizing the Accuracy -- Efficiency Trade-off of Low-rank Decomposition in Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Moar, Chakshu, Tahmasebi, Faraz, Pellauer, Michael, Kwon, Hyoukjun
Format:	Preprint
Publié:	2024
Sujets:	Machine Learning Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

D-com: Accelerating Iterative Processing to Enable Low-rank Decomposition of Activations
par: Tahmasebi, Faraz, et autres
Publié: (2025)

Exploring Accuracy-Fairness Trade-off in Large Language Models
par: Zhang, Qingquan, et autres
Publié: (2024)

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
par: Guo, Han, et autres
Publié: (2023)

Architectural Trade-offs in Small Language Models Under Compute Constraints
par: Bhatti, Shivraj Singh
Publié: (2025)

Evaluating Cost-Accuracy Trade-offs in Multimodal Search Relevance Judgements
par: Terragni, Silvia, et autres
Publié: (2024)

Demystifying Language Model Forgetting with Low-rank Example Associations
par: Jin, Xisen, et autres
Publié: (2024)

Optimizing Humor Generation in Large Language Models: Temperature Configurations and Architectural Trade-offs
par: Evstafev, Evgenii
Publié: (2025)

Understanding the Quality-Diversity Trade-off in Diffusion Language Models
par: Buzzard, Zak
Publié: (2025)

Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses
par: Liu, Yongfan, et autres
Publié: (2024)

Brainformers: Trading Simplicity for Efficiency
par: Zhou, Yanqi, et autres
Publié: (2023)

Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations
par: Xie, Zhihui, et autres
Publié: (2024)

Emissions and Performance Trade-off Between Small and Large Language Models
par: Garg, Anandita, et autres
Publié: (2025)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
par: Chen, Pin-Yu, et autres
Publié: (2025)

GRU: Mitigating the Trade-off between Unlearning and Retention for LLMs
par: Wang, Yue, et autres
Publié: (2025)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
par: Nawrot, Piotr, et autres
Publié: (2025)

Conformal Linguistic Calibration: Trading-off between Factuality and Specificity
par: Jiang, Zhengping, et autres
Publié: (2025)

Better To Ask in English? Evaluating Factual Accuracy of Multilingual LLMs in English and Low-Resource Languages
par: Rohera, Pritika, et autres
Publié: (2025)

Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applications
par: Li, Yang, et autres
Publié: (2024)

MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization
par: Zhao, Yang, et autres
Publié: (2026)

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
par: Rajabzadeh, Hossein, et autres
Publié: (2024)

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation
par: Si, Chongjie, et autres
Publié: (2025)

Demystifying the Accuracy-Interpretability Trade-Off: A Case Study of Inferring Ratings from Reviews
par: Atrey, Pranjal, et autres
Publié: (2025)

Detecting and Characterizing Planning in Language Models
par: Nainani, Jatin, et autres
Publié: (2025)

Text to Trust: Evaluating Fine-Tuning and LoRA Trade-offs in Language Models for Unfair Terms of Service Detection
par: Juttu, Noshitha Padma Pratyusha, et autres
Publié: (2025)

Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptation
par: Zhang, Biao, et autres
Publié: (2025)

Low-rank Optimization Trajectories Modeling for LLM RLVR Acceleration
par: Chen, Zhipeng, et autres
Publié: (2026)

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
par: Fonseca, Joao, et autres
Publié: (2025)

$100K or 100 Days: Trade-offs when Pre-Training with Academic Resources
par: Khandelwal, Apoorv, et autres
Publié: (2024)

Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
par: Yang, Seongjun, et autres
Publié: (2023)

SCALPEL: Selective Capability Ablation via Low-rank Parameter Editing for Large Language Model Interpretability Analysis
par: Fu, Zihao, et autres
Publié: (2026)

Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
par: He, Zhengfu, et autres
Publié: (2025)

Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation
par: Das, Devleena, et autres
Publié: (2025)

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
par: Li, Zichao, et autres
Publié: (2026)

The Structure-Content Trade-off in Knowledge Graph Retrieval
par: Six, Valentin, et autres
Publié: (2025)

Graph-Based Spectral Decomposition for Parameter Coordination in Language Model Fine-Tuning
par: Zhang, Hanlu, et autres
Publié: (2025)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
par: Cao, Jinghan, et autres
Publié: (2026)

Invariant Features in Language Models: Geometric Characterization and Model Attribution
par: Dasgupta, Agnibh, et autres
Publié: (2026)

Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergence
par: Yang, Yibo, et autres
Publié: (2025)

Adversarial Evasion Attack Efficiency against Large Language Models
par: Vitorino, João, et autres
Publié: (2024)

Energy Considerations of Large Language Model Inference and Efficiency Optimizations
par: Fernandez, Jared, et autres
Publié: (2025)