Imagen de Portada

Data-free Weight Compress and Denoise for Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Peng, Runyu, Zhou, Yunhua, Guo, Qipeng, Gao, Yang, Yan, Hang, Qiu, Xipeng, Lin, Dahua
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
por: Peng, Runyu, et al.
Publicado: (2026)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
por: Peng, Runyu, et al.
Publicado: (2026)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
por: Guo, Xu, et al.
Publicado: (2026)

LongWanjuan: Towards Systematic Measurement for Long Text Quality
por: Lv, Kai, et al.
Publicado: (2024)

Code Needs Comments: Enhancing Code LLMs with Comment Augmentation
por: Song, Demin, et al.
Publicado: (2024)

Balanced Data Sampling for Language Model Training with Clustering
por: Shao, Yunfan, et al.
Publicado: (2024)

Identifying Semantic Induction Heads to Understand In-Context Learning
por: Ren, Jie, et al.
Publicado: (2024)

Full Parameter Fine-tuning for Large Language Models with Limited Resources
por: Lv, Kai, et al.
Publicado: (2023)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
por: Zeng, Zhiyuan, et al.
Publicado: (2024)

Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
por: Fei, Zhaoye, et al.
Publicado: (2024)

BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments
por: Wang, Xinghao, et al.
Publicado: (2024)

F-Eval: Assessing Fundamental Abilities with Refined Evaluation Methods
por: Sun, Yu, et al.
Publicado: (2024)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
por: Wang, Bo, et al.
Publicado: (2025)

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
por: Lv, Kai, et al.
Publicado: (2023)

Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem
por: Sun, Yuhong, et al.
Publicado: (2024)

DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning
por: Wang, Xinghao, et al.
Publicado: (2024)

Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation
por: Zhu, Qin, et al.
Publicado: (2024)

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
por: Ye, Jiasheng, et al.
Publicado: (2024)

DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting
por: Lv, Kai, et al.
Publicado: (2025)

Scaling Laws of RoPE-based Extrapolation
por: Liu, Xiaoran, et al.
Publicado: (2023)

Calibrating the Confidence of Large Language Models by Eliciting Fidelity
por: Zhang, Mozhi, et al.
Publicado: (2024)

AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
por: Zhu, Qin, et al.
Publicado: (2025)

Evolution of Concepts in Language Model Pre-Training
por: Ge, Xuyang, et al.
Publicado: (2025)

In-Memory Learning: A Declarative Learning Framework for Large Language Models
por: Wang, Bo, et al.
Publicado: (2024)

Can Language Models Learn to Skip Steps?
por: Liu, Tengxiao, et al.
Publicado: (2024)

Case2Code: Scalable Synthetic Data for Code Generation
por: Shao, Yunfan, et al.
Publicado: (2024)

Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
por: Zhang, Yechen, et al.
Publicado: (2026)

Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
por: Wang, Junxuan, et al.
Publicado: (2024)

Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
por: Li, Jinsong, et al.
Publicado: (2025)

The Open-World Lottery Ticket Hypothesis for OOD Intent Classification
por: Zhou, Yunhua, et al.
Publicado: (2022)

Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences
por: Liu, Xiangyang, et al.
Publicado: (2025)

Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?
por: Zeng, Zhiyuan, et al.
Publicado: (2025)

Thus Spake Long-Context Large Language Model
por: Liu, Xiaoran, et al.
Publicado: (2025)

LoKI: Low-damage Knowledge Implanting of Large Language Models
por: Wang, Runyu, et al.
Publicado: (2025)

FastMCTS: A Simple Sampling Strategy for Data Synthesis
por: Li, Peiji, et al.
Publicado: (2025)

Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia
por: Zhou, Zhejian, et al.
Publicado: (2024)

AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data
por: Song, Zifan, et al.
Publicado: (2024)

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
por: Yin, Zhangyue, et al.
Publicado: (2024)

MLP Memory: A Retriever-Pretrained Memory for Large Language Models
por: Wei, Rubin, et al.
Publicado: (2025)

Training-Free Long-Context Scaling of Large Language Models
por: An, Chenxin, et al.
Publicado: (2024)