Copertina

Is It a Free Lunch for Removing Outliers during Pretraining?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Liao, Baohao, Monz, Christof
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability
di: Liao, Baohao, et al.
Pubblicazione: (2024)

Self-Hinting Language Models Enhance Reinforcement Learning
di: Liao, Baohao, et al.
Pubblicazione: (2026)

ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

Reward-Guided Speculative Decoding for Efficient LLM Reasoning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

Fractured Chain-of-Thought Reasoning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectives
di: Xiong, Wei, et al.
Pubblicazione: (2025)

IKUN for WMT24 General MT Task: LLMs Are here for Multilingual Machine Translation
di: Liao, Baohao, et al.
Pubblicazione: (2024)

How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translation
di: Meng, Yan, et al.
Pubblicazione: (2024)

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models
di: Rajaee, Sara, et al.
Pubblicazione: (2024)

Disentangling the Roles of Target-Side Transfer and Regularization in Multilingual Machine Translation
di: Meng, Yan, et al.
Pubblicazione: (2024)

Do Language Models Reason Across Languages?
di: Meng, Yan, et al.
Pubblicazione: (2026)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
di: Liao, Baohao, et al.
Pubblicazione: (2024)

Communicating with Speakers and Listeners of Different Pragmatic Levels
di: Naszadi, Kata, et al.
Pubblicazione: (2024)

On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations?
di: Choenni, Rochelle, et al.
Pubblicazione: (2024)

Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning
di: Rajaee, Sara, et al.
Pubblicazione: (2025)

Asking LLMs to Verify First is Almost Free Lunch
di: Wu, Shiguang, et al.
Pubblicazione: (2025)

Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models
di: Lamont, Sean, et al.
Pubblicazione: (2026)

Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness
di: Bi, Baolong, et al.
Pubblicazione: (2024)

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
di: Qin, Zhen, et al.
Pubblicazione: (2024)

Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
di: Troshin, Sergey, et al.
Pubblicazione: (2025)

No Free Lunch in Active Learning: LLM Embedding Quality Dictates Query Strategy Success
di: Rauch, Lukas, et al.
Pubblicazione: (2025)

No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biases
di: Chand, Shireen, et al.
Pubblicazione: (2025)

CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
di: Li, Haoran, et al.
Pubblicazione: (2026)

The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models
di: Chen, Xinyi, et al.
Pubblicazione: (2024)

LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation
di: Zhang, Weizhi, et al.
Pubblicazione: (2025)

LiveMathematicianBench: A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches
di: He, Linyang, et al.
Pubblicazione: (2026)

Outliers Dimensions that Disrupt Transformers Are Driven by Frequency
di: Puccetti, Giovanni, et al.
Pubblicazione: (2022)

Outlier Dimensions Encode Task-Specific Knowledge
di: Rudman, William, et al.
Pubblicazione: (2023)

ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection
di: Datta, Debajyoti, et al.
Pubblicazione: (2026)

Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation
di: Vasilev, Stefan, et al.
Pubblicazione: (2025)

OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
di: Gadhikar, Advait, et al.
Pubblicazione: (2025)

Lost at the Beginning of Reasoning
di: Liao, Baohao, et al.
Pubblicazione: (2025)

Rethinking the Outlier Distribution in Large Language Models: An In-depth Study
di: Raman, Rahul, et al.
Pubblicazione: (2025)

From Noise to Signal: When Outliers Seed New Topics
di: Zve, Evangelia, et al.
Pubblicazione: (2026)

Identifying Narrative Patterns and Outliers in Holocaust Testimonies Using Topic Modeling
di: Ifergan, Maxim, et al.
Pubblicazione: (2024)

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models
di: Huang, Yukun, et al.
Pubblicazione: (2025)

CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large Language Models
di: Liu, Guang, et al.
Pubblicazione: (2025)

Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models
di: Wu, Wei, et al.
Pubblicazione: (2026)

Systematic Outliers in Large Language Models
di: An, Yongqi, et al.
Pubblicazione: (2025)

AI Managed Emergency Documentation with a Pretrained Model
di: Menzies, David, et al.
Pubblicazione: (2024)