Image de couverture de livre

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Chao, Li, Bei, Zhang, Jiaqi, Liu, Xinyu, Fan, Yuchun, Lyu, Linkun, Chen, Xin, Wang, Jingang, Xiao, Tong, Pei, Peng, Cai, Xunliang
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
par: Fan, Yuchun, et autres
Publié: (2026)

Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning
par: Li, Bei, et autres
Publié: (2024)

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
par: Liu, Jiahao, et autres
Publié: (2024)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
par: Tang, Hongyin, et autres
Publié: (2024)

GradPower: Powering Gradients for Faster Language Model Pre-Training
par: Wang, Jinbo, et autres
Publié: (2025)

MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks
par: Ruan, Junhao, et autres
Publié: (2026)

Causal Autoregressive Diffusion Language Model
par: Ruan, Junhao, et autres
Publié: (2026)

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
par: Shao, Ruichen, et autres
Publié: (2025)

Beyond the Known: Investigating LLMs Performance on Out-of-Domain Intent Detection
par: Wang, Pei, et autres
Publié: (2024)

Scaling Embeddings Outperforms Scaling Experts in Language Models
par: Liu, Hong, et autres
Publié: (2026)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
par: Yuan, Danlong, et autres
Publié: (2024)

FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
par: Xu, Liangyu, et autres
Publié: (2025)

Large-Scale Diverse Synthesis for Mid-Training
par: Zhang, Xuemiao, et autres
Publié: (2025)

OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation
par: Li, Han, et autres
Publié: (2025)

Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
par: Chen, Zhengyu, et autres
Publié: (2025)

Scaling and Transferability of Annealing Strategies in Large Language Model Training
par: Wang, Siqi, et autres
Publié: (2025)

Libra: Assessing and Improving Reward Model by Learning to Think
par: Zhou, Meng, et autres
Publié: (2025)

Dynamic Fisher-weighted Model Merging via Bayesian Optimization
par: Lee, Sanwoo, et autres
Publié: (2025)

IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
par: Liu, Xinyu, et autres
Publié: (2025)

A Survey on LLM Mid-Training
par: Tu, Chengying, et autres
Publié: (2025)

BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search
par: Liu, Shiyu, et autres
Publié: (2026)

Aligning with Human Values to Enhance Interaction: An eHMI-Mediated Lane-Changing Negotiation Strategy Using Bayesian Inference
par: Peng, Boyao, et autres
Publié: (2025)

What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbation
par: Gong, Zhuocheng, et autres
Publié: (2024)

Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces
par: Lyu, Yang, et autres
Publié: (2025)

Deep Learning based Quasi-consciousness Training for Robot Intelligent Model
par: Li, Yuchun, et autres
Publié: (2025)

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection
par: Wang, Pei, et autres
Publié: (2023)

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
par: Xu, Zhihao, et autres
Publié: (2026)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
par: Bai, Yang, et autres
Publié: (2026)

Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models
par: Liu, Xinyu, et autres
Publié: (2024)

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
par: Wang, Lanrui, et autres
Publié: (2025)

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
par: Diao, Muxi, et autres
Publié: (2024)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
par: Zhang, Xuemiao, et autres
Publié: (2025)

A Preliminary Study on the Promises and Challenges of Native Top-$k$ Sparse Attention
par: Xiu, Di, et autres
Publié: (2025)

NDP: Next Distribution Prediction as a More Broad Target
par: Ruan, Junhao, et autres
Publié: (2024)

Refactoring Deep Learning Code: A Study of Practices and Unsatisfied Tool Needs
par: Wang, Siqi, et autres
Publié: (2024)

GIFT: Reconciling Post-Training Objectives via Finite-Temperature Gibbs Initialization
par: Zhao, Zhengyang, et autres
Publié: (2026)

Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspective
par: Kong, Deyang, et autres
Publié: (2025)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
par: Zhang, Xuemiao, et autres
Publié: (2025)

Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
par: Wu, Pengfei, et autres
Publié: (2024)

FIRP: Faster LLM inference via future intermediate representation prediction
par: Wu, Pengfei, et autres
Publié: (2024)