Image de couverture de livre

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhuo, Zhijian, Zeng, Yutao, Wang, Ya, Zhang, Sijun, Yang, Jian, Li, Xiaoqing, Zhou, Xun, Ma, Jinwen
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
par: Zhuo, Zhijian, et autres
Publié: (2024)

Scale-Distribution Decoupling: Enabling Stable and Effective Training of Large Language Models
par: Wang, Ya, et autres
Publié: (2025)

Efficient Pretraining Length Scaling
par: Wu, Bohong, et autres
Publié: (2025)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
par: Wang, Chao, et autres
Publié: (2026)

Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures
par: Wang, Shenran, et autres
Publié: (2025)

From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors
par: Zhou, Yitian, et autres
Publié: (2026)

Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximation
par: Yusuyin, Saierdaer, et autres
Publié: (2025)

Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
par: NVIDIA, et autres
Publié: (2025)

Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling
par: Huang, Hongzhi, et autres
Publié: (2025)

Distilling to Hybrid Attention Models via KL-Guided Layer Selection
par: Li, Yanhong, et autres
Publié: (2025)

Boosting Summarization with Normalizing Flows and Aggressive Training
par: Yang, Yu, et autres
Publié: (2023)

TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understanding
par: Xu, Boshen, et autres
Publié: (2025)

NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
par: NVIDIA, et autres
Publié: (2025)

HNCSE: Advancing Sentence Embeddings via Hybrid Contrastive Learning with Hard Negatives
par: Liu, Wenxiao, et autres
Publié: (2024)

Hybrid Quantum Transformer for Language Generation
par: Kong, Desheng, et autres
Publié: (2025)

Towards Efficient Post-Training via Fourier-Driven Adapter Architectures
par: Bae, Donggyun, et autres
Publié: (2025)

Frac-Connections: Fractional Extension of Hyper-Connections
par: Zhu, Defa, et autres
Publié: (2025)

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs
par: Yue, Chongjian, et autres
Publié: (2023)

Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
par: NVIDIA, et autres
Publié: (2026)

Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
par: NVIDIA, et autres
Publié: (2025)

HES-SQL: Hybrid Reasoning for Efficient Text-to-SQL with Structural Skeleton Guidance
par: Qiu, Suming, et autres
Publié: (2025)

Affine-Scaled Attention: Towards Flexible and Stable Transformer Attention
par: Bae, Jeongin, et autres
Publié: (2026)

Hybrid CNN-Transformer Architecture for Arabic Speech Emotion Recognition
par: Gheffari, Youcef Soufiane, et autres
Publié: (2026)

Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training
par: Huang, Junqin, et autres
Publié: (2024)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
par: Liu, Jun, et autres
Publié: (2024)

Native Hybrid Attention for Efficient Sequence Modeling
par: Du, Jusen, et autres
Publié: (2025)

Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models
par: Byun, Hoyoon, et autres
Publié: (2025)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
par: Ling Team, et autres
Publié: (2025)

NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization
par: Nahid, Md Mahadi Hasan, et autres
Publié: (2024)

Hybrid-SQuAD: Hybrid Scholarly Question Answering Dataset
par: Taffa, Tilahun Abedissa, et autres
Publié: (2024)

H-Mem: A Novel Memory Mechanism for Evolving and Retrieving Agent Memory via a Hybrid Structure
par: Yu, Jiawei, et autres
Publié: (2026)

Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents
par: Shao, Chenyang, et autres
Publié: (2025)

DuCCAE: A Hybrid Engine for Immersive Conversation via Collaboration, Augmentation, and Evolution
par: Shen, Xin, et autres
Publié: (2026)

Detecting AI-Generated Sentences in Human-AI Collaborative Hybrid Texts: Challenges, Strategies, and Insights
par: Zeng, Zijie, et autres
Publié: (2024)

Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
par: Luo, Haotian, et autres
Publié: (2025)

Heterogeneous Subgraph Transformer for Fake News Detection
par: Zhang, Yuchen, et autres
Publié: (2024)

A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning
par: Chen, Qianben, et autres
Publié: (2025)

Efficient Training for Cross-lingual Speech Language Models
par: Zhou, Yan, et autres
Publié: (2026)

TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
par: Wu, Bingheng, et autres
Publié: (2025)

SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifolds
par: Cheng, Wuxinlin, et autres
Publié: (2025)