Image de couverture de livre

Disentangling Feature Structure: A Mathematically Provable Two-Stage Training Dynamics in Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Gong, Zixuan, Li, Shijia, Liu, Yong, Teng, Jiaye
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

What Makes Looped Transformers Perform Better Than Non-Recursive Ones
par: Gong, Zixuan, et autres
Publié: (2025)

Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
par: Chen, Siyu, et autres
Publié: (2024)

Two-Stage Regularization-Based Structured Pruning for LLMs
par: Feng, Mingkuan, et autres
Publié: (2025)

Brain-Inspired Two-Stage Approach: Enhancing Mathematical Reasoning by Imitating Human Thought Processes
par: Chen, Yezeng, et autres
Publié: (2024)

GraphDancer: Training LLMs to Explore and Reason over Graphs via Two-Stage Curriculum Post-Training
par: Bai, Yuyang, et autres
Publié: (2026)

2SSP: A Two-Stage Framework for Structured Pruning of LLMs
par: Sandri, Fabrizio, et autres
Publié: (2025)

$Q\sharp$: Provably Optimal Distributional RL for LLM Post-Training
par: Zhou, Jin Peng, et autres
Publié: (2025)

An Investigation of Robustness of LLMs in Mathematical Reasoning: Benchmarking with Mathematically-Equivalent Transformation of Advanced Mathematical Problems
par: Hao, Yuren, et autres
Publié: (2025)

Latent Concept Disentanglement in Transformer-based Language Models
par: Hong, Guan Zhe, et autres
Publié: (2025)

Can A Gamer Train A Mathematical Reasoning Model?
par: Shin, Andrew
Publié: (2025)

Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
par: Yao, Xinhao, et autres
Publié: (2025)

LycheeCluster: Efficient Long-Context Inference with Structure-Aware Chunking and Hierarchical KV Indexing
par: Li, Dongfang, et autres
Publié: (2026)

Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks
par: Liu, Haoyu, et autres
Publié: (2026)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
par: Hu, Junhao, et autres
Publié: (2026)

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
par: Gan, Zeyu, et autres
Publié: (2024)

Evolving Demonstration Optimization for Chain-of-Thought Feature Transformation
par: Wang, Xinyuan, et autres
Publié: (2026)

Improving Word Translation via Two-Stage Contrastive Learning
par: Li, Yaoyiran, et autres
Publié: (2022)

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
par: Mistry, Deven Mahesh, et autres
Publié: (2025)

POET-X: Memory-efficient LLM Training by Scaling Orthogonal Transformation
par: Qiu, Zeju, et autres
Publié: (2026)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
par: Wang, Chao, et autres
Publié: (2026)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
par: Qiu, Zeju, et autres
Publié: (2025)

Provable Scaling Laws for the Test-Time Compute of Large Language Models
par: Chen, Yanxi, et autres
Publié: (2024)

Provable Interactive Learning with Hindsight Instruction Feedback
par: Misra, Dipendra, et autres
Publié: (2024)

CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning
par: Zheng, Congmin, et autres
Publié: (2025)

Time-Aware Feature Selection: Adaptive Temporal Masking for Stable Sparse Autoencoder Training
par: Li, T. Ed, et autres
Publié: (2025)

On the Mathematical Relationship Between Layer Normalization and Dynamic Activation Functions
par: Stollenwerk, Felix
Publié: (2025)

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning
par: Chegini, Atoosa, et autres
Publié: (2026)

STAT: Shrinking Transformers After Training
par: Flynn, Megan, et autres
Publié: (2024)

TSO: Self-Training with Scaled Preference Optimization
par: Chen, Kaihui, et autres
Publié: (2024)

Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
par: Labiad, Ismail, et autres
Publié: (2025)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
par: Yu, Fengming, et autres
Publié: (2025)

DISPO: Enhancing Training Efficiency and Stability in Reinforcement Learning for Large Language Model Mathematical Reasoning
par: Karaman, Batuhan K., et autres
Publié: (2026)

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
par: Lin, Licong, et autres
Publié: (2023)

Provable Length Generalization in Sequence Prediction via Spectral Filtering
par: Marsden, Annie, et autres
Publié: (2024)

Interpretable Steering of Large Language Models with Feature Guided Activation Additions
par: Soo, Samuel, et autres
Publié: (2025)

Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making
par: Hsu, Aliyah R., et autres
Publié: (2023)

Teaching Transformers Causal Reasoning through Axiomatic Training
par: Vashishtha, Aniket, et autres
Publié: (2024)

Can Post-Training Transform LLMs into Causal Reasoners?
par: Chen, Junqi, et autres
Publié: (2026)

Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions
par: Nair, Inderjeet, et autres
Publié: (2024)

Learning Dynamics in Continual Pre-Training for Large Language Models
par: Wang, Xingjin, et autres
Publié: (2025)