Image de couverture de livre

Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Huang, Jianhao, Wang, Zixuan, Lee, Jason D.
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?
par: Gatmiry, Khashayar, et autres
Publié: (2024)

Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent
par: Chen, Bo, et autres
Publié: (2024)

Multi-head Transformers Provably Learn Symbolic Multi-step Reasoning via Gradient Descent
par: Yang, Tong, et autres
Publié: (2025)

Elastic Multi-Gradient Descent for Parallel Continual Learning
par: Lyu, Fan, et autres
Publié: (2024)

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
par: Huang, Sili, et autres
Publié: (2024)

Conflict-Averse Gradient Descent for Multi-task Learning
par: Liu, Bo, et autres
Publié: (2021)

How Transformers Learn to Plan via Multi-Token Prediction
par: Huang, Jianhao, et autres
Publié: (2026)

The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?
par: Pengmei, Zihan, et autres
Publié: (2025)

Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillation
par: Kim, Juno, et autres
Publié: (2025)

Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization
par: Huang, Yu, et autres
Publié: (2025)

Do pretrained Transformers Learn In-Context by Gradient Descent?
par: Shen, Lingfeng, et autres
Publié: (2023)

Gradient Descent Algorithm Survey
par: Fucheng, Deng, et autres
Publié: (2025)

Learning Associative Memories with Gradient Descent
par: Cabannes, Vivien, et autres
Publié: (2024)

Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought
par: Ildiz, Muhammed Emrullah, et autres
Publié: (2026)

Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
par: Lu, Wenquan, et autres
Publié: (2025)

The Initialization Determines Whether In-Context Learning Is Gradient Descent
par: Xie, Shifeng, et autres
Publié: (2025)

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization
par: Xu, Mingjing, et autres
Publié: (2024)

Evolving Demonstration Optimization for Chain-of-Thought Feature Transformation
par: Wang, Xinyuan, et autres
Publié: (2026)

Stochastic Gradient Descent with Momentum is Algorithmically Stable
par: Lei, Yunwen, et autres
Publié: (2026)

Finite-Time Analysis of Gradient Descent for Shallow Transformers
par: Arda, Enes, et autres
Publié: (2026)

Compositional Reasoning with Transformers, RNNs, and Chain of Thought
par: Yehudai, Gilad, et autres
Publié: (2025)

Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
par: Garg, Ishir, et autres
Publié: (2026)

Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
par: Yang, Shan, et autres
Publié: (2026)

Chain-of-Thought Predictive Control
par: Jia, Zhiwei, et autres
Publié: (2023)

E-CGL: An Efficient Continual Graph Learner
par: Guo, Jianhao, et autres
Publié: (2024)

Randomness and Interpolation Improve Gradient Descent
par: Li, Jiawen, et autres
Publié: (2025)

ONG: Orthogonal Natural Gradient Descent
par: Yadav, Yajat, et autres
Publié: (2025)

Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training
par: Ma, Yuhan, et autres
Publié: (2024)

To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers
par: Xu, Kevin, et autres
Publié: (2025)

Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descent
par: Kumar, Mohit, et autres
Publié: (2024)

GradTree: Learning Axis-Aligned Decision Trees with Gradient Descent
par: Marton, Sascha, et autres
Publié: (2023)

Dynamic Chain-of-Thought: Towards Adaptive Deep Reasoning
par: Wang, Libo
Publié: (2025)

Adaptive Heavy-Tailed Stochastic Gradient Descent
par: Gong, Bodu, et autres
Publié: (2025)

Vanilla Gradient Descent for Oblique Decision Trees
par: Panda, Subrat Prasad, et autres
Publié: (2024)

Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO
par: Yu, Bowen, et autres
Publié: (2026)

DAG-Math: Graph-of-Thought Guided Mathematical Reasoning in LLMs
par: Zhang, Yuanhe, et autres
Publié: (2025)

Ehrenfeucht-Haussler Rank and Chain of Thought
par: Barceló, Pablo, et autres
Publié: (2025)

Activation Steering for Chain-of-Thought Compression
par: Azizi, Seyedarmin, et autres
Publié: (2025)

Geodesic Gradient Descent: A Generic and Learning-rate-free Optimizer on Objective Function-induced Manifolds
par: Hu, Liwei, et autres
Publié: (2026)

Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers
par: Huang, Xingyue, et autres
Publié: (2025)