Imagen de Portada

Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhang, Yifan, Du, Wenyu, Jin, Dongming, Fu, Jie, Jin, Zhi
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer
por: Zhang, Yifan, et al.
Publicado: (2026)

Iteration Head: A Mechanistic Study of Chain-of-Thought
por: Cabannes, Vivien, et al.
Publicado: (2024)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
por: Zhang, Xuan, et al.
Publicado: (2024)

Evolving Demonstration Optimization for Chain-of-Thought Feature Transformation
por: Wang, Xinyuan, et al.
Publicado: (2026)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
por: Mu, Yongyu, et al.
Publicado: (2025)

When Chain-of-Thought Fails, the Solution Hides in the Hidden States
por: Mehrafarin, Houman, et al.
Publicado: (2026)

Learning from Failures in Multi-Attempt Reinforcement Learning
por: Chung, Stephen, et al.
Publicado: (2025)

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study
por: Xu, Yang, et al.
Publicado: (2024)

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
por: Bian, Tingcheng, et al.
Publicado: (2026)

Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
por: Li, Hongkang, et al.
Publicado: (2024)

Rethinking Regularization Methods for Knowledge Graph Completion
por: Li, Linyu, et al.
Publicado: (2025)

The Expressive Power of Transformers with Chain of Thought
por: Merrill, William, et al.
Publicado: (2023)

Compositional Reasoning with Transformers, RNNs, and Chain of Thought
por: Yehudai, Gilad, et al.
Publicado: (2025)

Chain of Thought Explanation for Dialogue State Tracking
por: Xu, Lin, et al.
Publicado: (2024)

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
por: Wen, Kaiyue, et al.
Publicado: (2024)

Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding
por: Li, Linyu, et al.
Publicado: (2025)

IntentCoding: Amplifying User Intent in Code Generation
por: Fang, Zheng, et al.
Publicado: (2026)

Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
por: Lu, Wenquan, et al.
Publicado: (2025)

Value-Guided Search for Efficient Chain-of-Thought Reasoning
por: Wang, Kaiwen, et al.
Publicado: (2025)

The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought
por: Brösamle, Moritz, et al.
Publicado: (2026)

Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
por: Li, Zihao, et al.
Publicado: (2025)

Constraint-Rectified Training for Efficient Chain-of-Thought
por: Wu, Qinhang, et al.
Publicado: (2026)

Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment
por: Zhang, Yunfan, et al.
Publicado: (2025)

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
por: Li, Xintong, et al.
Publicado: (2026)

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
por: Jin, Bowen, et al.
Publicado: (2024)

Automata Extraction from Transformers
por: Zhang, Yihao, et al.
Publicado: (2024)

OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification
por: Wu, Zijian, et al.
Publicado: (2025)

Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL
por: Yao, Jiarui, et al.
Publicado: (2025)

CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought
por: Zhang, Boxuan, et al.
Publicado: (2025)

Tracking Equivalent Mechanistic Interpretations Across Neural Networks
por: Sun, Alan, et al.
Publicado: (2026)

Demystifying Long Chain-of-Thought Reasoning in LLMs
por: Yeo, Edward, et al.
Publicado: (2025)

Understanding Hidden Computations in Chain-of-Thought Reasoning
por: Bharadwaj, Aryasomayajula Ram
Publicado: (2024)

Reliable Chain-of-Thought via Prefix Consistency
por: Iwase, Naoto, et al.
Publicado: (2026)

A Formal Comparison Between Chain of Thought and Latent Thought
por: Xu, Kevin, et al.
Publicado: (2025)

Mechanistic Interpretability of Binary and Ternary Transformers
por: Li, Jason
Publicado: (2024)

When More is Less: Understanding Chain-of-Thought Length in LLMs
por: Wu, Yuyang, et al.
Publicado: (2025)

On the Diagram of Thought
por: Zhang, Yifan, et al.
Publicado: (2024)

Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
por: Zaman, Kerem, et al.
Publicado: (2025)

Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning
por: Gu, Yu, et al.
Publicado: (2026)

Diable: Efficient Dialogue State Tracking as Operations on Tables
por: Lesci, Pietro, et al.
Publicado: (2023)