Cover Image

Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yifan, Du, Wenyu, Jin, Dongming, Fu, Jie, Jin, Zhi
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer
by: Zhang, Yifan, et al.
Published: (2026)

Iteration Head: A Mechanistic Study of Chain-of-Thought
by: Cabannes, Vivien, et al.
Published: (2024)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
by: Zhang, Xuan, et al.
Published: (2024)

Evolving Demonstration Optimization for Chain-of-Thought Feature Transformation
by: Wang, Xinyuan, et al.
Published: (2026)

Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Study
by: Mu, Yongyu, et al.
Published: (2025)

When Chain-of-Thought Fails, the Solution Hides in the Hidden States
by: Mehrafarin, Houman, et al.
Published: (2026)

Learning from Failures in Multi-Attempt Reinforcement Learning
by: Chung, Stephen, et al.
Published: (2025)

Tracking the Feature Dynamics in LLM Training: A Mechanistic Study
by: Xu, Yang, et al.
Published: (2024)

ExpThink: Experience-Guided Reinforcement Learning for Adaptive Chain-of-Thought Compression
by: Bian, Tingcheng, et al.
Published: (2026)

Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
by: Li, Hongkang, et al.
Published: (2024)

Rethinking Regularization Methods for Knowledge Graph Completion
by: Li, Linyu, et al.
Published: (2025)

The Expressive Power of Transformers with Chain of Thought
by: Merrill, William, et al.
Published: (2023)

Compositional Reasoning with Transformers, RNNs, and Chain of Thought
by: Yehudai, Gilad, et al.
Published: (2025)

Chain of Thought Explanation for Dialogue State Tracking
by: Xu, Lin, et al.
Published: (2024)

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
by: Wen, Kaiyue, et al.
Published: (2024)

Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding
by: Li, Linyu, et al.
Published: (2025)

IntentCoding: Amplifying User Intent in Code Generation
by: Fang, Zheng, et al.
Published: (2026)

Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
by: Lu, Wenquan, et al.
Published: (2025)

Value-Guided Search for Efficient Chain-of-Thought Reasoning
by: Wang, Kaiwen, et al.
Published: (2025)

The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought
by: Brösamle, Moritz, et al.
Published: (2026)

Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models
by: Li, Zihao, et al.
Published: (2025)

Constraint-Rectified Training for Efficient Chain-of-Thought
by: Wu, Qinhang, et al.
Published: (2026)

Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment
by: Zhang, Yunfan, et al.
Published: (2025)

Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
by: Li, Xintong, et al.
Published: (2026)

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
by: Jin, Bowen, et al.
Published: (2024)

Automata Extraction from Transformers
by: Zhang, Yihao, et al.
Published: (2024)

OPV: Outcome-based Process Verifier for Efficient Long Chain-of-Thought Verification
by: Wu, Zijian, et al.
Published: (2025)

Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL
by: Yao, Jiarui, et al.
Published: (2025)

CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought
by: Zhang, Boxuan, et al.
Published: (2025)

Tracking Equivalent Mechanistic Interpretations Across Neural Networks
by: Sun, Alan, et al.
Published: (2026)

Demystifying Long Chain-of-Thought Reasoning in LLMs
by: Yeo, Edward, et al.
Published: (2025)

Understanding Hidden Computations in Chain-of-Thought Reasoning
by: Bharadwaj, Aryasomayajula Ram
Published: (2024)

Reliable Chain-of-Thought via Prefix Consistency
by: Iwase, Naoto, et al.
Published: (2026)

A Formal Comparison Between Chain of Thought and Latent Thought
by: Xu, Kevin, et al.
Published: (2025)

Mechanistic Interpretability of Binary and Ternary Transformers
by: Li, Jason
Published: (2024)

When More is Less: Understanding Chain-of-Thought Length in LLMs
by: Wu, Yuyang, et al.
Published: (2025)

On the Diagram of Thought
by: Zhang, Yifan, et al.
Published: (2024)

Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
by: Zaman, Kerem, et al.
Published: (2025)

Pause and Reflect: Conformal Aggregation for Chain-of-Thought Reasoning
by: Gu, Yu, et al.
Published: (2026)

Diable: Efficient Dialogue State Tracking as Operations on Tables
by: Lesci, Pietro, et al.
Published: (2023)