Towards Interpretable and Inference-Optimal COT Reasoning with Sparse Autoencoder-Guided Generation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911188748599296 |
|---|---|
| author | Zhao, Daniel Shankarampeta, Abhilash Hu, Lanxiang Rosing, Tajana Zhang, Hao |
| author_facet | Zhao, Daniel Shankarampeta, Abhilash Hu, Lanxiang Rosing, Tajana Zhang, Hao |
| contents | We propose a novel method that leverages sparse autoencoders (SAEs) and clustering techniques to analyze the internal token representations of large language models (LLMs) and guide generations in mathematical reasoning tasks. Our approach first trains an SAE to generate sparse vector representations for training tokens, then applies k-means clustering to construct a graph where vertices represent token clusters and weighted edges capture sequential token transitions. Using this graph, we define an edge-weight based reward function to quantify adherence to established reasoning traces, thereby identifying exploitative reasoning trajectories. Additionally, we measure generation diversity from clustering to assess the extent of exploration. Our findings indicate that balancing both exploitation and exploration is crucial for achieving high accuracy in mathematical reasoning tasks. During generation, the SAE can serve as a scalable reward model to guide generations, ensuring a balanced trade-off between exploitation and exploration. This prevents extreme behaviors in either direction, ultimately fostering a higher-quality reasoning process in LLMs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_01528 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Interpretable and Inference-Optimal COT Reasoning with Sparse Autoencoder-Guided Generation Zhao, Daniel Shankarampeta, Abhilash Hu, Lanxiang Rosing, Tajana Zhang, Hao Artificial Intelligence Machine Learning We propose a novel method that leverages sparse autoencoders (SAEs) and clustering techniques to analyze the internal token representations of large language models (LLMs) and guide generations in mathematical reasoning tasks. Our approach first trains an SAE to generate sparse vector representations for training tokens, then applies k-means clustering to construct a graph where vertices represent token clusters and weighted edges capture sequential token transitions. Using this graph, we define an edge-weight based reward function to quantify adherence to established reasoning traces, thereby identifying exploitative reasoning trajectories. Additionally, we measure generation diversity from clustering to assess the extent of exploration. Our findings indicate that balancing both exploitation and exploration is crucial for achieving high accuracy in mathematical reasoning tasks. During generation, the SAE can serve as a scalable reward model to guide generations, ensuring a balanced trade-off between exploitation and exploration. This prevents extreme behaviors in either direction, ultimately fostering a higher-quality reasoning process in LLMs. |
| title | Towards Interpretable and Inference-Optimal COT Reasoning with Sparse Autoencoder-Guided Generation |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2510.01528 |