Image de couverture de livre

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Guo, Zhenyu, Chen, Wenguang
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
par: Chen, Yilong, et autres
Publié: (2024)

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention
par: Tian, Yuandong, et autres
Publié: (2023)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
par: Ling Team, et autres
Publié: (2025)

Advancing General-Purpose Reasoning Models with Modular Gradient Surgery
par: Cai, Min, et autres
Publié: (2026)

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
par: Choi, Sehyun
Publié: (2024)

Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?
par: Yin, Yutong, et autres
Publié: (2025)

FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning
par: Luo, Haozheng, et autres
Publié: (2026)

ASTRA: Adaptive Semantic Tree Reasoning Architecture for Complex Table Question Answering
par: Guo, Xiaoke, et autres
Publié: (2026)

Integrating Locality-Aware Attention with Transformers for General Geometry PDEs
par: Koh, Minsu, et autres
Publié: (2025)

METHOD: Modular Efficient Transformer for Health Outcome Discovery
par: Qian, Linglong, et autres
Publié: (2025)

RecurFormer: Not All Transformer Heads Need Self-Attention
par: Yan, Ruiqing, et autres
Publié: (2024)

Scaling Reasoning without Attention
par: Zhao, Xueliang, et autres
Publié: (2025)

From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
par: Ildiz, M. Emrullah, et autres
Publié: (2024)

A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs
par: Hettige, Kethmi Hirushini, et autres
Publié: (2025)

MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
par: Xi, Ningyuan, et autres
Publié: (2024)

DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection
par: Yu, Xiao, et autres
Publié: (2023)

Token-Budget-Aware LLM Reasoning
par: Han, Tingxu, et autres
Publié: (2024)

Can Post-Training Transform LLMs into Causal Reasoners?
par: Chen, Junqi, et autres
Publié: (2026)

Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
par: Kohli, Harsh, et autres
Publié: (2026)

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing
par: Tang, Yang, et autres
Publié: (2025)

Knowledge Fusion of Large Language Models Via Modular SkillPacks
par: Du, Guodong, et autres
Publié: (2025)

Selective Attention Improves Transformer
par: Leviathan, Yaniv, et autres
Publié: (2024)

Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers
par: Chen, Lei, et autres
Publié: (2024)

Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
par: Ma, Zhengzhao, et autres
Publié: (2026)

Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
par: Knupp, Jonas, et autres
Publié: (2026)

Logical Structure as Knowledge: Enhancing LLM Reasoning via Structured Logical Knowledge Density Estimation
par: Bi, Zhen, et autres
Publié: (2025)

R$^2$PO: Decoupling Training Trajectories from Inference Responses for LLM Reasoning
par: Wang, Jingchu, et autres
Publié: (2026)

DASH: Fast Differentiable Architecture Search for Hybrid Attention in Minutes on a Single GPU
par: Chen, Weizhe, et autres
Publié: (2026)

Agents Thinking Fast and Slow: A Talker-Reasoner Architecture
par: Christakopoulou, Konstantina, et autres
Publié: (2024)

Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process
par: Zhang, Zhenyu, et autres
Publié: (2025)

DEBATE, TRAIN, EVOLVE: Self Evolution of Language Model Reasoning
par: Srivastava, Gaurav, et autres
Publié: (2025)

Power Transformer Fault Prediction Based on Knowledge Graphs
par: Wang, Chao, et autres
Publié: (2024)

Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
par: Vendrell, Victor Conchello, et autres
Publié: (2026)

Supernova: Achieving More with Less in Transformer Architectures
par: Tanase, Andrei-Valentin, et autres
Publié: (2025)

Residual Stream Duality in Modern Transformer Architectures
par: Zhang, Yifan
Publié: (2026)

Counterfactual Reasoning with Knowledge Graph Embeddings
par: Zellinger, Lena, et autres
Publié: (2024)

What Matters in Transformers? Not All Attention is Needed
par: He, Shwai, et autres
Publié: (2024)

The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning
par: Aghajohari, Milad, et autres
Publié: (2025)

MoBayes: A Modular Bayesian Framework for Separating Reasoning from Language in Conversational Clinical Decision Support
par: Kesmen, Yusuf, et autres
Publié: (2026)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
par: Chen, Yingfa, et autres
Publié: (2026)