Cover Image

Residual Stream Duality in Modern Transformer Architectures

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Zhang, Yifan
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modeling
by: Kerce, J. Clayton, et al.
Published: (2026)

Momentum Streams for Optimizer-Inspired Transformers
by: Gai, Jingchu, et al.
Published: (2026)

The Diffusion Duality
by: Sahoo, Subham Sekhar, et al.
Published: (2025)

Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering
by: Gupta, Manan, et al.
Published: (2026)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
by: Xiao, Da, et al.
Published: (2025)

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture
by: Pugh, Samuel L, et al.
Published: (2026)

Supernova: Achieving More with Less in Transformer Architectures
by: Tanase, Andrei-Valentin, et al.
Published: (2025)

M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference
by: Bhendawade, Nikhil, et al.
Published: (2025)

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
by: Kim, Jeonghoon, et al.
Published: (2025)

State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence
by: Aviss, Thea
Published: (2025)

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
by: Choi, Sehyun
Published: (2024)

Training and Evaluating Language Models with Template-based Data Generation
by: Zhang, Yifan
Published: (2024)

A Markov Categorical Framework for Language Modeling
by: Zhang, Yifan
Published: (2025)

Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams
by: Llorente-Saguer, Isaac
Published: (2026)

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
by: Guo, Zhenyu, et al.
Published: (2025)

Zero-Training Temporal Drift Detection for Transformer Sentiment Models: A Comprehensive Analysis on Authentic Social Media Streams
by: Bansal, Aayam, et al.
Published: (2025)

What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers
by: Jacopin, Éric
Published: (2026)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
by: Zhu, Shiyi, et al.
Published: (2023)

Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
by: Gu, Xiaojie, et al.
Published: (2026)

Neutral Residues: Revisiting Adapters for Model Extension
by: Talla, Franck Signe, et al.
Published: (2024)

Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts
by: Zhang, Yifan, et al.
Published: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
by: Chen, Bo, et al.
Published: (2024)

Augmenting Math Word Problems via Iterative Question Composing
by: Liu, Haoxiong, et al.
Published: (2024)

Patent Language Model Pretraining with ModernBERT
by: Yousefiramandi, Amirhossein, et al.
Published: (2025)

Stream of Search (SoS): Learning to Search in Language
by: Gandhi, Kanishk, et al.
Published: (2024)

SR-TTT: Surprisal-Aware Residual Test-Time Training
by: P, Swamynathan V
Published: (2026)

The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams
by: Llorente-Saguer, Isaac
Published: (2026)

Jet Expansions of Residual Computation
by: Chen, Yihong, et al.
Published: (2024)

A Transformer-based Neural Architecture Search Method
by: Wang, Shang, et al.
Published: (2025)

Residual Matrix Transformers: Scaling the Size of the Residual Stream
by: Mak, Brian, et al.
Published: (2025)

DINT Transformer
by: Cang, Yueyang, et al.
Published: (2025)

Speculative Streaming: Fast LLM Inference without Auxiliary Models
by: Bhendawade, Nikhil, et al.
Published: (2024)

Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
by: Paglieri, Davide, et al.
Published: (2024)

Clinical ModernBERT: An efficient and long context encoder for biomedical text
by: Lee, Simon A., et al.
Published: (2025)

Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors
by: Zhou, Ying, et al.
Published: (2024)

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
by: Wang, Xinyan, et al.
Published: (2026)

Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction
by: Lou, Hantao, et al.
Published: (2025)

Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning
by: Yuan, Yurun, et al.
Published: (2025)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
by: Tamayo, Daniel, et al.
Published: (2026)

AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements
by: Bora, Adriana Eufrosina, et al.
Published: (2025)