Imagen de Portada

Residual Stream Duality in Modern Transformer Architectures

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Zhang, Yifan
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modeling
por: Kerce, J. Clayton, et al.
Publicado: (2026)

Momentum Streams for Optimizer-Inspired Transformers
por: Gai, Jingchu, et al.
Publicado: (2026)

The Diffusion Duality
por: Sahoo, Subham Sekhar, et al.
Publicado: (2025)

Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering
por: Gupta, Manan, et al.
Publicado: (2026)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
por: Xiao, Da, et al.
Publicado: (2025)

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture
por: Pugh, Samuel L, et al.
Publicado: (2026)

Supernova: Achieving More with Less in Transformer Architectures
por: Tanase, Andrei-Valentin, et al.
Publicado: (2025)

M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference
por: Bhendawade, Nikhil, et al.
Publicado: (2025)

Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
por: Kim, Jeonghoon, et al.
Publicado: (2025)

State Stream Transformer (SST) : Emergent Metacognitive Behaviours Through Latent State Persistence
por: Aviss, Thea
Publicado: (2025)

Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
por: Choi, Sehyun
Publicado: (2024)

Training and Evaluating Language Models with Template-based Data Generation
por: Zhang, Yifan
Publicado: (2024)

A Markov Categorical Framework for Language Modeling
por: Zhang, Yifan
Publicado: (2025)

Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams
por: Llorente-Saguer, Isaac
Publicado: (2026)

Decoupling Knowledge and Reasoning in Transformers: A Modular Architecture with Generalized Cross-Attention
por: Guo, Zhenyu, et al.
Publicado: (2025)

Zero-Training Temporal Drift Detection for Transformer Sentiment Models: A Comprehensive Analysis on Authentic Social Media Streams
por: Bansal, Aayam, et al.
Publicado: (2025)

What Is the Minimum Architecture for Prolepsis? Early Irrevocable Commitment Across Tasks in Small Transformers
por: Jacopin, Éric
Publicado: (2026)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
por: Zhu, Shiyi, et al.
Publicado: (2023)

Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
por: Gu, Xiaojie, et al.
Publicado: (2026)

Neutral Residues: Revisiting Adapters for Model Extension
por: Talla, Franck Signe, et al.
Publicado: (2024)

Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts
por: Zhang, Yifan, et al.
Publicado: (2024)

Circuit Complexity Bounds for RoPE-based Transformer Architecture
por: Chen, Bo, et al.
Publicado: (2024)

Augmenting Math Word Problems via Iterative Question Composing
por: Liu, Haoxiong, et al.
Publicado: (2024)

Patent Language Model Pretraining with ModernBERT
por: Yousefiramandi, Amirhossein, et al.
Publicado: (2025)

Stream of Search (SoS): Learning to Search in Language
por: Gandhi, Kanishk, et al.
Publicado: (2024)

SR-TTT: Surprisal-Aware Residual Test-Time Training
por: P, Swamynathan V
Publicado: (2026)

The Geometry of Harmful Intent: Training-Free Anomaly Detection via Angular Deviation in LLM Residual Streams
por: Llorente-Saguer, Isaac
Publicado: (2026)

Jet Expansions of Residual Computation
por: Chen, Yihong, et al.
Publicado: (2024)

A Transformer-based Neural Architecture Search Method
por: Wang, Shang, et al.
Publicado: (2025)

Residual Matrix Transformers: Scaling the Size of the Residual Stream
por: Mak, Brian, et al.
Publicado: (2025)

DINT Transformer
por: Cang, Yueyang, et al.
Publicado: (2025)

Speculative Streaming: Fast LLM Inference without Auxiliary Models
por: Bhendawade, Nikhil, et al.
Publicado: (2024)

Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMs
por: Paglieri, Davide, et al.
Publicado: (2024)

Clinical ModernBERT: An efficient and long context encoder for biomedical text
por: Lee, Simon A., et al.
Publicado: (2025)

Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors
por: Zhou, Ying, et al.
Publicado: (2024)

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
por: Wang, Xinyan, et al.
Publicado: (2026)

Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction
por: Lou, Hantao, et al.
Publicado: (2025)

Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning
por: Yuan, Yurun, et al.
Publicado: (2025)

MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptation
por: Tamayo, Daniel, et al.
Publicado: (2026)

AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements
por: Bora, Adriana Eufrosina, et al.
Publicado: (2025)