Imagen de Portada

Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Shenran, Tse, Timothy Tin-Long, Zhu, Jian
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Developing multilingual speech synthesis system for Ojibwe, Mi'kmaq, and Maliseet
por: Wang, Shenran, et al.
Publicado: (2025)

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
por: Bajaj, Anooshka, et al.
Publicado: (2025)

TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding
por: Wu, Bingheng, et al.
Publicado: (2025)

Advancements in Natural Language Processing: Exploring Transformer-Based Architectures for Text Understanding
por: Wu, Tianhao, et al.
Publicado: (2025)

Understanding Transformers via N-gram Statistics
por: Nguyen, Timothy
Publicado: (2024)

Moving Beyond Next-Token Prediction: Transformers are Context-Sensitive Language Generators
por: Rhee, Phill Kyu
Publicado: (2025)

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
por: Nie, Ercong, et al.
Publicado: (2024)

Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long Contexts
por: Chen, Yingfa, et al.
Publicado: (2026)

Hybrid CNN-Transformer Architecture for Arabic Speech Emotion Recognition
por: Gheffari, Youcef Soufiane, et al.
Publicado: (2026)

Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning
por: Ling Team, et al.
Publicado: (2025)

Identifying Semantic Induction Heads to Understand In-Context Learning
por: Ren, Jie, et al.
Publicado: (2024)

Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs
por: Zhang, Yifang, et al.
Publicado: (2025)

Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?
por: Rawat, Shivam, et al.
Publicado: (2026)

Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models
por: Sakib, Fardin Ahsan, et al.
Publicado: (2025)

Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps
por: Liu, Fuxiao, et al.
Publicado: (2023)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
por: Li, Jiaqi, et al.
Publicado: (2023)

Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning
por: Niu, Jingcheng, et al.
Publicado: (2025)

A Survey on Transformer Context Extension: Approaches and Evaluation
por: Liu, Yijun, et al.
Publicado: (2025)

Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM
por: Yuen, Robin Shing-Hei, et al.
Publicado: (2024)

In-Context Learning State Vector with Inner and Momentum Optimization
por: Li, Dongfang, et al.
Publicado: (2024)

Self-Taught Agentic Long Context Understanding
por: Zhuang, Yufan, et al.
Publicado: (2025)

Irony Detection, Reasoning and Understanding in Zero-shot Learning
por: Yi, Peiling, et al.
Publicado: (2025)

Beyond Plain Demos: A Demo-centric Anchoring Paradigm for In-Context Learning in Alzheimer's Disease Detection
por: Su, Puzhen, et al.
Publicado: (2025)

Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
por: Neo, Clement, et al.
Publicado: (2024)

ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks
por: Van Huynh, Tin, et al.
Publicado: (2026)

FocusLLM: Precise Understanding of Long Context by Dynamic Condensing
por: Li, Zhenyu, et al.
Publicado: (2024)

Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence
por: Minegishi, Gouki, et al.
Publicado: (2025)

Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
por: Bigelow, Eric, et al.
Publicado: (2026)

PICASO: Permutation-Invariant Context Composition with State Space Models
por: Liu, Tian Yu, et al.
Publicado: (2025)

TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understanding
por: Xu, Boshen, et al.
Publicado: (2025)

Conversation Kernels: A Flexible Mechanism to Learn Relevant Context for Online Conversation Understanding
por: Agarwal, Vibhor, et al.
Publicado: (2025)

Reinforced Context Order Recovery for Adaptive Reasoning and Planning
por: Ma, Long, et al.
Publicado: (2025)

Do Large Language Models Understand Logic or Just Mimick Context?
por: Yan, Junbing, et al.
Publicado: (2024)

HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
por: Zhuo, Zhijian, et al.
Publicado: (2025)

Understanding Emergent In-Context Learning from a Kernel Regression Perspective
por: Han, Chi, et al.
Publicado: (2023)

Text Understanding and Generation Using Transformer Models for Intelligent E-commerce Recommendations
por: Xiang, Yafei, et al.
Publicado: (2024)

Hybrid Quantum-Classical Selective State Space Artificial Intelligence
por: Ebrahimi, Amin, et al.
Publicado: (2025)

Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning
por: Wang, Jianing, et al.
Publicado: (2023)

In-Context Learning for Preserving Patient Privacy: A Framework for Synthesizing Realistic Patient Portal Messages
por: Gatto, Joseph, et al.
Publicado: (2024)

Hierarchical Resolution Transformers: A Wavelet-Inspired Architecture for Multi-Scale Language Understanding
por: Sar, Ayan, et al.
Publicado: (2025)