Cover Image

TransXSSM: A Hybrid Transformer State Space Model with Unified Rotary Position Embedding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wu, Bingheng, Shi, Jingze, Wu, Yifan, Tang, Nan, Luo, Yuyu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architecture
by: Shi, Jingze, et al.
Published: (2024)

Trainable Dynamic Mask Sparse Attention
by: Shi, Jingze, et al.
Published: (2025)

OTCE: Hybrid SSM and Attention with Cross Domain Mixture of Experts to construct Observer-Thinker-Conceiver-Expresser
by: Shi, Jingze, et al.
Published: (2024)

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale
by: Shi, Jingze, et al.
Published: (2026)

Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting
by: Wu, Yifan, et al.
Published: (2025)

ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answering
by: Wu, Yifan, et al.
Published: (2024)

Benchmarking Rotary Position Embeddings for Automatic Speech Recognition
by: Zhang, Shucong, et al.
Published: (2025)

Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers
by: Liu, Feilong
Published: (2026)

Are Large Language Models Good Statisticians?
by: Zhu, Yizhang, et al.
Published: (2024)

Length-Aware Rotary Position Embedding for Text-Speech Alignment
by: Kim, Hyeongju, et al.
Published: (2025)

EXCLAIM: An Explainable Cross-Modal Agentic System for Misinformation Detection with Hierarchical Retrieval
by: Wu, Yin, et al.
Published: (2025)

ChartCards: A Chart-Metadata Generation Framework for Multi-Task Chart Understanding
by: Wu, Yifan, et al.
Published: (2025)

ReCode: Unify Plan and Action for Universal Granularity Control
by: Yu, Zhaoyang, et al.
Published: (2025)

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
by: Zhu, Shiyi, et al.
Published: (2023)

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models
by: Dai, Chang, et al.
Published: (2025)

EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing
by: Zhu, Yizhang, et al.
Published: (2025)

Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks
by: Shi, Jingze, et al.
Published: (2024)

Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures
by: Wang, Shenran, et al.
Published: (2025)

Cross-Axis Transformer with 3D Rotary Positional Embeddings
by: Erickson, Lily
Published: (2023)

nvBench 2.0: Resolving Ambiguity in Text-to-Visualization through Stepwise Reasoning
by: Luo, Tianqi, et al.
Published: (2025)

Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
by: Liang, Zhuowen, et al.
Published: (2026)

AskChart: Universal Chart Understanding through Textual Enhancement
by: Yang, Xudong, et al.
Published: (2024)

Unifying Large Language Models and Knowledge Graphs: A Roadmap
by: Pan, Shirui, et al.
Published: (2023)

Piccolo2: General Text Embedding with Multi-task Hybrid Loss Training
by: Huang, Junqin, et al.
Published: (2024)

Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
by: Zhang, Zhibo, et al.
Published: (2025)

Developing Healthcare Language Model Embedding Spaces
by: Taylor, Niall, et al.
Published: (2024)

PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead
by: Tan, Tao, et al.
Published: (2024)

Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
by: Li, Li, et al.
Published: (2025)

Beyond Semantics: How Temporal Biases Shape Retrieval in Transformer and State-Space Models
by: Bajaj, Anooshka, et al.
Published: (2025)

Bi-Mamba: Towards Accurate 1-Bit State Space Models
by: Tang, Shengkun, et al.
Published: (2024)

Atom of Thoughts for Markov LLM Test-Time Scaling
by: Teng, Fengwei, et al.
Published: (2025)

Fine-Tuned Language Models for Domain-Specific Summarization and Tagging
by: Wang, Jun, et al.
Published: (2025)

Conan-Embedding-v2: Training an LLM from Scratch for Text Embeddings
by: Li, Shiyu, et al.
Published: (2025)

ICPC: In-context Prompt Compression with Faster Inference
by: Yu, Ziyang, et al.
Published: (2025)

Quantifying Positional Biases in Text Embedding Models
by: Lee, Reagan J., et al.
Published: (2024)

TransERR: Translation-based Knowledge Graph Embedding via Efficient Relation Rotation
by: Li, Jiang, et al.
Published: (2023)

Fine-Grained Knowledge Structuring and Retrieval for Visual Question Answering
by: Zhang, Zhengxuan, et al.
Published: (2025)

Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings
by: Gelberg, Yoav, et al.
Published: (2025)

Enhancing Safety of Large Language Models via Embedding Space Separation
by: Zhao, Xu, et al.
Published: (2026)

VRoPE: Rotary Position Embedding for Video Large Language Models
by: Liu, Zikang, et al.
Published: (2025)