Cover Image

Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Feng, Aosong, Ying, Rex, Tassiulas, Leandros
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MTBench: A Multimodal Time Series Benchmark for Temporal Reasoning and Question Answering
by: Chen, Jialin, et al.
Published: (2025)

An Item is Worth a Prompt: Versatile Image Editing with Disentangled Control
by: Feng, Aosong, et al.
Published: (2024)

Folding Tensor and Sequence Parallelism for Memory-Efficient Transformer Training & Inference
by: Shyam, Vasu, et al.
Published: (2026)

Efficient High-Resolution Time Series Classification via Attention Kronecker Decomposition
by: Feng, Aosong, et al.
Published: (2024)

Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling
by: Zeghidour, Neil, et al.
Published: (2025)

From Similarity to Superiority: Channel Clustering for Time Series Forecasting
by: Chen, Jialin, et al.
Published: (2024)

Long-range Modeling and Processing of Multimodal Event Sequences
by: Li, Jichu, et al.
Published: (2026)

Neural Sequence-to-Sequence Modeling with Attention by Leveraging Deep Learning Architectures for Enhanced Contextual Understanding in Abstractive Text Summarization
by: Challagundla, Bhavith Chandra, et al.
Published: (2024)

Linear Attention Sequence Parallelism
by: Sun, Weigao, et al.
Published: (2024)

Hyperbolic Fine-Tuning for Large Language Models
by: Yang, Menglin, et al.
Published: (2024)

Star Attention: Efficient LLM Inference over Long Sequences
by: Acharya, Shantanu, et al.
Published: (2024)

Gated Slot Attention for Efficient Linear-Time Sequence Modeling
by: Zhang, Yu, et al.
Published: (2024)

Training Tensor Attention Efficiently: From Cubic to Almost Linear Time
by: Cao, Yang, et al.
Published: (2024)

Subspace Kernel Learning on Tensor Sequences
by: Wang, Lei, et al.
Published: (2026)

MAMI: Multi-Attentional Mutual-Information for Long Sequence Neuron Captioning
by: Fauzulhaq, Alfirsa Damasyifa, et al.
Published: (2024)

Benchmark for Assessing Olfactory Perception of Large Language Models
by: Makri, Eftychia, et al.
Published: (2026)

TRACE: Grounding Time Series in Context for Multimodal Embedding and Retrieval
by: Chen, Jialin, et al.
Published: (2025)

Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
by: He, Ziwei, et al.
Published: (2023)

Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications
by: Maatouk, Ali, et al.
Published: (2024)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
by: Yuan, Danlong, et al.
Published: (2024)

Native Hybrid Attention for Efficient Sequence Modeling
by: Du, Jusen, et al.
Published: (2025)

Tensor Product Attention Is All You Need
by: Zhang, Yifan, et al.
Published: (2025)

How Well Can a Long Sequence Model Model Long Sequences? Comparing Architechtural Inductive Biases on Long-Context Abilities
by: Huang, Jerry
Published: (2024)

CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling
by: Bai, Yu, et al.
Published: (2024)

On Sequence-to-Sequence Models for Automated Log Parsing
by: Sorrenti, Adam, et al.
Published: (2026)

FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
by: Lai, Xunhao, et al.
Published: (2025)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
by: Dentamaro, Vincenzo
Published: (2025)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
by: Dong, Yihe, et al.
Published: (2025)

RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling
by: Wei, Xiuying, et al.
Published: (2025)

CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling
by: Li, Wenhao, et al.
Published: (2025)

LatentLLM: Attention-Aware Joint Tensor Compression
by: Koike-Akino, Toshiaki, et al.
Published: (2025)

From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP
by: Ganesan, Adithya V, et al.
Published: (2026)

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
by: Shen, Shuaijie, et al.
Published: (2024)

CSPLADE: Learned Sparse Retrieval with Causal Language Models
by: Xu, Zhichao, et al.
Published: (2025)

Language Modeling Using Tensor Trains
by: Su, Zhan, et al.
Published: (2024)

Hierarchical Federated Learning for Networked AI: From Communication Saving to Architecture-Aware Design
by: Azimi-Abarghouyi, Seyed Mohammad, et al.
Published: (2026)

Towards Cultural Bridge by Bahnaric-Vietnamese Translation Using Transfer Learning of Sequence-To-Sequence Pre-training Language Model
by: Dat, Phan Tran Minh, et al.
Published: (2025)

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention
by: Huang, Yuxiang, et al.
Published: (2026)

TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
by: Wu, Tong, et al.
Published: (2025)

Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback
by: Zhou, Jiayi, et al.
Published: (2024)