Cover Image

LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Zicheng, Wang, Li, Li, Siyuan, Wang, Zedong, Lin, Haitao, Li, Stan Z.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences
by: Liu, Zicheng, et al.
Published: (2024)

SemiReward: A General Reward Model for Semi-supervised Learning
by: Li, Siyuan, et al.
Published: (2023)

VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling
by: Li, Siyuan, et al.
Published: (2024)

PSC-CPI: Multi-Scale Protein Sequence-Structure Contrasting for Efficient and Generalizable Compound-Protein Interaction Prediction
by: Wu, Lirong, et al.
Published: (2024)

Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions
by: Tan, Cheng, et al.
Published: (2024)

Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning
by: Li, Siyuan, et al.
Published: (2024)

Calo-VQ: Vector-Quantized Two-Stage Generative Model in Calorimeter Simulation
by: Liu, Qibin, et al.
Published: (2024)

VQ-SAD: Vector Quantized Structure Aware Diffusion For Molecule Generation
by: Noravesh, Farshad, et al.
Published: (2026)

BandVQ: Band-Wise Vector-Quantized EEG Foundation Model
by: Sukhbaatar, Jamiyan, et al.
Published: (2026)

SMR: State Memory Replay for Long Sequence Modeling
by: Qi, Biqing, et al.
Published: (2024)

Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting
by: Wu, Lirong, et al.
Published: (2024)

A Survey on Mixup Augmentations and Beyond
by: Jin, Xin, et al.
Published: (2024)

An Empirical Study: Extensive Deep Temporal Point Process
by: Lin, Haitao, et al.
Published: (2021)

Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification
by: Liu, Zicheng, et al.
Published: (2025)

BeamVQ: Beam Search with Vector Quantization to Mitigate Data Scarcity in Physical Spatiotemporal Forecasting
by: Wang, Weiyan, et al.
Published: (2025)

RDesign: Hierarchical Data-efficient Representation Learning for Tertiary Structure-based RNA Design
by: Tan, Cheng, et al.
Published: (2023)

FoldToken: Learning Protein Language via Vector Quantization and Beyond
by: Gao, Zhangyang, et al.
Published: (2024)

Switch EMA: A Free Lunch for Better Flatness and Sharpness
by: Li, Siyuan, et al.
Published: (2024)

GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models
by: Liu, Zicheng, et al.
Published: (2024)

HyperVQ: MLR-based Vector Quantization in Hyperbolic Space
by: Goswami, Nabarun, et al.
Published: (2024)

Learning Long Sequences in Spiking Neural Networks
by: Stan, Matei Ioan, et al.
Published: (2023)

Discovering the Representation Bottleneck of Graph Neural Networks
by: Wu, Fang, et al.
Published: (2022)

Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning
by: Wang, Zedong, et al.
Published: (2025)

CBGBench: Fill in the Blank of Protein-Molecule Complex Binding Graph
by: Lin, Haitao, et al.
Published: (2024)

Functional-Group-Based Diffusion for Pocket-Specific Molecule Generation and Elaboration
by: Lin, Haitao, et al.
Published: (2023)

HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression
by: Khasia, Vladimer
Published: (2026)

IceCache: Memory-efficient KV-cache Management for Long-Sequence LLMs
by: Mao, Yuzhen, et al.
Published: (2026)

Taming LLMs by Scaling Learning Rates with Gradient Grouping
by: Li, Siyuan, et al.
Published: (2025)

GenURL: A General Framework for Unsupervised Representation Learning
by: Li, Siyuan, et al.
Published: (2021)

Transformer-VQ: Linear-Time Transformers via Vector Quantization
by: Lingle, Lucas D.
Published: (2023)

MAPE-PPI: Towards Effective and Efficient Protein-Protein Interaction Prediction via Microenvironment-Aware Protein Embedding
by: Wu, Lirong, et al.
Published: (2024)

SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learning
by: Tan, Cheng, et al.
Published: (2022)

VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization
by: Zhong, Hongliang, et al.
Published: (2023)

StreamBP: Memory-Efficient Exact Backpropagation for Long Sequence Training of LLMs
by: Luo, Qijun, et al.
Published: (2025)

VQ-DSC-R: Robust Vector Quantized-Enabled Digital Semantic Communication With OFDM Transmission
by: Chen, Jianqiao, et al.
Published: (2026)

Dynamics-inspired Structure Hallucination for Protein-protein Interaction Modeling
by: Wu, Fang, et al.
Published: (2026)

Teach Harder, Learn Poorer: Rethinking Hard Sample Distillation for GNN-to-MLP Knowledge Distillation
by: Wu, Lirong, et al.
Published: (2024)

A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation
by: Wu, Lirong, et al.
Published: (2024)

PPFlow: Target-aware Peptide Design with Torsional Flow Matching
by: Lin, Haitao, et al.
Published: (2024)

Sparse-VQ Transformer: An FFN-Free Framework with Vector Quantization for Enhanced Time Series Forecasting
by: Zhao, Yanjun, et al.
Published: (2024)