Image de couverture de livre

Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	He, Ziwei, Yang, Meng, Feng, Minwei, Yin, Jingcheng, Wang, Xinbing, Leng, Jingwen, Lin, Zhouhan
Format:	Preprint
Publié:	2023
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

AWM: Accurate Weight-Matrix Fingerprint for Large Language Models
par: Zeng, Boyi, et autres
Publié: (2025)

Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention
par: He, Ziwei, et autres
Publié: (2023)

PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous Space
par: Zeng, Boyi, et autres
Publié: (2025)

PonderLM: Pretraining Language Models to Ponder in Continuous Space
par: Zeng, Boyi, et autres
Publié: (2025)

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
par: Shen, Junzhe, et autres
Publié: (2026)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
par: Lu, Liming, et autres
Publié: (2026)

Towards Controlled Table-to-Text Generation with Scientific Reasoning
par: Guo, Zhixin, et autres
Publié: (2023)

WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models
par: Yuan, Jian, et autres
Publié: (2025)

Gumbel Reranking: Differentiable End-to-End Reranker Optimization
par: Huang, Siyuan, et autres
Publié: (2025)

TreeKV: Smooth Key-Value Cache Compression with Tree Structures
par: He, Ziwei, et autres
Publié: (2025)

VQKV: High-Fidelity and High-Ratio Cache Compression via Vector-Quantization
par: Wang, Yixuan, et autres
Publié: (2026)

HuRef: HUman-REadable Fingerprint for Large Language Models
par: Zeng, Boyi, et autres
Publié: (2023)

PonderLM-3: Adaptive Token-Wise Pondering with Differentiable Masking
par: Li, He, et autres
Publié: (2026)

Context-level Language Modeling by Learning Predictive Context Embeddings
par: Dai, Beiya, et autres
Publié: (2025)

AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth
par: Song, Shixiang, et autres
Publié: (2026)

The NLP Task Effectiveness of Long-Range Transformers
par: Qin, Guanghui, et autres
Publié: (2022)

FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
par: Kai, Jushi, et autres
Publié: (2025)

Pretraining with Token-Level Adaptive Latent Chain-of-Thought
par: Zeng, Boyi, et autres
Publié: (2026)

Training-free LLM-generated Text Detection by Mining Token Probability Sequences
par: Xu, Yihuai, et autres
Publié: (2024)

CTC Blank Triggered Dynamic Layer-Skipping for Efficient CTC-based Speech Recognition
par: Hou, Junfeng, et autres
Publié: (2024)

Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning
par: Feng, Aosong, et autres
Publié: (2024)

How Do Different Forms of Note‐Taking Affect Second Language Vocabulary Learning?
par: Zhouhan Jin, et autres
Publié: (2025)

FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
par: Kiruluta, Andrew, et autres
Publié: (2025)

Graph with Sequence: Broad-Range Semantic Modeling for Fake News Detection
par: Yin, Junwei, et autres
Publié: (2024)

Adapting Knowledge for Few-shot Table-to-Text Generation
par: Guo, Zhixin, et autres
Publié: (2023)

Critical Data Size of Language Models from a Grokking Perspective
par: Zhu, Xuekai, et autres
Publié: (2024)

Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models
par: Wang, Huanyu, et autres
Publié: (2025)

Dimensionality Reduction in Sentence Transformer Vector Databases with Fast Fourier Transform
par: Bulgakov, Vitaly, et autres
Publié: (2024)

FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processing
par: Guo, Shoutao, et autres
Publié: (2025)

FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
par: Zhang, Runzhe, et autres
Publié: (2026)

Mirror-Consistency: Harnessing Inconsistency in Majority Voting
par: Huang, Siyuan, et autres
Publié: (2024)

Long-range Modeling and Processing of Multimodal Event Sequences
par: Li, Jichu, et autres
Publié: (2026)

SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
par: Kai, Jushi, et autres
Publié: (2024)

Leveraging Grammar Induction for Language Understanding and Generation
par: Kai, Jushi, et autres
Publié: (2024)

Fast Compressed-Domain N-Point Discrete Fourier Transform: The "Twiddless" FFT Algorithm
par: Queiroz, Saulo
Publié: (2025)

TurboFFT: A High-Performance Fast Fourier Transform with Fault Tolerance on GPU
par: Wu, Shixun, et autres
Publié: (2024)

How Redundant Is the Transformer Stack in Speech Representation Models?
par: Dorszewski, Teresa, et autres
Publié: (2024)

PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning
par: Zhu, Xuekai, et autres
Publié: (2023)

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
par: Shen, Shuaijie, et autres
Publié: (2024)

FourierSampler: Unlocking Non-Autoregressive Potential in Diffusion Language Models via Frequency-Guided Generation
par: He, Siyang, et autres
Publié: (2026)