Copertina

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kiruluta, Andrew, Raju, Preethi, Burity, Priscilla
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attention
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

From Eigenmodes to Proofs: Integrating Graph Spectral Operators with Symbolic Interpretable Reasoning
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

CS-VLM: Compressed Sensing Attention for Efficient Vision-Language Representation Learning
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Spectral Neuro-Symbolic Reasoning II: Semantic Node Merging, Entailment Filtering, and Knowledge Graph Alignment
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Adaptive Two Sided Laplace Transforms: A Learnable, Interpretable, and Scalable Replacement for Self-Attention
di: Kiruluta, Andrew
Pubblicazione: (2025)

Entropic-Time Inference: Self-Organizing Large Language Model Decoding Beyond Attention
di: Kiruluta, Andrew
Pubblicazione: (2026)

Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery
di: Kiruluta, Andrew
Pubblicazione: (2026)

Operator-Based Machine Intelligence: A Hilbert Space Framework for Spectral Learning and Symbolic Reasoning
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
di: Kiruluta, Andrew
Pubblicazione: (2026)

Beyond Neural Networks: Symbolic Reasoning over Wavelet Logic Graph Signals
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Spectral Generative Flow Models: A Physics-Inspired Replacement for Vectorized Large Language Models
di: Kiruluta, Andrew
Pubblicazione: (2026)

From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
di: Kiruluta, Andrew
Pubblicazione: (2026)

Beyond Self Attention: A Subquadratic Fourier Wavelet Transformer with Multi Modal Fusion
di: Kiruluta, Andrew, et al.
Pubblicazione: (2021)

FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

From Pixels and Words to Waves: A Unified Framework for Spectral Dictionary vLLMs
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

State Fourier Diffusion Language Model (SFDLM): A Scalable, Novel Iterative Approach to Language Modeling
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Scaling Long-Horizon LLM Agent via Context-Folding
di: Sun, Weiwei, et al.
Pubblicazione: (2025)

LLMSteer: Improving Long-Context LLM Inference by Steering Attention on Reused Contexts
di: Gu, Zhuohan, et al.
Pubblicazione: (2024)

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
di: Liu, Di, et al.
Pubblicazione: (2024)

XLGoBench: Detecting cross-lingual skill gaps with algorithmic tasks
di: Jain, Purvam, et al.
Pubblicazione: (2026)

Multipole Attention for Efficient Long Context Reasoning
di: Hooper, Coleman, et al.
Pubblicazione: (2025)

MTraining: Distributed Dynamic Sparse Attention for Efficient Ultra-Long Context Training
di: Li, Wenxuan, et al.
Pubblicazione: (2025)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
di: Tang, Hongyin, et al.
Pubblicazione: (2024)

Breaking the Attention Bottleneck
di: Hilsenbek, Kalle
Pubblicazione: (2024)

Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
di: Shyam, Vasudev, et al.
Pubblicazione: (2024)

Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications?
di: Intrator, Yotam, et al.
Pubblicazione: (2024)

SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention
di: Zhu, Qianchao, et al.
Pubblicazione: (2024)

A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signals
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents
di: Wang, Jiawei, et al.
Pubblicazione: (2025)

From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
di: Kiruluta, Andrew
Pubblicazione: (2025)

S$^3$-Attention:Attention-Aligned Endogenous Retrieval for Memory-Bounded Long-Context Inference
di: Ma, Qingsen, et al.
Pubblicazione: (2026)

Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
di: Donhauser, Konstantin, et al.
Pubblicazione: (2025)

Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
di: Wang, Zhenting, et al.
Pubblicazione: (2026)

AgentFold: Long-Horizon Web Agents with Proactive Context Management
di: Ye, Rui, et al.
Pubblicazione: (2025)

Quantum Spectral Reasoning: A Non-Neural Architecture for Interpretable Machine Learning
di: Kiruluta, Andrew
Pubblicazione: (2025)

Recursive Models for Long-Horizon Reasoning
di: Yang, Chenxiao, et al.
Pubblicazione: (2026)

DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning
di: Zarch, Hossein Entezari, et al.
Pubblicazione: (2025)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
di: Jo, Dongwon, et al.
Pubblicazione: (2026)