Imagen de Portada

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Munkhdalai, Tsendsuren, Faruqui, Manaal, Gopal, Siddharth
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence Machine Learning Neural and Evolutionary Computing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Deferred NAM: Low-latency Top-K Context Injection via Deferred Context Encoding for Non-Streaming ASR
por: Wu, Zelin, et al.
Publicado: (2024)

Hierarchical Recurrent Adapters for Efficient Multi-Task Adaptation of Large Speech Models
por: Munkhdalai, Tsendsuren, et al.
Publicado: (2024)

Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity
por: Chaudhary, Siddharth
Publicado: (2025)

Folded Context Condensation in Path Integral Formalism for Infinite Context Transformers
por: Paeng, Won-Gi, et al.
Publicado: (2024)

General-Purpose In-Context Learning by Meta-Learning Transformers
por: Kirsch, Louis, et al.
Publicado: (2022)

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
por: Wang, Jianyu, et al.
Publicado: (2025)

What Matters for Model Merging at Scale?
por: Yadav, Prateek, et al.
Publicado: (2024)

SkipSNN: Efficiently Classifying Spike Trains with Event-attention
por: Yin, Hang, et al.
Publicado: (2024)

GLU Attention Improve Transformer
por: Wang, Zehao
Publicado: (2025)

Reinforced In-Context Black-Box Optimization
por: Song, Lei, et al.
Publicado: (2024)

A Transformer-based Neural Architecture Search Method
por: Wang, Shang, et al.
Publicado: (2025)

NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branches
por: Smith, Ethan
Publicado: (2026)

RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers
por: Mia, Md Zesun Ahmed, et al.
Publicado: (2026)

Position as Probability: Self-Supervised Transformers that Think Past Their Training for Length Extrapolation
por: Lee, Philip Heejun
Publicado: (2025)

Elastic Architecture Search for Efficient Language Models
por: Wang, Shang
Publicado: (2025)

Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts
por: Zhang, Buze, et al.
Publicado: (2026)

MAR: Efficient Large Language Models via Module-aware Architecture Refinement
por: Cai, Junhong, et al.
Publicado: (2026)

W-PCA Based Gradient-Free Proxy for Efficient Search of Lightweight Language Models
por: Wang, Shang
Publicado: (2025)

Context-sensitive neocortical neurons transform the effectiveness and efficiency of neural information processing
por: Ahmed, Khubaib, et al.
Publicado: (2022)

LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression
por: Zhang, Hengzhe, et al.
Publicado: (2025)

KV Cache Recycling to Expand Usable Context Capacity in Low Parameter LLMs
por: Pandey, Prashant
Publicado: (2025)

Gate-level boolean evolutionary geometric attention neural networks
por: Shi, Xianshuai, et al.
Publicado: (2025)

Differentiable architecture search with multi-dimensional attention for spiking neural networks
por: Man, Yilei, et al.
Publicado: (2024)

LPC-SM: Local Predictive Coding and Sparse Memory for Long-Context Language Modeling
por: Xie, Keqin
Publicado: (2026)

Interlocking-free Selective Rationalization Through Genetic-based Learning
por: Ruggeri, Federico, et al.
Publicado: (2024)

Evolutionary Multi-Objective Optimization of Large Language Model Prompts for Balancing Sentiments
por: Baumann, Jill, et al.
Publicado: (2024)

EvoGPT-f: An Evolutionary GPT Framework for Benchmarking Formal Math Languages
por: Mercer, Johnathan
Publicado: (2024)

Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security
por: Qu, Youyang, et al.
Publicado: (2024)

Hyperbolic Fine-Tuning for Large Language Models
por: Yang, Menglin, et al.
Publicado: (2024)

Fleet of Agents: Coordinated Problem Solving with Large Language Models
por: Klein, Lars, et al.
Publicado: (2024)

Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
por: Petruzzellis, Flavio, et al.
Publicado: (2024)

Early stopping by correlating online indicators in neural networks
por: Ferro, Manuel Vilares, et al.
Publicado: (2024)

EvoMerge: Neuroevolution for Large Language Models
por: Jiang, Yushu
Publicado: (2024)

LLM-SR: Scientific Equation Discovery via Programming with Large Language Models
por: Shojaee, Parshin, et al.
Publicado: (2024)

When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges
por: Wang, Chao, et al.
Publicado: (2024)

Fine-Tuning and Evaluating Open-Source Large Language Models for the Army Domain
por: Ruiz, Daniel C., et al.
Publicado: (2024)

Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboard
por: Topsakal, Oguzhan, et al.
Publicado: (2024)

A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations
por: Javidnia, Hossein
Publicado: (2026)

CAPO: Cost-Aware Prompt Optimization
por: Zehle, Tom, et al.
Publicado: (2025)

Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces
por: Javidnia, Hossein
Publicado: (2026)