Image de couverture de livre

LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteur principal:	Karmore, Aryan
Format:	Preprint
Publié:	2026
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

ButterflyMoE: Sub-Linear Ternary Experts via Structured Butterfly Orbits
par: Karmore, Aryan
Publié: (2026)

ButterflyViT: 354$\times$ Expert Compression for Edge Vision Transformers
par: Karmore, Aryan
Publié: (2026)

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning
par: Liu, Dong, et autres
Publié: (2026)

Tri-Accel: Curvature-Aware Precision-Adaptive and Memory-Elastic Optimization for Efficient GPU Usage
par: Sheibanian, Mohsen, et autres
Publié: (2025)

AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
par: Deiseroth, Björn, et autres
Publié: (2023)

L$^3$: Large Lookup Layers
par: Tseng, Albert, et autres
Publié: (2026)

Echo State Transformer: Attention Over Finite Memories
par: Bendi-Ouis, Yannis, et autres
Publié: (2025)

Sample-Efficient Bayesian Optimization with Transfer Learning for Heterogeneous Search Spaces
par: Deshwal, Aryan, et autres
Publié: (2024)

In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness
par: Collins, Liam, et autres
Publié: (2024)

Resource-Efficient Transformer Architecture: Optimizing Memory and Execution Time for Real-Time Applications
par: V, Krisvarish, et autres
Publié: (2024)

QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention
par: Oh, Sehyeon, et autres
Publié: (2026)

Hamming Attention Distillation: Binarizing Keys and Queries for Efficient Long-Context Transformers
par: Horton, Mark, et autres
Publié: (2025)

COM-BOM: Bayesian Exemplar Search for Efficiently Exploring the Accuracy-Calibration Pareto Frontier
par: Luo, Gaoxiang, et autres
Publié: (2025)

LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator
par: Li, Guoyu, et autres
Publié: (2025)

Key and Value Weights Are Probably All You Need: On the Necessity of the Query, Key, Value weight Triplet in Self-Attention Transformers
par: Karbevski, Marko, et autres
Publié: (2025)

ZeroS: Zero-Sum Linear Attention for Efficient Transformers
par: Lu, Jiecheng, et autres
Publié: (2026)

FlashOptim: Optimizers for Memory-Efficient Training
par: Ortiz, Jose Javier Gonzalez, et autres
Publié: (2026)

Lookup multivariate Kolmogorov-Arnold Networks
par: Pozdnyakov, Sergey, et autres
Publié: (2025)

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
par: Zhao, Runsong, et autres
Publié: (2026)

Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers
par: Nidhi, Amrit
Publié: (2026)

Modular Memory is the Key to Continual Learning Agents
par: Dorovatas, Vaggelis, et autres
Publié: (2026)

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization
par: Park, Kwangryeol, et autres
Publié: (2024)

From Arithmetic to Logic: The Resilience of Logic and Lookup-Based Neural Networks Under Parameter Bit-Flips
par: Bacellar, Alan T. L., et autres
Publié: (2026)

Self-Tuning Sparse Attention: Multi-Fidelity Hyperparameter Optimization for Transformer Acceleration
par: Dev, Arundhathi, et autres
Publié: (2026)

A Survey on Memory-Efficient Transformer-Based Model Training in AI for Science
par: Tian, Kaiyuan, et autres
Publié: (2025)

VecFormer: Towards Efficient and Generalizable Graph Transformer with Graph Token Attention
par: Zhou, Jingbo, et autres
Publié: (2026)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
par: Liu, Yuxi, et autres
Publié: (2025)

Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization
par: Shen, Qianli, et autres
Publié: (2024)

SCOUT: Toward Sub-Quadratic Attention via Segment Compression for Optimized Utility in Transformers
par: Jafari, Aref, et autres
Publié: (2025)

Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers
par: Liang, Yingyu, et autres
Publié: (2024)

Adaptive Memory Decay for Log-Linear Attention
par: Amin, Yaxita, et autres
Publié: (2026)

Fusing Memory and Attention: A study on LSTM, Transformer and Hybrid Architectures for Symbolic Music Generation
par: Ghoshal, Soudeep, et autres
Publié: (2026)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
par: Xi, Haocheng, et autres
Publié: (2024)

Recurrent Action Transformer with Memory
par: Cherepanov, Egor, et autres
Publié: (2023)

The Bayesian Geometry of Transformer Attention
par: Agarwal, Naman, et autres
Publié: (2025)

Offline Model-Based Optimization via Policy-Guided Gradient Search
par: Chemingui, Yassine, et autres
Publié: (2024)

Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
par: Ranganath, Aditya
Publié: (2026)

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
par: Yang, Zi, et autres
Publié: (2024)

Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models
par: Vendrell, Victor Conchello, et autres
Publié: (2026)

ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
par: Yang, David H., et autres
Publié: (2026)