Cover Image

Attamba: Attending To Multi-Token States

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Akhauri, Yash, Huda, Safeen, Abdelfattah, Mohamed S.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Compute Where it Counts: Self Optimizing Language Models
by: Akhauri, Yash, et al.
Published: (2026)

ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models
by: Akhauri, Yash, et al.
Published: (2024)

Encodings for Prediction-based Neural Architecture Search
by: Akhauri, Yash, et al.
Published: (2024)

On Latency Predictors for Neural Architecture Search
by: Akhauri, Yash, et al.
Published: (2024)

TokenButler: Token Importance is Predictable
by: Akhauri, Yash, et al.
Published: (2025)

Regression Language Models for Code
by: Akhauri, Yash, et al.
Published: (2025)

xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extraction
by: Chang, Chi-Chih, et al.
Published: (2025)

Radial Networks: Dynamic Layer Routing for High-Performance Large Language Models
by: Dotzel, Jordan, et al.
Published: (2024)

FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
by: Hu, Zhanqiu, et al.
Published: (2025)

SplitReason: Learning To Offload Reasoning
by: Akhauri, Yash, et al.
Published: (2025)

Learning When to Attend: Conditional Memory Access for Long-Context LLMs
by: Choudhary, Sakshi, et al.
Published: (2026)

Rethinking Token Reduction for State Space Models
by: Zhan, Zheng, et al.
Published: (2024)

Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models
by: Chiang, Hung-Yueh, et al.
Published: (2025)

EchoAtt: Attend, Copy, then Adjust for More Efficient Large Language Models
by: Rajabzadeh, Hossein, et al.
Published: (2024)

Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
by: Zhang, Qingru, et al.
Published: (2023)

Multi-word Tokenization for Sequence Compression
by: Gee, Leonidas, et al.
Published: (2024)

High-Stakes Personalization: Rethinking LLM Customization for Individual Investor Decision-Making
by: Sawant, Yash Ganpat
Published: (2026)

The Expressive Capacity of State Space Models: A Formal Language Perspective
by: Sarrof, Yash, et al.
Published: (2024)

Faster LLM Inference via Sequential Monte Carlo
by: Emara, Yahya, et al.
Published: (2026)

Multi-Token Prediction via Self-Distillation
by: Kirchenbauer, John, et al.
Published: (2026)

MambaByte: Token-free Selective State Space Model
by: Wang, Junxiong, et al.
Published: (2024)

Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings
by: Jiang, Yichen, et al.
Published: (2024)

Multi-Faceted Question Complexity Estimation Targeting Topic Domain-Specificity
by: R, Sujay, et al.
Published: (2024)

An Analysis of Embedding Layers and Similarity Scores using Siamese Neural Networks
by: Bingi, Yash, et al.
Published: (2023)

Luna-2: Scalable Single-Token Evaluation with Small Language Models
by: Goel, Vatsal, et al.
Published: (2026)

PATENTWRITER: A Benchmarking Study for Patent Drafting with LLMs
by: Shomee, Homaira Huda, et al.
Published: (2025)

RaZeR: Pushing the Limits of NVFP4 Quantization with Redundant Zero Remapping
by: Chen, Yuzong, et al.
Published: (2025)

Future Lens: Anticipating Subsequent Tokens from a Single Hidden State
by: Pal, Koyena, et al.
Published: (2023)

Optimized Multi-Token Joint Decoding with Auxiliary Model for LLM Inference
by: Qin, Zongyue, et al.
Published: (2024)

Should We Attend More or Less? Modulating Attention for Fairness
by: Zayed, Abdelrahman, et al.
Published: (2023)

SparAMX: Accelerating Compressed LLMs Token Generation on AMX-powered CPUs
by: AbouElhamayed, Ahmed F., et al.
Published: (2025)

Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
by: Samragh, Mohammad, et al.
Published: (2025)

How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines
by: Sengupta, Ayan, et al.
Published: (2025)

Self-Distillation for Multi-Token Prediction
by: Zhao, Guoliang, et al.
Published: (2026)

TokenShapley: Token Level Context Attribution with Shapley Value
by: Xiao, Yingtai, et al.
Published: (2025)

Token Distillation: Attention-aware Input Embeddings For New Tokens
by: Dobler, Konstantin, et al.
Published: (2025)

X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2026)

Token Sparse Attention: Efficient Long-Context Inference with Interleaved Token Selection
by: Jo, Dongwon, et al.
Published: (2026)

$μ^2$Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for Radiology Report Generation
by: Li, Siyou, et al.
Published: (2025)

Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation
by: Ehab, Mohamed, et al.
Published: (2026)