Copertina

PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kacham, Praneeth, Mirrokni, Vahab, Zhong, Peilin
Natura:	Preprint
Pubblicazione:	2023
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Trellis: Learning to Compress Key-Value Memory in Attention Models
di: Karami, Mahdi, et al.
Pubblicazione: (2025)

PolarQuant: Quantizing KV Caches with Polar Transformation
di: Han, Insu, et al.
Pubblicazione: (2025)

TNT: Improving Chunkwise Training for Test-Time Memorization
di: Li, Zeman, et al.
Pubblicazione: (2025)

Titans: Learning to Memorize at Test Time
di: Behrouz, Ali, et al.
Pubblicazione: (2024)

Tensor Sketch: Fast and Scalable Polynomial Kernel Approximation
di: Pham, Ninh, et al.
Pubblicazione: (2025)

High-Dimensional Geometric Streaming for Nearly Low Rank Data
di: Esfandiari, Hossein, et al.
Pubblicazione: (2024)

Optimal Communication for Classic Functions in the Coordinator Model and Beyond
di: Esfandiari, Hossein, et al.
Pubblicazione: (2024)

It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
di: Behrouz, Ali, et al.
Pubblicazione: (2025)

Nested Learning: The Illusion of Deep Learning Architectures
di: Behrouz, Ali, et al.
Pubblicazione: (2025)

PiKE: Adaptive Data Mixing for Large-Scale Multi-Task Learning Under Low Gradient Conflicts
di: Li, Zeman, et al.
Pubblicazione: (2025)

MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling
di: Karami, Mahdi, et al.
Pubblicazione: (2025)

Differentially Private Graph Learning via Sensitivity-Bounded Personalized PageRank
di: Epasto, Alessandro, et al.
Pubblicazione: (2022)

Sketch In, Sketch Out: Accelerating both Learning and Inference for Structured Prediction with Kernels
di: Ahmad, Tamim El, et al.
Pubblicazione: (2023)

Memory Caching: RNNs with Growing Memory
di: Behrouz, Ali, et al.
Pubblicazione: (2026)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
di: Li, Zeman, et al.
Pubblicazione: (2024)

PolyFormer: Scalable Node-wise Filters via Polynomial Graph Transformer
di: Ma, Jiahong, et al.
Pubblicazione: (2024)

Fast Second-Order Online Kernel Learning through Incremental Matrix Sketching and Decomposition
di: Wen, Dongxie, et al.
Pubblicazione: (2024)

Deep Sketched Output Kernel Regression for Structured Prediction
di: Ahmad, Tamim El, et al.
Pubblicazione: (2024)

ATLAS: Learning to Optimally Memorize the Context at Test Time
di: Behrouz, Ali, et al.
Pubblicazione: (2025)

Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing
di: Javanmard, Adel, et al.
Pubblicazione: (2025)

Perturb-and-Project: Differentially Private Similarities and Marginals
di: Cohen-Addad, Vincent, et al.
Pubblicazione: (2024)

Lattice: Learning to Efficiently Compress the Memory
di: Karami, Mahdi, et al.
Pubblicazione: (2025)

Understanding the Role of Training Data in Test-Time Scaling
di: Javanmard, Adel, et al.
Pubblicazione: (2025)

Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models
di: Javanmard, Adel, et al.
Pubblicazione: (2026)

Sketch to Adapt: Fine-Tunable Sketches for Efficient LLM Adaptation
di: Zhang, Tianyi, et al.
Pubblicazione: (2024)

Revisiting Matrix Sketching in Linear Bandits: Achieving Sublinear Regret via Dyadic Block Sketching
di: Wen, Dongxie, et al.
Pubblicazione: (2024)

PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses
di: Javanmard, Adel, et al.
Pubblicazione: (2024)

LLM-Sketch: Enhancing Network Sketches with LLM
di: Li, Yuanpeng, et al.
Pubblicazione: (2025)

Sketching the Heat Kernel: Using Gaussian Processes to Embed Data
di: Gilbert, Anna C., et al.
Pubblicazione: (2024)

Sketch-and-Verify: Structured Inference-Time Scaling via Program Sketching
di: Jiang, Shan, et al.
Pubblicazione: (2026)

Retraining with Predicted Hard Labels Provably Increases Model Accuracy
di: Das, Rudrajit, et al.
Pubblicazione: (2024)

Sublinear Sketches for Approximate Nearest Neighbor and Kernel Density Estimation
di: Danait, Ved, et al.
Pubblicazione: (2025)

Efficient Data Selection at Scale via Influence Distillation
di: Nikdan, Mahdi, et al.
Pubblicazione: (2025)

Improving the Variance of Differentially Private Randomized Experiments through Clustering
di: Javanmard, Adel, et al.
Pubblicazione: (2023)

LevAttention: Time, Space, and Streaming Efficient Algorithm for Heavy Attentions
di: Kannan, Ravindran, et al.
Pubblicazione: (2024)

Lego Sketch: A Scalable Memory-augmented Neural Network for Sketching Data Streams
di: Feng, Yuan, et al.
Pubblicazione: (2025)

Adaptive-GraphSketch: Real-Time Edge Anomaly Detection via Multi-Layer Tensor Sketching and Temporal Decay
di: Ekle, Ocheme Anthony, et al.
Pubblicazione: (2025)

DeepCrossAttention: Supercharging Transformer Residual Connections
di: Heddes, Mike, et al.
Pubblicazione: (2025)

On the (Generative) Linear Sketching Problem
di: Yuan, Xinyu, et al.
Pubblicazione: (2026)

Private Sketches for Linear Regression
di: Das, Shrutimoy, et al.
Pubblicazione: (2025)