Cover Image

Compute Better Spent: Replacing Dense Layers with Structured Matrices

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Qiu, Shikai, Potapczynski, Andres, Finzi, Marc, Goldblum, Micah, Wilson, Andrew Gordon
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
by: Potapczynski, Andres, et al.
Published: (2024)

The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning
by: Goldblum, Micah, et al.
Published: (2023)

Customizing the Inductive Biases of Softmax Attention using Structured Matrices
by: Kuang, Yilun, et al.
Published: (2025)

The Lie Derivative for Measuring Learned Equivariance
by: Gruver, Nate, et al.
Published: (2022)

Large Language Models Are Zero-Shot Time Series Forecasters
by: Gruver, Nate, et al.
Published: (2023)

Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
by: Lotfi, Sanae, et al.
Published: (2024)

From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
by: Finzi, Marc, et al.
Published: (2026)

Non-Vacuous Generalization Bounds for Large Language Models
by: Lotfi, Sanae, et al.
Published: (2023)

Training Flexible Models of Genetic Variant Effects from Functional Annotations using Accelerated Linear Algebra
by: Amin, Alan N., et al.
Published: (2025)

Compute-Optimal LLMs Provably Generalize Better With Scale
by: Finzi, Marc, et al.
Published: (2025)

Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
by: Marek, Martin, et al.
Published: (2025)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
by: Qiu, Shikai, et al.
Published: (2025)

Just How Flexible are Neural Networks in Practice?
by: Shwartz-Ziv, Ravid, et al.
Published: (2024)

Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales
by: Qiu, Shikai, et al.
Published: (2025)

Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay
by: Marek, Martin, et al.
Published: (2026)

Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding
by: Thomas, Rahul, et al.
Published: (2026)

Transferring Knowledge from Large Foundation Models to Small Downstream Models
by: Qiu, Shikai, et al.
Published: (2024)

Knowing What You Know Is Not Enough: Large Language Model Confidences Don't Align With Their Actions
by: Pal, Arka, et al.
Published: (2025)

vTune: Verifiable Fine-Tuning for LLMs Through Backdooring
by: Zhang, Eva, et al.
Published: (2024)

Predicting the Performance of Black-box LLMs through Follow-up Queries
by: Sam, Dylan, et al.
Published: (2025)

Privacy-Preserving Mechanisms Enable Cheap Verifiable Inference of LLMs
by: Pal, Arka, et al.
Published: (2026)

Adaptive Retention & Correction: Test-Time Training for Continual Learning
by: Chen, Haoran, et al.
Published: (2024)

Large Language Models Must Be Taught to Know What They Don't Know
by: Kapoor, Sanyam, et al.
Published: (2024)

On Training in Imagination
by: Timor, Nadav, et al.
Published: (2026)

Deep Learning is Not So Mysterious or Different
by: Wilson, Andrew Gordon
Published: (2025)

Out-of-Distribution Detection Methods Answer the Wrong Questions
by: Li, Yucen Lily, et al.
Published: (2025)

Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
by: Li, Ang, et al.
Published: (2025)

Diffusing Differentiable Representations
by: Savani, Yash, et al.
Published: (2024)

Transformers Boost the Performance of Decision Trees on Tabular Data across Sample Sizes
by: Jayawardhana, Mayuka, et al.
Published: (2025)

Generating Potent Poisons and Backdoors from Scratch with Guided Diffusion
by: Souri, Hossein, et al.
Published: (2024)

Cascade: Token-Sharded Private LLM Inference
by: Thomas, Rahul, et al.
Published: (2025)

Multi-Token Prediction via Self-Distillation
by: Kirchenbauer, John, et al.
Published: (2026)

An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMs
by: Thomas, Rahul, et al.
Published: (2025)

Identifying and Evaluating Inactive Heads in Pretrained LLMs
by: Sandoval-Segura, Pedro, et al.
Published: (2025)

Time-Aware Prior Fitted Networks for Zero-Shot Forecasting with Exogenous Variables
by: Potapczynski, Andres, et al.
Published: (2026)

A More Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models
by: Olivares, Kin G., et al.
Published: (2025)

Closing the Train-Test Gap in World Models for Gradient-Based Planning
by: Parthasarathy, Arjun, et al.
Published: (2025)

Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices
by: Viset, Frida, et al.
Published: (2024)

TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
by: Feuer, Benjamin, et al.
Published: (2024)

Scalable and Flexible Causal Discovery with an Efficient Test for Adjacency
by: Amin, Alan Nawzad, et al.
Published: (2024)