Cover Image

The Power of Second Order Methods for Sequence Preconditioning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Marsden, Annie, Hazan, Elad
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Universal Sequence Preconditioning
by: Marsden, Annie, et al.
Published: (2025)

Second Order Methods for Bandit Optimization and Control
by: Suggala, Arun, et al.
Published: (2024)

Provable Length Generalization in Sequence Prediction via Spectral Filtering
by: Marsden, Annie, et al.
Published: (2024)

Spectral Filtering for Complex Linear Dynamical Systems
by: Hazan, Elad, et al.
Published: (2026)

RubiConv -- Efficient Boundary-Respecting Convolutions
by: Friso, Linda, et al.
Published: (2026)

Introduction to Online Control
by: Hazan, Elad, et al.
Published: (2022)

Universal Learning of Nonlinear Dynamics
by: Dogariu, Evan, et al.
Published: (2025)

Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
by: Xia, Wenhan, et al.
Published: (2024)

Spectral State Space Models
by: Agarwal, Naman, et al.
Published: (2023)

Boosting Simple Learners
by: Alon, Noga, et al.
Published: (2020)

Research Program: Theory of Learning in Dynamical Systems
by: Hazan, Elad, et al.
Published: (2025)

The Hidden Game Problem
by: Buzaglo, Gon, et al.
Published: (2025)

SFO: Learning PDE Operators via Spectral Filtering
by: Koren, Noam, et al.
Published: (2026)

AI Alignment via Incentives and Correction
by: Agarwal, Rohit, et al.
Published: (2026)

SpectraLDS: Provable Distillation for Linear Dynamical Systems
by: Shah, Devan, et al.
Published: (2025)

A New Approach to Controlling Linear Dynamical Systems
by: Brahmbhatt, Anand, et al.
Published: (2025)

Efficient Spectral Control of Partially Observed Linear Dynamical Systems
by: Brahmbhatt, Anand, et al.
Published: (2025)

Adaptive Regret for Bandits Made Possible: Two Queries Suffice
by: Lu, Zhou, et al.
Published: (2024)

FutureFill: Fast Generation from Convolutional Sequence Models
by: Agarwal, Naman, et al.
Published: (2024)

Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing
by: Romanov, Elad, et al.
Published: (2024)

Online Control in Population Dynamics
by: Golowich, Noah, et al.
Published: (2024)

Adaptive Online Learning of Quantum States
by: Chen, Xinyi, et al.
Published: (2022)

Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences
by: Tumma, Neehal, et al.
Published: (2026)

Preconditioned Discrete-HAMS: A Second-order Irreversible Discrete Sampler
by: Zhou, Yuze, et al.
Published: (2025)

Flash STU: Fast Spectral Transform Units
by: Liu, Y. Isabel, et al.
Published: (2024)

Stochastic Gradient Methods with Preconditioned Updates
by: Sadiev, Abdurakhmon, et al.
Published: (2022)

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
by: Liu, Ziyue, et al.
Published: (2026)

SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning
by: Jia, Jinghan, et al.
Published: (2024)

Better LMO-based Momentum Methods with Second-Order Information
by: Khirirat, Sarit, et al.
Published: (2025)

PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling
by: Bonfanti, Andrea, et al.
Published: (2025)

Don't Be So Positive: Negative Step Sizes in Second-Order Methods
by: Shea, Betty, et al.
Published: (2024)

The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing
by: Xu, Xingyu, et al.
Published: (2023)

Efficient Convex Optimization Requires Superlinear Memory
by: Marsden, Annie, et al.
Published: (2022)

The Minimax Rate of Second-Order Calibration
by: Ciosek, Kamil, et al.
Published: (2026)

On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning
by: Zhang, Thomas T., et al.
Published: (2025)

Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method
by: Kuang, Simon, et al.
Published: (2026)

First and Second Order Approximations to Stochastic Gradient Descent Methods with Momentum Terms
by: Lu, Eric
Published: (2025)

On The Statistical Representation Properties Of The Perturb-Softmax And The Perturb-Argmax Probability Distributions
by: Indelman, Hedda Cohen, et al.
Published: (2024)

FOSI: Hybrid First and Second Order Optimization
by: Sivan, Hadar, et al.
Published: (2023)

LUCID: Attention with Preconditioned Representations
by: Duvvuri, Sai Surya, et al.
Published: (2026)