Cover Image

Improving Sparse Memory Finetuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Goyal, Satyam, Kanchi, Anirudh, Shah, Garv, Gupta, Prakhar
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Sparse Memory Finetuning as a Low-Forgetting Alternative to LoRA and Full Finetuning
by: Gupta, Prakhar, et al.
Published: (2026)

Self-Mined Hardness for Safety Fine-Tuning
by: Gupta, Prakhar, et al.
Published: (2026)

Sparse Upcycling: Inference Inefficient Finetuning
by: Doubov, Sasha, et al.
Published: (2024)

DualDiffusion: A Speculative Decoding Strategy for Masked Diffusion Models
by: Goyal, Satyam, et al.
Published: (2026)

Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance
by: Goyal, Sachin, et al.
Published: (2024)

Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning
by: Kaur, Simran, et al.
Published: (2024)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
by: Zhang, Biao, et al.
Published: (2024)

LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Model
by: Salles, Marcel Mateos, et al.
Published: (2025)

Predicting Emergent Capabilities by Finetuning
by: Snell, Charlie, et al.
Published: (2024)

Lightweight Query Routing for Adaptive RAG: A Baseline Study on RAGRouter-Bench
by: Bansal, Prakhar, et al.
Published: (2026)

SpecAttn: Speculating Sparse Attention
by: Shah, Harsh
Published: (2025)

Can Models Learn Skill Composition from Examples?
by: Zhao, Haoyu, et al.
Published: (2024)

MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
by: Teo, Rachel S. Y., et al.
Published: (2025)

Understanding Finetuning for Factual Knowledge Extraction
by: Ghosal, Gaurav, et al.
Published: (2024)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
by: Indurthi, Sathish Reddy, et al.
Published: (2024)

GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence
by: Krishna, Kundan, et al.
Published: (2024)

Ensembling Finetuned Language Models for Text Classification
by: Arango, Sebastian Pineda, et al.
Published: (2024)

Transfer Learning for Finetuning Large Language Models
by: Strangmann, Tobias, et al.
Published: (2024)

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
by: Shi, Taiwei, et al.
Published: (2025)

Finetuning Language Models to Emit Linguistic Expressions of Uncertainty
by: Chaudhry, Arslan, et al.
Published: (2024)

Order Independence With Finetuning
by: Brown, Katrina, et al.
Published: (2025)

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
by: Lyu, Kaifeng, et al.
Published: (2024)

End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF: A Reproducibility Study
by: Ganesh, Anirudh, et al.
Published: (2025)

Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systems
by: Agrawal, Aakriti, et al.
Published: (2025)

Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders
by: Kurochkin, Vadim, et al.
Published: (2025)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
by: Feng, Jiahai, et al.
Published: (2024)

Representation Noising: A Defence Mechanism Against Harmful Finetuning
by: Rosati, Domenic, et al.
Published: (2024)

KnowLA: Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation
by: Luo, Xindi, et al.
Published: (2024)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Towards Active Synthetic Data Generation for Finetuning Language Models
by: Kessler, Samuel, et al.
Published: (2025)

From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning
by: Zhao, Eric, et al.
Published: (2025)

Towards More Realistic Extraction Attacks: An Adversarial Perspective
by: More, Yash, et al.
Published: (2024)

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data
by: Xiong, Zheyang, et al.
Published: (2024)

Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs
by: Guo, Siyuan, et al.
Published: (2024)

Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?
by: Park, Simon, et al.
Published: (2025)

Revisiting In-Context Learning with Long Context Language Models
by: Baek, Jinheon, et al.
Published: (2024)

Learning Dynamics of LLM Finetuning
by: Ren, Yi, et al.
Published: (2024)

Internalizing World Models via Self-Play Finetuning for Agentic RL
by: Chen, Shiqi, et al.
Published: (2025)

ApiQ: Finetuning of 2-Bit Quantized Large Language Model
by: Liao, Baohao, et al.
Published: (2024)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
by: Page, Saurabh, et al.
Published: (2025)