Cover Image

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Zihang, Pang, Tianyu, Balabanov, Oleg, Yang, Chaoqun, Huang, Tianjin, Yin, Lu, Yang, Yaoqing, Liu, Shiwei
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Learning to Discover Iterative Spectral Algorithms
by: Liu, Zihang, et al.
Published: (2026)

Model Balancing Helps Low-data Training and Fine-tuning
by: Liu, Zihang, et al.
Published: (2024)

LIFT: LLM-Based Pragma Insertion for HLS via GNN Supervised Fine-Tuning
by: Prakriya, Neha, et al.
Published: (2025)

RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization
by: Deng, Shenyang, et al.
Published: (2026)

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction
by: Pang, Tianyu, et al.
Published: (2026)

LIFT: Last-Mile Fine-Tuning for Table Explicitation
by: Khaitan, Divij, et al.
Published: (2026)

LIFT+: Lightweight Fine-Tuning for Long-Tail Learning
by: Shi, Jiang-Xin, et al.
Published: (2025)

Suspicious Alignment of SGD: A Fine-Grained Step Size Condition Analysis
by: Deng, Shenyang, et al.
Published: (2026)

From Spikes to Heavy Tails: Unveiling the Spectral Evolution of Neural Networks
by: Kothapalli, Vignesh, et al.
Published: (2024)

Randomized block Gram-Schmidt process for solution of linear systems and eigenvalue problems
by: Balabanov, Oleg, et al.
Published: (2021)

LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning
by: Zhang, Hongbin, et al.
Published: (2024)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2024)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
by: Pang, Jinlong, et al.
Published: (2025)

LIFT: A Novel Framework for Enhancing Long-Context Understanding of LLMs via Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2025)

Depth, Not Data: An Analysis of Hessian Spectral Bifurcation
by: Deng, Shenyang, et al.
Published: (2026)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
by: Hsiung, Lei, et al.
Published: (2025)

Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
by: Yang, Zhaorui, et al.
Published: (2024)

Random sketching of operators with application to learning preconditioners
by: Balabanov, Oleg, et al.
Published: (2021)

On-Policy Supervised Fine-Tuning for Efficient Reasoning
by: Zhao, Anhao, et al.
Published: (2026)

RL4RLA: Teaching ML to Discover Randomized Linear Algebra Algorithms Through Curriculum Design and Graph-Based Search
by: Xiong, Jinglong, et al.
Published: (2026)

Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
by: Pang, Tianyu, et al.
Published: (2026)

Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models
by: Fernandez-Lopez, Adriana, et al.
Published: (2024)

Anchored Supervised Fine-Tuning
by: Zhu, He, et al.
Published: (2025)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

Parameter-Efficient Fine-Tuning With Adapters
by: Chen, Keyu, et al.
Published: (2024)

Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning
by: Xu, Xin, et al.
Published: (2025)

Outcome of transanal advancement flap repair (TAFR) as salvage therapy after failed ligation of intersphincteric fistula tract (LIFT)—A proposed algorithm after failed LIFT
by: E. Ergüder, et al.
Published: (2025)

AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
by: Lu, Haiquan, et al.
Published: (2024)

UFT: Unifying Supervised and Reinforcement Fine-Tuning
by: Liu, Mingyang, et al.
Published: (2025)

SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training
by: Huang, Tianjin, et al.
Published: (2025)

Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning
by: Cong, Peizhuang, et al.
Published: (2025)

Block subsampled randomized Hadamard transform for low-rank approximation on distributed architectures
by: Balabanov, Oleg, et al.
Published: (2022)

Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth
by: Zhang, Yichi, et al.
Published: (2025)

Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness of Smoothed Classifiers
by: Jin, Gaojie, et al.
Published: (2025)

FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence
by: Wan, Guoan, et al.
Published: (2025)

O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
by: Luo, Haotian, et al.
Published: (2025)

MathSE: Improving Multimodal Mathematical Reasoning via Self-Evolving Iterative Reflection and Reward-Guided Fine-Tuning
by: Chen, Jinhao, et al.
Published: (2025)

Response to: Outcome of transanal advancement flap repair ( TAFR ) as salvage therapy after failed ligation of intersphincteric fistula tract ( LIFT )—A proposed algorithm after failed LIFT —A video vignette
by: Vipul D. Yagnik
Published: (2025)

RPO:Reinforcement Fine-Tuning with Partial Reasoning Optimization
by: Yi, Hongzhu, et al.
Published: (2026)

Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation
by: Wu, Fei, et al.
Published: (2025)