Cover Image

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hao, Jie, Yu, Rui, Zhang, Wei, Wang, Huixia, Xu, Jie, Liu, Mingrui
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2024)

Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
by: Hao, Jie, et al.
Published: (2024)

On the Convergence of Adam-Type Algorithm for Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2025)

A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
by: Gong, Xiaochuan, et al.
Published: (2024)

Bilevel Optimization with Lower-Level Uniform Convexity: Theory and Algorithm
by: Wu, Yuman, et al.
Published: (2026)

Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training
by: Hao, Jie, et al.
Published: (2025)

MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
by: Yu, Zichun, et al.
Published: (2024)

Group-Level Data Selection for Efficient Pretraining
by: Yu, Zichun, et al.
Published: (2025)

Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
by: Gong, Xiaochuan, et al.
Published: (2025)

Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model
by: Yan, Hao, et al.
Published: (2024)

Personalized Federated Fine-tuning for Heterogeneous Data: An Automatic Rank Learning Approach via Two-Level LoRA
by: Hao, Jie, et al.
Published: (2025)

A Bayesian Model Selection Criterion for Selecting Pretraining Checkpoints
by: Munn, Michael, et al.
Published: (2024)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
by: Sam, Dylan, et al.
Published: (2025)

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting
by: Pan, Rui, et al.
Published: (2024)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
by: Chen, Zhixun, et al.
Published: (2025)

Privacy-Utility Trade-off in Data Publication: A Bilevel Optimization Framework with Curvature-Guided Perturbation
by: Yin, Yi, et al.
Published: (2025)

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
by: Wang, Qibin, et al.
Published: (2024)

Federated Learning for ICD Classification with Lightweight Models and Pretrained Embeddings
by: Xu, Binbin, et al.
Published: (2025)

Language Generation with Strictly Proper Scoring Rules
by: Shao, Chenze, et al.
Published: (2024)

FedNano: Toward Lightweight Federated Tuning for Pretrained Multimodal Large Language Models
by: Zhang, Yao, et al.
Published: (2025)

A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization
by: Chu, Tianshu, et al.
Published: (2025)

Effective Bilevel Optimization via Minimax Reformulation
by: Wang, Xiaoyu, et al.
Published: (2023)

First-Order Methods for Linearly Constrained Bilevel Optimization
by: Kornowski, Guy, et al.
Published: (2024)

Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models
by: Lin, Jinxu, et al.
Published: (2024)

Multi-Objective Bilevel Learning
by: Zhang, Zhiyao, et al.
Published: (2025)

TPLLM: A Traffic Prediction Framework Based on Pretrained Large Language Models
by: Ren, Yilong, et al.
Published: (2024)

qNBO: quasi-Newton Meets Bilevel Optimization
by: Fang, Sheng, et al.
Published: (2025)

GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models
by: Zhang, Baoquan, et al.
Published: (2025)

Exploring Multiple High-Scoring Subspaces in Generative Flow Networks
by: Yu, Xuan, et al.
Published: (2026)

Dynamic Influence Tracker: Measuring Time-Varying Sample Influence During Training
by: Xu, Jie, et al.
Published: (2025)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection
by: Messmer, Bettina, et al.
Published: (2025)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
by: Shen, Han, et al.
Published: (2024)

Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimization
by: Wang, Jie
Published: (2025)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining
by: Bal, Melis Ilayda, et al.
Published: (2025)

A Lightweight Sparse Interaction Network for Time Series Forecasting
by: Zhang, Xu, et al.
Published: (2026)

Learning with Complementary Labels Revisited: The Selected-Completely-at-Random Setting Is More Practical
by: Wang, Wei, et al.
Published: (2023)

Pre-trained Language Model and Knowledge Distillation for Lightweight Sequential Recommendation
by: Li, Li, et al.
Published: (2024)

PolicyEvolve: Evolving Programmatic Policies by LLMs for multi-player games via Population-Based Training
by: Lv, Mingrui, et al.
Published: (2025)

PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
by: Lin, Qihao, et al.
Published: (2026)