Copertina

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Qibin, Hu, Xiaolin, Xu, Weikai, Liu, Wei, Luan, Jian, Wang, Bin
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge
di: Zhao, Xuyang, et al.
Pubblicazione: (2024)

DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models
di: Hu, Xiaolin, et al.
Pubblicazione: (2024)

Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
di: Zhou, Huichi, et al.
Pubblicazione: (2025)

LLM-AutoDP: Automatic Data Processing via LLM Agents for Model Fine-tuning
di: Huang, Wei, et al.
Pubblicazione: (2026)

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning
di: Zou, Heming, et al.
Pubblicazione: (2025)

A General Framework to Enhance Fine-tuning-based LLM Unlearning
di: Ren, Jie, et al.
Pubblicazione: (2025)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
di: Zhang, Hengxiang, et al.
Pubblicazione: (2024)

Two-stage LLM Fine-tuning with Less Specialization and More Generalization
di: Wang, Yihan, et al.
Pubblicazione: (2022)

Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay
di: Sun, Yifan, et al.
Pubblicazione: (2025)

Outlier-weighed Layerwise Sampling for LLM Fine-tuning
di: Li, Pengxiang, et al.
Pubblicazione: (2024)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
di: Shen, Han, et al.
Pubblicazione: (2024)

Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
di: Wang, Dongwei, et al.
Pubblicazione: (2024)

DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
di: Huang, Wei, et al.
Pubblicazione: (2025)

Enhancing Multilingual LLM Pretraining with Model-Based Data Selection
di: Messmer, Bettina, et al.
Pubblicazione: (2025)

Information Guided Regularization for Fine-tuning Language Models
di: Sharma, Mandar, et al.
Pubblicazione: (2024)

HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
di: Chen, Yuhan, et al.
Pubblicazione: (2024)

SEE: Continual Fine-tuning with Sequential Ensemble of Experts
di: Wang, Zhilin, et al.
Pubblicazione: (2025)

GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning
di: Solatorio, Aivin V.
Pubblicazione: (2024)

CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning
di: Feng, Jinyuan, et al.
Pubblicazione: (2025)

On the Loss of Context-awareness in General Instruction Fine-tuning
di: Wang, Yihan, et al.
Pubblicazione: (2024)

Vaccine: Perturbation-aware Alignment for Large Language Models against Harmful Fine-tuning Attack
di: Huang, Tiansheng, et al.
Pubblicazione: (2024)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
di: Hu, Yujia, et al.
Pubblicazione: (2024)

S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning
di: Zeng, Hanqing, et al.
Pubblicazione: (2025)

Exploring Memorization in Fine-tuned Language Models
di: Zeng, Shenglai, et al.
Pubblicazione: (2023)

Fine-tuning Large Language Models for Domain-specific Machine Translation
di: Zheng, Jiawei, et al.
Pubblicazione: (2024)

GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
di: Chen, Tianhao, et al.
Pubblicazione: (2025)

Advancing Parameter Efficiency in Fine-tuning via Representation Editing
di: Wu, Muling, et al.
Pubblicazione: (2024)

Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets
di: Li, Dongyue, et al.
Pubblicazione: (2025)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
di: Lin, Haowei, et al.
Pubblicazione: (2024)

SASA: Semantic-Aware Contrastive Learning Framework with Separated Attention for Triple Classification
di: Xiaodan, Xu, et al.
Pubblicazione: (2026)

Latent Traits and Cross-Task Transfer: Deconstructing Dataset Interactions in LLM Fine-tuning
di: Krishna, Shambhavi, et al.
Pubblicazione: (2025)

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
di: Tan, Qitao, et al.
Pubblicazione: (2025)

Dissecting Outlier Dynamics in LLM NVFP4 Pretraining
di: Dong, Peijie, et al.
Pubblicazione: (2026)

Strong Teacher Not Needed? On Distillation in LLM Pretraining
di: Lu, Taiming, et al.
Pubblicazione: (2026)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
di: Perin, Gabriel J., et al.
Pubblicazione: (2025)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
di: Hsiung, Lei, et al.
Pubblicazione: (2025)

Selective Fine-tuning on LLM-labeled Data May Reduce Reliance on Human Annotation: A Case Study Using Schedule-of-Event Table Detection
di: Kumar, Bhawesh, et al.
Pubblicazione: (2024)

Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models
di: Yao, Kai, et al.
Pubblicazione: (2024)

Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
di: Wang, Guoli, et al.
Pubblicazione: (2026)

APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
di: Marín, Javier
Pubblicazione: (2025)