Cover Image

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Somayajula, Sai Ashish, Liang, Youwei, Singh, Abhishek, Zhang, Li, Xie, Pengtao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

AutoLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning
by: Zhang, Ruiyi, et al.
Published: (2024)

Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models
by: Huo, Mingjia, et al.
Published: (2024)

Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning
by: Hu, Bokai, et al.
Published: (2024)

TapWeight: Reweighting Pretraining Objectives for Task-Adaptive Pretraining
by: Zhang, Ruiyi, et al.
Published: (2024)

DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning
by: Cao, Qi, et al.
Published: (2025)

Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs
by: Gao, Xin, et al.
Published: (2025)

Stacking Small Language Models for Generalizability
by: Liang, Laurence
Published: (2024)

IndiText Boost: Text Augmentation for Low Resource India Languages
by: Litake, Onkar, et al.
Published: (2024)

Large Language Models to Diffusion Finetuning
by: Cetin, Edoardo, et al.
Published: (2025)

DreamPRM-Code: Function-as-Step Process Reward Model with Label Correction for LLM Coding
by: Zhang, Ruiyi, et al.
Published: (2025)

Vanishing Gradients in Reinforcement Finetuning of Language Models
by: Razin, Noam, et al.
Published: (2023)

ReFT: Representation Finetuning for Language Models
by: Wu, Zhengxuan, et al.
Published: (2024)

HMI: Hierarchical Knowledge Management for Efficient Multi-Tenant Inference in Pretrained Language Models
by: Zhang, Jun, et al.
Published: (2025)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Models
by: Li, Shuozhe, et al.
Published: (2026)

Sequences of Logits Reveal the Low Rank Structure of Language Models
by: Golowich, Noah, et al.
Published: (2025)

Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model
by: Liang, Jing, et al.
Published: (2025)

Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
by: Chen, Yanda, et al.
Published: (2024)

Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
by: McLeish, Sean, et al.
Published: (2025)

Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning
by: Xu, Zhuoyan, et al.
Published: (2024)

Unfamiliar Finetuning Examples Control How Language Models Hallucinate
by: Kang, Katie, et al.
Published: (2024)

Transfer Learning of Tabular Data by Finetuning Large Language Models
by: Rabbani, Shourav B., et al.
Published: (2025)

AI-Generated Text Detection in Low-Resource Languages: A Case Study on Urdu
by: Ammar, Muhammad, et al.
Published: (2025)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
by: Devine, Peter, et al.
Published: (2026)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
by: Liu, Z, et al.
Published: (2024)

BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
by: Merzah, Baqer M., et al.
Published: (2025)

Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation
by: Das, Devleena, et al.
Published: (2025)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
by: Bhatt, Gantavya, et al.
Published: (2024)

Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector
by: Zhang, Andi, et al.
Published: (2024)

FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
by: Liu, Xiao-Yang, et al.
Published: (2024)

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
by: Hong, Matthew M., et al.
Published: (2026)

Large Multimodal Models for Low-Resource Languages: A Survey
by: Lupascu, Marian, et al.
Published: (2025)

Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning
by: Ye, Jiasheng, et al.
Published: (2023)

Pretraining Large Language Models with NVFP4
by: NVIDIA, et al.
Published: (2025)

In-context Pretraining: Language Modeling Beyond Document Boundaries
by: Shi, Weijia, et al.
Published: (2023)

Patent Language Model Pretraining with ModernBERT
by: Yousefiramandi, Amirhossein, et al.
Published: (2025)

Output Embedding Centering for Stable LLM Pretraining
by: Stollenwerk, Felix, et al.
Published: (2026)

On the Reasoning Abilities of Masked Diffusion Language Models
by: Svete, Anej, et al.
Published: (2025)

Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
by: Ulm, Jannek, et al.
Published: (2025)

Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
by: Ghaffarzadeh-Esfahani, Mohammadreza, et al.
Published: (2026)