Cover Image

Making the Most of your Model: Methods for Finetuning and Applying Pretrained Transformers

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Yoshida, Davis
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods
by: Finkelstein, Mara, et al.
Published: (2023)

Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models
by: Balde, Gunjan, et al.
Published: (2024)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
by: Vasudeva, Bhavya, et al.
Published: (2026)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
by: Xie, Wanyun, et al.
Published: (2025)

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
by: Zhang, Biao, et al.
Published: (2024)

Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
by: Feng, Jiahai, et al.
Published: (2024)

A Tutorial on the Pretrain-Finetune Paradigm for Natural Language Processing
by: Wang, Yu, et al.
Published: (2024)

Generalizable and Stable Finetuning of Pretrained Language Models on Low-Resource Texts
by: Somayajula, Sai Ashish, et al.
Published: (2024)

Scaling Laws for Forgetting during Finetuning with Pretraining Data Injection
by: Bethune, Louis, et al.
Published: (2025)

Alignment Helps Make the Most of Multimodal Data
by: Arnold, Christian, et al.
Published: (2024)

Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
by: You, Chong, et al.
Published: (2025)

Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models
by: Horawalavithana, Sameera, et al.
Published: (2026)

ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality
by: Longpre, Shayne, et al.
Published: (2025)

A Family of Pretrained Transformer Language Models for Russian
by: Zmitrovich, Dmitry, et al.
Published: (2023)

Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision
by: Yoshida, Ryo, et al.
Published: (2024)

How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments
by: Ide, Yusuke, et al.
Published: (2024)

XL-DURel: Finetuning Sentence Transformers for Ordinal Word-in-Context Classification
by: Yadav, Sachin, et al.
Published: (2025)

SpiroLLM: Finetuning Pretrained LLMs to Understand Spirogram Time Series with Clinical Validation in COPD Reporting
by: Mei, Shuhao, et al.
Published: (2025)

Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs
by: Itzhak, Itay, et al.
Published: (2025)

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
by: Rubin, Ohad, et al.
Published: (2023)

FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction
by: Jain, Akriti, et al.
Published: (2024)

Time is Encoded in the Weights of Finetuned Language Models
by: Nylund, Kai, et al.
Published: (2023)

FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
by: Liu, Xiao-Yang, et al.
Published: (2024)

Mitigating Large Language Model Hallucination with Faithful Finetuning
by: Hu, Minda, et al.
Published: (2024)

CU-ICU: Customizing Unsupervised Instruction-Finetuned Language Models for ICU Datasets via Text-to-Text Transfer Transformer
by: Panboonyuen, Teerapong
Published: (2025)

XplaiNLP at CheckThat! 2025: Multilingual Subjectivity Detection with Finetuned Transformers and Prompt-Based Inference with Large Language Models
by: Sahitaj, Ariana, et al.
Published: (2025)

A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks
by: Labrak, Yanis, et al.
Published: (2023)

Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning
by: Liu, Z, et al.
Published: (2024)

Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
by: Üstün, Ahmet, et al.
Published: (2024)

Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers
by: Gong, Linyuan, et al.
Published: (2023)

Applying Ensemble Methods to Model-Agnostic Machine-Generated Text Detection
by: Ong, Ivan, et al.
Published: (2024)

TPTT: Transforming Pretrained Transformers into Titans
by: Furfaro, Fabien
Published: (2025)

The Hallucination Tax of Reinforcement Finetuning
by: Song, Linxin, et al.
Published: (2025)

Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
by: Tan, Bowen, et al.
Published: (2025)

Maya: An Instruction Finetuned Multilingual Multimodal Model
by: Alam, Nahid, et al.
Published: (2024)

Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
by: Hu, Xiang, et al.
Published: (2024)

Multi-Task Pre-Finetuning of Lightweight Transformer Encoders for Text Classification and NER
by: Zhu, Junyi, et al.
Published: (2025)

Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
by: Wilcoxson, Max, et al.
Published: (2024)

On Instruction-Finetuning Neural Machine Translation Models
by: Raunak, Vikas, et al.
Published: (2024)

Ensembling Finetuned Language Models for Text Classification
by: Arango, Sebastian Pineda, et al.
Published: (2024)