Cover Image

Learning Mechanism Underlying NLP Pre-Training and Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tzach, Yarden, Gross, Ronit D., Koresh, Ella, Rosner, Shalom, Shpringer, Or, Halevi, Tal, Kanter, Ido
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Tiny language models
by: Gross, Ronit D., et al.
Published: (2025)

Self-attention vector output similarities reveal how machines pay attention
by: Halevi, Tal, et al.
Published: (2025)

Single-Nodal Spontaneous Symmetry Breaking in NLP Models
by: Rosner, Shalom, et al.
Published: (2026)

Low-latency vision transformers via large-scale multi-head attention
by: Gross, Ronit D., et al.
Published: (2025)

Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi
by: Koresh, Ella, et al.
Published: (2025)

Advanced deep architecture pruning using single filter performance
by: Tzach, Yarden, et al.
Published: (2025)

Translation Entropy: A Statistical Framework for Evaluating Translation Systems
by: Gross, Ronit D., et al.
Published: (2025)

Towards a universal mechanism for successful deep learning
by: Meir, Yuval, et al.
Published: (2023)

Role of Delay in Brain Dynamics
by: Meir, Yuval, et al.
Published: (2024)

Small LLMs for Medical NLP: a Systematic Analysis of Few-Shot, Constraint Decoding, Fine-Tuning and Continual Pre-Training in Italian
by: Ferrazzi, Pietro, et al.
Published: (2026)

Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking
by: Prakash, Nikhil, et al.
Published: (2024)

Fine-Tuning Pre-Trained Code Models for AI-Generated Code Detection
by: Ispas, Jany-Gabriel, et al.
Published: (2026)

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs
by: Jindal, Ishan, et al.
Published: (2024)

Legal Documents Drafting with Fine-Tuned Pre-Trained Large Language Model
by: Lin, Chun-Hsien, et al.
Published: (2024)

Build, Borrow, or Just Fine-Tune? A Political Scientist's Guide to Choosing NLP Models
by: Meher, Shreyas
Published: (2026)

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models
by: Doering, Nigel, et al.
Published: (2024)

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging
by: Farn, Hua, et al.
Published: (2024)

LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs
by: Kim, Taeho, et al.
Published: (2024)

Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement
by: Yu, Le, et al.
Published: (2024)

SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
by: Sun, Chengwei, et al.
Published: (2024)

Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models
by: Sun, Kaiser, et al.
Published: (2024)

EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints
by: Geng, Jiaxiang, et al.
Published: (2026)

QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning
by: AL-Smadi, Mohammad
Published: (2026)

Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
by: Rannon, Ella, et al.
Published: (2025)

DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection
by: Das, Devleena, et al.
Published: (2023)

User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuning
by: Otsuka, Atsushi, et al.
Published: (2024)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
by: Yano, Kazuki, et al.
Published: (2026)

Fine-Tuning BERT for Domain-Specific Question Answering: Toward Educational NLP Resources at University Scale
by: Montfrond, Aurélie
Published: (2025)

UW-BioNLP at ChemoTimelines 2025: Thinking, Fine-Tuning, and Dictionary-Enhanced LLM Systems for Chemotherapy Timeline Extraction
by: Zhang, Tianmai M., et al.
Published: (2025)

Attribute First, then Generate: Locally-attributable Grounded Text Generation
by: Slobodkin, Aviv, et al.
Published: (2024)

Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks
by: Hu, Jiacheng, et al.
Published: (2024)

Training Prompt Matters: State-Adaptive Optimization for Robust Fine-Tuning
by: Shi, Wenhang, et al.
Published: (2026)

PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning
by: Pentyala, Shiva Kumar, et al.
Published: (2024)

InfiMed-Foundation: Pioneering Advanced Multimodal Medical Models with Compute-Efficient Pre-Training and Multi-Stage Fine-Tuning
by: Zhu, Guanghao, et al.
Published: (2025)

MELABenchv1: Benchmarking Large Language Models against Smaller Fine-Tuned Models for Low-Resource Maltese NLP
by: Micallef, Kurt, et al.
Published: (2025)

Recovering the Pre-Fine-Tuning Weights of Generative Models
by: Horwitz, Eliahu, et al.
Published: (2024)

CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning
by: Ye, Yangfan, et al.
Published: (2025)

IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages
by: Khan, Mohammed Safi Ur Rahman, et al.
Published: (2024)

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
by: Patel, Ajay, et al.
Published: (2026)

Fine-Tuning Pre-trained Language Models to Detect In-Game Trash Talks
by: Fesalbon, Daniel, et al.
Published: (2024)