Cover Image

Sensitivity of Small Language Models to Fine-tuning Data Contamination

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Scaria, Nicy, Kennedy, Silvester John Joseph, Subramani, Deepak
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcripts
by: Scaria, Nicy, et al.
Published: (2024)

Learning in Blocks: A Multi Agent Debate Assisted Personalized Adaptive Learning Framework for Language Learning
by: Scaria, Nicy, et al.
Published: (2026)

Dissecting Physics Reasoning in Small Language Models: A Multi-Dimensional Analysis from an Educational Perspective
by: Scaria, Nicy, et al.
Published: (2025)

Can Small Language Models Learn, Unlearn, and Retain Noise Patterns?
by: Scaria, Nicy, et al.
Published: (2024)

Harnessing Structured Knowledge: A Concept Map-Based Approach for High-Quality Multiple Choice Question Generation with Effective Distractors
by: Scaria, Nicy, et al.
Published: (2025)

Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation
by: Scaria, Nicy, et al.
Published: (2024)

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
by: Kim, Kyeonghyun, et al.
Published: (2025)

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2023)

Exploring the Knowledge Mismatch Hypothesis: Hallucination Propensity in Small Models Fine-tuned on Data from Larger Models
by: Wee, Phil, et al.
Published: (2024)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelers
by: Kang, Yue, et al.
Published: (2026)

SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks
by: Khan, Muhammad Junaid, et al.
Published: (2024)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction
by: Li, Yucheng, et al.
Published: (2023)

Federated Co-tuning Framework for Large and Small Language Models
by: Fan, Tao, et al.
Published: (2024)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
by: Wang, Yibo, et al.
Published: (2025)

Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models
by: Ji, Xiaoshuang, et al.
Published: (2025)

Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation
by: Bellaouar, Slimane, et al.
Published: (2025)

Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
by: Zhou, Andy
Published: (2025)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
by: Lyu, Yougang, et al.
Published: (2024)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

An Open Source Data Contamination Report for Large Language Models
by: Li, Yucheng, et al.
Published: (2023)

Investigating Data Contamination in Modern Benchmarks for Large Language Models
by: Deng, Chunyuan, et al.
Published: (2023)

Locale-Conditioned Few-Shot Prompting Mitigates Demonstration Regurgitation in On-Device PII Substitution with Small Language Models
by: Sadani, Anuj, et al.
Published: (2026)

You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Models
by: He, Wenchong, et al.
Published: (2025)

Prompting or Fine-tuning? Exploring Large Language Models for Causal Graph Validation
by: Susanti, Yuni, et al.
Published: (2024)

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
by: Huang, Shih-Cheng, et al.
Published: (2022)

Comparing Explanation Faithfulness between Multilingual and Monolingual Fine-tuned Language Models
by: Zhao, Zhixue, et al.
Published: (2024)

On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models
by: Sun, Qian, et al.
Published: (2024)

Fine-tuning vs Prompting, Can Language Models Understand Human Values?
by: Sun, Pingwei
Published: (2024)

Data Contamination Can Cross Language Barriers
by: Yao, Feng, et al.
Published: (2024)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide
by: Szep, Marton, et al.
Published: (2024)

Information Guided Regularization for Fine-tuning Language Models
by: Sharma, Mandar, et al.
Published: (2024)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
by: Yu, Zhang Ze, et al.
Published: (2023)

Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic
by: Bhardwaj, Rishabh, et al.
Published: (2024)

Fine-tuning Language Models for Recipe Generation: A Comparative Analysis and Benchmark Study
by: Vij, Anneketh, et al.
Published: (2025)

Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection
by: Rahman, Md Abdur, et al.
Published: (2024)