Cover Image

Exploring the Knowledge Mismatch Hypothesis: Hallucination Propensity in Small Models Fine-tuned on Data from Larger Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wee, Phil, Baghdadi, Riyadh
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning
by: Juneja, Gurusha, et al.
Published: (2023)

Curriculum Learning for Small Code Language Models
by: Naïr, Marwa, et al.
Published: (2024)

Sensitivity of Small Language Models to Fine-tuning Data Contamination
by: Scaria, Nicy, et al.
Published: (2025)

Small Updates, Big Doubts: Does Parameter-Efficient Fine-tuning Enhance Hallucination Detection ?
by: Hu, Xu, et al.
Published: (2026)

Seamless Deception: Larger Language Models Are Better Knowledge Concealers
by: Ashok, Dhananjay, et al.
Published: (2026)

Data-efficient Performance Modeling via Pre-training
by: Liu, Chunting, et al.
Published: (2025)

Exploring Memorization in Fine-tuned Language Models
by: Zeng, Shenglai, et al.
Published: (2023)

Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models
by: Lv, Qitan, et al.
Published: (2024)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
by: Tao, Chaofan, et al.
Published: (2024)

Knowledge AI: Fine-tuning NLP Models for Facilitating Scientific Knowledge Extraction and Understanding
by: Muralidharan, Balaji, et al.
Published: (2024)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
by: Lyu, Yougang, et al.
Published: (2024)

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
by: Kim, Kyeonghyun, et al.
Published: (2025)

Prompting or Fine-tuning? Exploring Large Language Models for Causal Graph Validation
by: Susanti, Yuni, et al.
Published: (2024)

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
by: Setty, Vinay
Published: (2024)

Hallucination Detection with Small Language Models
by: Cheung, Ming
Published: (2025)

Neural Architecture Search with Mixed Bio-inspired Learning Rules
by: Hamzaoui, Imane, et al.
Published: (2025)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
by: Yang, Yu, et al.
Published: (2024)

Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelers
by: Kang, Yue, et al.
Published: (2026)

Honest AI: Fine-Tuning "Small" Language Models to Say "I Don't Know", and Reducing Hallucination in RAG
by: Chen, Xinxi, et al.
Published: (2024)

Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models
by: Xiong, Guangzhi, et al.
Published: (2025)

SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks
by: Khan, Muhammad Junaid, et al.
Published: (2024)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

Beyond Fine-Tuning: Effective Strategies for Mitigating Hallucinations in Large Language Models for Data Analytics
by: Rumiantsau, Mikhail, et al.
Published: (2024)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
by: Zhang, Yudi, et al.
Published: (2025)

Federated Co-tuning Framework for Large and Small Language Models
by: Fan, Tao, et al.
Published: (2024)

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models
by: Luo, Haoran, et al.
Published: (2023)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
by: Wang, Yibo, et al.
Published: (2025)

Noise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Models
by: Khadangi, Afshin, et al.
Published: (2025)

Sign-Symmetry Learning Rules are Robust Fine-Tuners
by: Berriche, Aymene, et al.
Published: (2025)

Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research
by: Martinez, Richard Diehl, et al.
Published: (2025)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

Improving Scientific Hypothesis Generation with Knowledge Grounded Large Language Models
by: Xiong, Guangzhi, et al.
Published: (2024)

Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs
by: Su, Siyue, et al.
Published: (2026)

Progtuning: Progressive Fine-tuning Framework for Transformer-based Language Models
by: Ji, Xiaoshuang, et al.
Published: (2025)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation
by: Bellaouar, Slimane, et al.
Published: (2025)

Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
by: Zhou, Andy
Published: (2025)

You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Models
by: He, Wenchong, et al.
Published: (2025)

Fine-tuning Language Models with Generative Adversarial Reward Modelling
by: Yu, Zhang Ze, et al.
Published: (2023)

From Hallucinations to Facts: Enhancing Language Models with Curated Knowledge Graphs
by: Joshi, Ratnesh Kumar, et al.
Published: (2024)