Copertina

What Should Feature Distillation Transfer in LLMs? A Task-Tangent Geometry View

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Saadi, Khouloud, Wang, Di
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Validity-Calibrated Reasoning Distillation
di: Saadi, Khouloud, et al.
Pubblicazione: (2026)

JGU Mainz's Submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: MT and QA
di: Saadi, Hossain Shaikh, et al.
Pubblicazione: (2025)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
di: Yang, Junjie, et al.
Pubblicazione: (2025)

Distilling Text Style Transfer With Self-Explanation From LLMs
di: Zhang, Chiyu, et al.
Pubblicazione: (2024)

"Yeah Right!" -- Do LLMs Exhibit Multimodal Feature Transfer?
di: Reichman, Benjamin, et al.
Pubblicazione: (2025)

What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
di: Staufer, Dimitri
Pubblicazione: (2025)

What if LLMs Have Different World Views: Simulating Alien Civilizations with LLM-based Agents
di: Xue, Zhaoqian, et al.
Pubblicazione: (2024)

CoMMET: To What Extent Can LLMs Perform Theory of Mind Tasks?
di: Chen, Ruirui, et al.
Pubblicazione: (2026)

Exploring the Limits of Model Compression in LLMs: A Knowledge Distillation Study on QA Tasks
di: Datta, Joyeeta, et al.
Pubblicazione: (2025)

Feature Structure Distillation with Centered Kernel Alignment in BERT Transferring
di: Jung, Hee-Jun, et al.
Pubblicazione: (2022)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs
di: Wu, Xinwei, et al.
Pubblicazione: (2026)

Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance
di: Yin, Ziqi, et al.
Pubblicazione: (2024)

Divide-or-Conquer? Which Part Should You Distill Your LLM?
di: Wu, Zhuofeng, et al.
Pubblicazione: (2024)

Hybrid Policy Distillation for LLMs
di: Zhu, Wenhong, et al.
Pubblicazione: (2026)

Selecting Auxiliary Data via Neural Tangent Kernels for Low-Resource Domains
di: Wang, Pingjie, et al.
Pubblicazione: (2025)

LLMs Should Express Uncertainty Explicitly
di: Guo, Junyu, et al.
Pubblicazione: (2026)

The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism
di: Song, Yifan, et al.
Pubblicazione: (2024)

Cross-Examination Framework: A Task-Agnostic Diagnostic for Information Fidelity in Text-to-Text Generation
di: Raha, Tathagata, et al.
Pubblicazione: (2026)

LLMs Should Incorporate Explicit Mechanisms for Human Empathy
di: You, Xiaoxing, et al.
Pubblicazione: (2026)

Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks
di: Bao, Yuntai, et al.
Pubblicazione: (2025)

Bridging Language Barriers in Healthcare: A Study on Arabic LLMs
di: Saadi, Nada, et al.
Pubblicazione: (2025)

Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs
di: Boizard, Nicolas, et al.
Pubblicazione: (2024)

Keypoint-based Progressive Chain-of-Thought Distillation for LLMs
di: Feng, Kaituo, et al.
Pubblicazione: (2024)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
di: Tian, Yijun, et al.
Pubblicazione: (2024)

Exploring the Personality Traits of LLMs through Latent Features Steering
di: Yang, Shu, et al.
Pubblicazione: (2024)

The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?
di: Tang, Zhenheng, et al.
Pubblicazione: (2025)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
di: Chen, Jennifer, et al.
Pubblicazione: (2025)

Retrieval Augmented Question Answering: When Should LLMs Admit Ignorance?
di: Wang, Dingmin, et al.
Pubblicazione: (2025)

BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
di: Du, Dayou, et al.
Pubblicazione: (2024)

Dense X Retrieval: What Retrieval Granularity Should We Use?
di: Chen, Tong, et al.
Pubblicazione: (2023)

Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning
di: Yue, Yuanhao, et al.
Pubblicazione: (2024)

Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models
di: Wang, Shuo, et al.
Pubblicazione: (2024)

Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation
di: Klimaszewski, Mateusz, et al.
Pubblicazione: (2024)

Training Task Experts through Retrieval Based Distillation
di: Ge, Jiaxin, et al.
Pubblicazione: (2024)

What Does Neuro Mean to Cardio? Investigating the Role of Clinical Specialty Data in Medical LLMs
di: Yan, Xinlan, et al.
Pubblicazione: (2025)

What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks
di: Kirch, Nathalie, et al.
Pubblicazione: (2024)

Efficient Knowledge Transfer in Multi-Task Learning through Task-Adaptive Low-Rank Representation
di: Zhang, Xiao, et al.
Pubblicazione: (2025)

SELT: Self-Evaluation Tree Search for LLMs with Task Decomposition
di: Wu, Mengsong, et al.
Pubblicazione: (2025)

LLMs as Implicit Imputers: Uncertainty Should Scale with Missing Information
di: van Buuren, Stef
Pubblicazione: (2026)