Imagen de Portada

Tracr-Injection: Distilling Algorithms into Pre-trained Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Vergara-Browne, Tomás, Soto, Álvaro
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

TracrBench: Generating Interpretability Testbeds with Large Language Models
por: Thurnherr, Hannes, et al.
Publicado: (2024)

GenDistiller: Distilling Pre-trained Language Models based on an Autoregressive Generative Model
por: Gao, Yingying, et al.
Publicado: (2024)

Large Language Models are biased to overestimate profoundness
por: Herrera-Berg, Eugenio, et al.
Publicado: (2023)

Pre-training Distillation for Large Language Models: A Design Space Exploration
por: Peng, Hao, et al.
Publicado: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
por: Zhang, Ying, et al.
Publicado: (2024)

Detection Method for Prompt Injection by Integrating Pre-trained Model and Heuristic Feature Engineering
por: Ji, Yi, et al.
Publicado: (2025)

From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP
por: Mosbach, Marius, et al.
Publicado: (2024)

Metadata Conditioning Accelerates Language Model Pre-training
por: Gao, Tianyu, et al.
Publicado: (2025)

Evaluating Discourse Cohesion in Pre-trained Language Models
por: He, Jie, et al.
Publicado: (2025)

Development of Cognitive Intelligence in Pre-trained Language Models
por: Shah, Raj Sanjay, et al.
Publicado: (2024)

Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
por: Zhang, Haode, et al.
Publicado: (2022)

COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training
por: Kim, Sanghwan, et al.
Publicado: (2024)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
por: Gu, Yuxian, et al.
Publicado: (2024)

Collapse of Self-trained Language Models
por: Herel, David, et al.
Publicado: (2024)

BLSP-KD: Bootstrapping Language-Speech Pre-training via Knowledge Distillation
por: Wang, Chen, et al.
Publicado: (2024)

Stable Language Model Pre-training by Reducing Embedding Variability
por: Chung, Woojin, et al.
Publicado: (2024)

Model Merging in Pre-training of Large Language Models
por: Li, Yunshui, et al.
Publicado: (2025)

Probing Language Models for Pre-training Data Detection
por: Liu, Zhenhua, et al.
Publicado: (2024)

DEPT: Decoupled Embeddings for Pre-training Language Models
por: Iacob, Alex, et al.
Publicado: (2024)

STEP: Staged Parameter-Efficient Pre-training for Large Language Models
por: Yano, Kazuki, et al.
Publicado: (2025)

Knowledge-augmented Pre-trained Language Models for Biomedical Relation Extraction
por: Sänger, Mario, et al.
Publicado: (2025)

Cross-layer Attention Sharing for Pre-trained Large Language Models
por: Mu, Yongyu, et al.
Publicado: (2024)

From Robustness to Improved Generalization and Calibration in Pre-trained Language Models
por: Jukić, Josip, et al.
Publicado: (2024)

KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates
por: Li, Yudong, et al.
Publicado: (2026)

SparseLLM: Towards Global Pruning for Pre-trained Language Models
por: Bai, Guangji, et al.
Publicado: (2024)

Projective Methods for Mitigating Gender Bias in Pre-trained Language Models
por: Dawkins, Hillary, et al.
Publicado: (2024)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
por: Li, Chen-An, et al.
Publicado: (2024)

A Survey of Pre-trained Language Models for Processing Scientific Text
por: Ho, Xanh, et al.
Publicado: (2024)

Spoken Language Identification with Pre-trained Models and Margin Loss
por: Fang, Zhihua, et al.
Publicado: (2026)

Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks
por: Yamaguchi, Atsuki, et al.
Publicado: (2026)

Applying Pre-trained Multilingual BERT in Embeddings for Improved Malicious Prompt Injection Attacks Detection
por: Rahman, Md Abdur, et al.
Publicado: (2024)

PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
por: Ranaldi, Leonardo, et al.
Publicado: (2023)

Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
por: Tu, Zhijun, et al.
Publicado: (2025)

Pre-trained Language Models and Few-shot Learning for Medical Entity Extraction
por: Wang, Xiaokai, et al.
Publicado: (2025)

Masked Structural Growth for 2x Faster Language Model Pre-training
por: Yao, Yiqun, et al.
Publicado: (2023)

Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages
por: Bahad, Sankalp, et al.
Publicado: (2024)

Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
por: Wu, Di, et al.
Publicado: (2022)

Prompting Disentangled Embeddings for Knowledge Graph Completion with Pre-trained Language Model
por: Geng, Yuxia, et al.
Publicado: (2023)

The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models
por: Liu, Yan, et al.
Publicado: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
por: Xie, Yong, et al.
Publicado: (2023)