Copertina

Benchmarking Distilled Language Models: Performance and Efficiency in Resource-Constrained Settings

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wani, Sachin Gopal, Page, Eric, Dholakia, Ajay, Ellison, David
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Leveraging Large Language Models for Web Scraping
di: Ahluwalia, Aman, et al.
Pubblicazione: (2024)

Generalizing Large Language Model Usability Across Resource-Constrained
di: Tsai, Yun-Da
Pubblicazione: (2025)

Kakugo: Distillation of Low-Resource Languages into Small Language Models
di: Devine, Peter, et al.
Pubblicazione: (2026)

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models
di: Page, Saurabh, et al.
Pubblicazione: (2025)

When Should a Language Model Trust Itself? Same-Model Self-Verification as a Conditional Confidence Signal
di: Phalod, Aditya Ajay
Pubblicazione: (2026)

Predicting Machine Translation Performance on Low-Resource Languages: The Role of Domain Similarity
di: Khiu, Eric, et al.
Pubblicazione: (2024)

Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
di: Kaddour, Jean, et al.
Pubblicazione: (2023)

Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models
di: Kumar, Sachin
Pubblicazione: (2026)

Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence
di: Hager, Sophia, et al.
Pubblicazione: (2025)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
di: Cao, Mingyu, et al.
Pubblicazione: (2024)

Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
di: Zhao, Siyan, et al.
Pubblicazione: (2026)

Optimization Strategies for Enhancing Resource Efficiency in Transformers & Large Language Models
di: Wallace, Tom, et al.
Pubblicazione: (2025)

BanglaEmbed: Efficient Sentence Embedding Models for a Low-Resource Language Using Cross-Lingual Distillation Techniques
di: Kabir, Muhammad Rafsan, et al.
Pubblicazione: (2024)

metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models
di: Kipnis, Alex, et al.
Pubblicazione: (2024)

Comparative Analysis of Different Efficient Fine Tuning Methods of Large Language Models (LLMs) in Low-Resource Setting
di: Srinivasan, Krishna Prasad Varadarajan, et al.
Pubblicazione: (2024)

DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation
di: Maekawa, Aru, et al.
Pubblicazione: (2024)

The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance?
di: Banerjee, Sourav, et al.
Pubblicazione: (2024)

Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios?
di: Riabi, Arij, et al.
Pubblicazione: (2021)

On Importance of Pruning and Distillation for Efficient Low Resource NLP
di: Mirashi, Aishwarya, et al.
Pubblicazione: (2024)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
di: Fang, Luyang, et al.
Pubblicazione: (2025)

A Survey of On-Policy Distillation for Large Language Models
di: Song, Mingyang, et al.
Pubblicazione: (2026)

Unsupervised Pretraining for Fact Verification by Language Model Distillation
di: Bazaga, Adrián, et al.
Pubblicazione: (2023)

Adversarial Moment-Matching Distillation of Large Language Models
di: Jia, Chen
Pubblicazione: (2024)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
di: Li, Zeman, et al.
Pubblicazione: (2024)

Knowledge Distillation from Large Language Models for Household Energy Modeling
di: Takrouri, Mohannad, et al.
Pubblicazione: (2025)

Cache & Distil: Optimising API Calls to Large Language Models
di: Ramírez, Guillem, et al.
Pubblicazione: (2023)

A Survey on Symbolic Knowledge Distillation of Large Language Models
di: Acharya, Kamal, et al.
Pubblicazione: (2024)

Distilling Large Language Models for Text-Attributed Graph Learning
di: Pan, Bo, et al.
Pubblicazione: (2024)

Language Model Knowledge Distillation for Efficient Question Answering in Spanish
di: Bazaga, Adrián, et al.
Pubblicazione: (2023)

Self-Refining Language Model Anonymizers via Adversarial Distillation
di: Kim, Kyuyoung, et al.
Pubblicazione: (2025)

Cross-Tokenizer Likelihood Scoring Algorithms for Language Model Distillation
di: Phan, Buu, et al.
Pubblicazione: (2025)

MiniDisc: Minimal Distillation Schedule for Language Model Compression
di: Zhang, Chen, et al.
Pubblicazione: (2022)

Medical Concept Normalization in a Low-Resource Setting
di: Patzelt, Tim
Pubblicazione: (2024)

SODA: Semi On-Policy Black-Box Distillation for Large Language Models
di: Chen, Xiwen, et al.
Pubblicazione: (2026)

Self-Calibrating Language Models via Test-Time Discriminative Distillation
di: Hedna, Mohamed Rissal, et al.
Pubblicazione: (2026)

Few-Step Diffusion Language Models via Trajectory Self-Distillation
di: Zhang, Tunyu, et al.
Pubblicazione: (2026)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
di: Yang, Junjie, et al.
Pubblicazione: (2025)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
di: Thangarasa, Vithursan, et al.
Pubblicazione: (2024)

Iterative Layer-wise Distillation for Efficient Compression of Large Language Models
di: Kovalev, Grigory, et al.
Pubblicazione: (2025)

Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
di: Cao, Jinghan, et al.
Pubblicazione: (2026)