Cover Image

Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rouf, Nirjhor, Amin, Fin, Franzon, Paul D.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Addressing Large Action Spaces in 3D Floorplanning via Spatial Generalization
by: Amin, Fin, et al.
Published: (2024)

Can an Actor-Critic Optimization Framework Improve Analog Design Optimization?
by: Dutta, Sounak, et al.
Published: (2026)

Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization
by: Wen, Yuejiang, et al.
Published: (2025)

The Over-Certainty Phenomenon in Modern Test-Time Adaptation Algorithms
by: Amin, Fin, et al.
Published: (2024)

Can LLMs Reconcile Knowledge Conflicts in Counterfactual Reasoning
by: Yamin, Khurram, et al.
Published: (2025)

Embedding Is (Almost) All You Need: Retrieval-Augmented Inference for Generalizable Genomic Prediction Tasks
by: Datta, Nirjhor, et al.
Published: (2025)

Repetitions are not all alike: distinct mechanisms sustain repetition in language models
by: Mahaut, Matéo, et al.
Published: (2025)

Low-Rank Correction for Quantized LLMs
by: Scetbon, Meyer, et al.
Published: (2024)

Alzheimer's Disease Brain Network Mining
by: Moayedikia, Alireza, et al.
Published: (2025)

Demystifying Low-Rank Knowledge Distillation in Large Language Models: Convergence, Generalization, and Information-Theoretic Guarantees
by: Soarez, Alberlucia Rafael, et al.
Published: (2026)

Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
by: Nemani, Lakshmana Sri Harsha, et al.
Published: (2025)

Locking Pretrained Weights via Deep Low-Rank Residual Distillation
by: Sakamoto, Keitaro, et al.
Published: (2026)

FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment
by: Zaccone, Riccardo, et al.
Published: (2026)

DistilLock: Safeguarding LLMs from Unauthorized Knowledge Distillation on the Edge
by: Mohanty, Asmita, et al.
Published: (2025)

DKDL-Net: A Lightweight Bearing Fault Detection Model via Decoupled Knowledge Distillation and Low-Rank Adaptation Fine-tuning
by: Petrosian, Ovanes, et al.
Published: (2024)

KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
by: Xu, Hongling, et al.
Published: (2025)

Reinforcement-aware Knowledge Distillation for LLM Reasoning
by: Zhang, Zhaoyang, et al.
Published: (2026)

Dynamic Temperature Scheduler for Knowledge Distillation
by: Islam, Sibgat Ul, et al.
Published: (2025)

Low-Rank Similarity Mining for Multimodal Dataset Distillation
by: Xu, Yue, et al.
Published: (2024)

Efficient Knowledge Injection in LLMs via Self-Distillation
by: Kujanpää, Kalle, et al.
Published: (2024)

The emergence of numerical representations in communicating artificial agents
by: Mihai, Daniela, et al.
Published: (2026)

Ranking Reasoning LLMs under Test-Time Scaling
by: Hariri, Mohsen, et al.
Published: (2026)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
by: Tian, Yijun, et al.
Published: (2024)

Stabilizing Native Low-Rank LLM Pretraining
by: Janson, Paul, et al.
Published: (2026)

Low-Dimensional Federated Knowledge Graph Embedding via Knowledge Distillation
by: Zhang, Xiaoxiong, et al.
Published: (2024)

LQER: Low-Rank Quantization Error Reconstruction for LLMs
by: Zhang, Cheng, et al.
Published: (2024)

LLMs Can Learn to Reason Via Off-Policy RL
by: Ritter, Daniel, et al.
Published: (2026)

Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers
by: Schneider, Dominic, et al.
Published: (2024)

Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation
by: Phan, Phuc, et al.
Published: (2024)

Dual Model Deep Learning for Alzheimer Prognostication
by: Moayedikia, Alireza, et al.
Published: (2025)

Low-Rank Quantization-Aware Training for LLMs
by: Bondarenko, Yelysei, et al.
Published: (2024)

Lillama: Large Language Models Compression via Low-Rank Feature Distillation
by: Sy, Yaya, et al.
Published: (2024)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
by: Liu, Ziyue, et al.
Published: (2025)

Learning to Reason: Temporal Saliency Distillation for Interpretable Knowledge Transfer
by: Dehigahawattage, Nilushika Udayangani Hewa, et al.
Published: (2026)

Pragmatist Intelligence: Where the Principle of Usefulness Can Take ANNs
by: Bikić, Antonio, et al.
Published: (2024)

When Can LLMs Learn to Reason with Weak Supervision?
by: Rahman, Salman, et al.
Published: (2026)

Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data
by: Shirgaonkar, Anup, et al.
Published: (2024)

AMiD: Knowledge Distillation for LLMs with $α$-mixture Assistant Distribution
by: Shin, Donghyeok, et al.
Published: (2025)

Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters
by: Xiang, Haotian, et al.
Published: (2026)

3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
by: Makni, Mehdi, et al.
Published: (2026)