Cover Image

Towards Optimal Trade-offs in Knowledge Distillation for CNNs and Vision Transformers at the Edge

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Violos, John, Papadopoulos, Symeon, Kompatsiaris, Ioannis
Format:	Preprint
Published:	2024
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reducing Inference Energy Consumption Using Dual Complementary CNNs
by: Kinnas, Michail, et al.
Published: (2024)

Sum of Group Error Differences: A Critical Examination of Bias Evaluation in Biometric Verification and a Dual-Metric Measure
by: Elobaid, Alaa, et al.
Published: (2024)

InDistill: Information flow-preserving knowledge distillation for model compression
by: Sarridis, Ioannis, et al.
Published: (2022)

A Brief Review for Compression and Transfer Learning Techniques in DeepFake Detection
by: Karathanasis, Andreas, et al.
Published: (2025)

IncSAR: A Dual Fusion Incremental Learning Framework for SAR Target Recognition
by: Karantaidis, George, et al.
Published: (2024)

Few-Shot Class-Incremental Learning For Efficient SAR Automatic Target Recognition
by: Karantaidis, George, et al.
Published: (2025)

Frugal Machine Learning for Energy-efficient, and Resource-aware Artificial Intelligence
by: Violos, John, et al.
Published: (2025)

Designing and Generating Diverse, Equitable Face Image Datasets for Face Verification Tasks
by: Baltsou, Georgia, et al.
Published: (2025)

SIDBench: A Python Framework for Reliably Assessing Synthetic Image Detection Methods
by: Schinas, Manos, et al.
Published: (2024)

Do DeepFake Attribution Models Generalize?
by: Baxavanakis, Spiros, et al.
Published: (2025)

Towards Quantitative Evaluation of Explainable AI Methods for Deepfake Detection
by: Tsigos, Konstantinos, et al.
Published: (2024)

Any-Resolution AI-Generated Image Detection by Spectral Learning
by: Karageorgiou, Dimitrios, et al.
Published: (2024)

Automated Image Captioning with CNNs and Transformers
by: Cahyono, Joshua Adrian, et al.
Published: (2024)

Unsupervised Training of Vision Transformers with Synthetic Negatives
by: Giakoumoglou, Nikolaos, et al.
Published: (2025)

StegOT: Trade-offs in Steganography via Optimal Transport
by: Lin, Chengde, et al.
Published: (2025)

Fake & Square: Training Self-Supervised Vision Transformers with Synthetic Data and Synthetic Hard Negatives
by: Giakoumoglou, Nikolaos, et al.
Published: (2025)

Unifying Biomedical Vision-Language Expertise: Towards a Generalist Foundation Model via Multi-CLIP Knowledge Distillation
by: Wang, Shansong, et al.
Published: (2025)

Adaptive Knowledge Distillation for Classification of Hand Images using Explainable Vision Transformers
by: Nguyen, Thanh Thi, et al.
Published: (2024)

ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Models
by: Wei, Guoyizhe, et al.
Published: (2025)

Dynamic Weight Adjustment for Knowledge Distillation: Leveraging Vision Transformer for High-Accuracy Lung Cancer Detection and Real-Time Deployment
by: Khan, Saif Ur Rehman, et al.
Published: (2025)

Efficient Partitioning Vision Transformer on Edge Devices for Distributed Inference
by: Liu, Xiang, et al.
Published: (2024)

Multi-Head Explainer: A General Framework to Improve Explainability in CNNs and Transformers
by: Sun, Bohang, et al.
Published: (2025)

Lightweight Remote Sensing Scene Classification on Edge Devices via Knowledge Distillation and Early-exit
by: Zhao, Yang, et al.
Published: (2025)

SiNGER: A Clearer Voice Distills Vision Transformers Further
by: Yu, Geunhyeok, et al.
Published: (2025)

VORTEX: Challenging CNNs at Texture Recognition by using Vision Transformers with Orderless and Randomized Token Encodings
by: Scabini, Leonardo, et al.
Published: (2025)

ButterflyViT: 354$\times$ Expert Compression for Edge Vision Transformers
by: Karmore, Aryan
Published: (2026)

When CNNs Outperform Transformers and Mambas: Revisiting Deep Architectures for Dental Caries Segmentation
by: Ghimire, Aashish, et al.
Published: (2025)

Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model
by: Feng, Qianhan, et al.
Published: (2024)

LLaVA-CKD: Bottom-Up Cascaded Knowledge Distillation for Vision-Language Models
by: Gkalelis, Nikolaos, et al.
Published: (2026)

TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge
by: Zhang, Shu-Hao, et al.
Published: (2025)

Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models
by: Xu, Zane, et al.
Published: (2025)

Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentation
by: Li, Muquan, et al.
Published: (2024)

NuWa: Deriving Lightweight Task-Specific Vision Transformers for Edge Devices
by: Wei, Ziteng, et al.
Published: (2025)

Edge-Enhanced Vision Transformer Framework for Accurate AI-Generated Image Detection
by: Das, Dabbrata, et al.
Published: (2025)

Does Bigger Mean Better? Comparitive Analysis of CNNs and Biomedical Vision Language Modles in Medical Diagnosis
by: Tong, Ran, et al.
Published: (2025)

Latent Distillation for Continual Object Detection at the Edge
by: Pasti, Francesco, et al.
Published: (2024)

Adaptive Explicit Knowledge Transfer for Knowledge Distillation
by: Park, Hyungkeun, et al.
Published: (2024)

Detecção da Psoríase Utilizando Visão Computacional: Uma Abordagem Comparativa Entre CNNs e Vision Transformers
by: Lucena, Natanael, et al.
Published: (2025)

Transforming Precision: A Comparative Analysis of Vision Transformers, CNNs, and Traditional ML for Knee Osteoarthritis Severity Diagnosis
by: Apon, Tasnim Sakib, et al.
Published: (2024)

Understanding Trade offs When Conditioning Synthetic Data
by: Trabucco, Brandon, et al.
Published: (2025)