Cover Image

Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Scardecchia, Mattia
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Machine Learning Image and Video Processing
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Image Classification Using a Diffusion Model as a Pre-Training Model
by: Ukita, Kosuke, et al.
Published: (2025)

A Lightweight Image Super-Resolution Transformer Trained on Low-Resolution Images Only
by: Möller, Björn, et al.
Published: (2025)

Cross Domain Early Crop Mapping using CropSTGAN
by: Wang, Yiqun, et al.
Published: (2024)

Equivariant Imaging Biomarkers for Robust Unsupervised Segmentation of Histopathology
by: Chen, Fuyao, et al.
Published: (2025)

Unsupervised Imaging Inverse Problems with Diffusion Distribution Matching
by: Meanti, Giacomo, et al.
Published: (2025)

Towards Universal Unsupervised Anomaly Detection in Medical Imaging
by: Bercea, Cosmin I., et al.
Published: (2024)

Preserving Tumor Volumes for Unsupervised Medical Image Registration
by: Dong, Qihua, et al.
Published: (2023)

SSVT: Self-Supervised Vision Transformer For Eye Disease Diagnosis Based On Fundus Images
by: Wang, Jiaqi, et al.
Published: (2024)

Patch-Based Deep Unsupervised Image Segmentation using Graph Cuts
by: Wasserman, Isaac, et al.
Published: (2023)

Temporal As a Plugin: Unsupervised Video Denoising with Pre-Trained Image Denoisers
by: Fu, Zixuan, et al.
Published: (2024)

Intelligent Anomaly Detection for Lane Rendering Using Transformer with Self-Supervised Pre-Training and Customized Fine-Tuning
by: Dong, Yongqi, et al.
Published: (2023)

UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights
by: Liang, Shijun, et al.
Published: (2025)

ViTGAN: Training GANs with Vision Transformers
by: Lee, Kwonjoon, et al.
Published: (2021)

The Missing Piece: A Case for Pre-Training in 3D Medical Object Detection
by: Eckstein, Katharina, et al.
Published: (2025)

Deep Self-knowledge Distillation: A hierarchical supervised learning for coronary artery segmentation
by: Lin, Mingfeng
Published: (2025)

A Comparative Study of Transformer and Convolutional Models for Crop Segmentation from Satellite Image Time Series
by: Gatti, Mattia, et al.
Published: (2024)

Deep Representation Learning for Unsupervised Clustering of Myocardial Fiber Trajectories in Cardiac Diffusion Tensor Imaging
by: Anand, Mohini, et al.
Published: (2025)

UniCompress: Enhancing Multi-Data Medical Image Compression with Knowledge Distillation
by: Yang, Runzhao, et al.
Published: (2024)

Randomized Principal Component Analysis for Hyperspectral Image Classification
by: Ustuner, Mustafa
Published: (2024)

Equivariant Imaging for Self-supervised Hyperspectral Image Inpainting
by: Li, Shuo, et al.
Published: (2024)

MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder
by: Li, Lei, et al.
Published: (2024)

Key-Graph Transformer for Image Restoration
by: Ren, Bin, et al.
Published: (2024)

Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images
by: Zhang, Yuan, et al.
Published: (2023)

A Contrastive Pre-trained Foundation Model for Deciphering Imaging Noisomics across Modalities
by: Gu, Yuanjie, et al.
Published: (2026)

Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training
by: Chen, Jiamin, et al.
Published: (2024)

GCS-M3VLT: Guided Context Self-Attention based Multi-modal Medical Vision Language Transformer for Retinal Image Captioning
by: Cherukuri, Teja Krishna, et al.
Published: (2024)

SDIP: Self-Reinforcement Deep Image Prior Framework for Image Processing
by: Shu, Ziyu, et al.
Published: (2024)

Unsupervised Learning of the Total Variation Flow
by: Grossmann, Tamara G., et al.
Published: (2022)

F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
by: Ho, Man M., et al.
Published: (2024)

FARM: Fine-Tuning Geospatial Foundation Models for Intra-Field Crop Yield Regression
by: Nejadshamsi, Shayan, et al.
Published: (2025)

Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization
by: Doerrich, Sebastian, et al.
Published: (2024)

Dynamic Perturbation-Adaptive Adversarial Training on Medical Image Classification
by: Li, Shuai, et al.
Published: (2024)

Pre-processing and Compression: Understanding Hidden Representation Refinement Across Imaging Domains via Intrinsic Dimension
by: Konz, Nicholas, et al.
Published: (2024)

FQGA-single: Towards Fewer Training Epochs and Fewer Model Parameters for Image-to-Image Translation Tasks
by: Yang, Cho
Published: (2024)

EffiSegNet: Gastrointestinal Polyp Segmentation through a Pre-Trained EfficientNet-based Network with a Simplified Decoder
by: Vezakis, Ioannis A., et al.
Published: (2024)

Early Stopping Criteria for Training Generative Adversarial Networks in Biomedical Imaging
by: Saad, Muhammad Muneeb, et al.
Published: (2024)

Unsupervised Domain Transfer with Conditional Invertible Neural Networks
by: Dreher, Kris K., et al.
Published: (2023)

Unsupervised Feature Orthogonalization for Learning Distortion-Invariant Representations
by: Doerrich, Sebastian, et al.
Published: (2024)

Joint Denoising of Cryo-EM Projection Images using Polar Transformers
by: Andén, Joakim, et al.
Published: (2025)

CECT: Controllable Ensemble CNN and Transformer for COVID-19 Image Classification
by: Liu, Zhaoshan, et al.
Published: (2023)