Cover Image

On the Asymptotics of Self-Supervised Pre-training: Two-Stage M-Estimation and Representation Symmetry

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tinati, Mohammad, Tu, Stephen
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem
by: Mohammadi, Hesameddin, et al.
Published: (2024)

DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training
by: Wu, Yuemin, et al.
Published: (2025)

The Impact of Semantic Pairs on Self-Supervised Representation Learning
by: Alkhalefi, Mohammad, et al.
Published: (2025)

SuPreME: A Supervised Pre-training Framework for Multimodal ECG Representation Learning
by: Cai, Mingsheng, et al.
Published: (2025)

A Self-Supervised Paradigm for Data-Efficient Medical Foundation Model Pre-training: V-information Optimization Framework
by: Yang, Wenxuan, et al.
Published: (2024)

Adaptive Kernel Density Estimation with Pre-training
by: Zhang, Ruitong, et al.
Published: (2026)

Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training
by: Dave, Vedant, et al.
Published: (2024)

Multi-instance Learning as Downstream Task of Self-Supervised Learning-based Pre-trained Model
by: Matsuishi, Koki, et al.
Published: (2025)

Self-Supervised Learning for Pre-training Capsule Networks: Overcoming Medical Imaging Dataset Challenges
by: El-Shimy, Heba, et al.
Published: (2025)

Improving Pre-trained Self-Supervised Embeddings Through Effective Entropy Maximization
by: Chakraborty, Deep, et al.
Published: (2024)

Self-Supervised Radio Pre-training: Toward Foundational Models for Spectrogram Learning
by: Aboulfotouh, Ahmed, et al.
Published: (2024)

Quantifying the Pre-training Dividend: Generative versus Latent Self-Supervised Learning for Time Series Foundation Models
by: Major, Noam, et al.
Published: (2026)

Self-Supervised Contrastive Pre-Training for Multivariate Point Processes
by: Shou, Xiao, et al.
Published: (2024)

Label-Efficient Sleep Staging Using Transformers Pre-trained with Position Prediction
by: Lala, Sayeri, et al.
Published: (2024)

Domain-Adaptive Pre-training of Self-Supervised Foundation Models for Medical Image Classification in Gastrointestinal Endoscopy
by: Roth, Marcel, et al.
Published: (2024)

Cross-Domain Pre-training with Language Models for Transferable Time Series Representations
by: Cheng, Mingyue, et al.
Published: (2024)

Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training
by: Guo, Quanjiang, et al.
Published: (2025)

Gaussian Joint Embeddings For Self-Supervised Representation Learning
by: Huang, Yongchao
Published: (2026)

Gaussian Joint Embeddings For Self-Supervised Representation Learning
by: Huang, Yongchao
Published: (2026)

Self-Supervised Representation Learning as Mutual Information Maximization
by: Sabby, Akhlaqur Rahman, et al.
Published: (2025)

On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning
by: Wang, Bokun, et al.
Published: (2024)

Self-Supervised Learning with Lie Symmetries for Partial Differential Equations
by: Mialon, Grégoire, et al.
Published: (2023)

Self-Supervised Pre-Training for Precipitation Post-Processor
by: An, Sojung, et al.
Published: (2023)

Contrastive-to-Self-Supervised: A Two-Stage Framework for Script Similarity Learning
by: Roman, Claire, et al.
Published: (2026)

Self-Supervised Learning for Covariance Estimation
by: Diskin, Tzvi, et al.
Published: (2024)

Self-Supervised Dynamical System Representations for Physiological Time-Series
by: Chen, Yenho, et al.
Published: (2025)

HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphs
by: Kim, Sunwoo, et al.
Published: (2024)

Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints
by: Cui, Xiaodong, et al.
Published: (2025)

Pre-training LLM without Learning Rate Decay Enhances Supervised Fine-Tuning
by: Yano, Kazuki, et al.
Published: (2026)

Pre-trained Tabular Foundation Models as Versatile Summary Networks for Neural Posterior Estimation
by: Pickens, Elliot, et al.
Published: (2026)

Pre-training Vision Transformers with Formula-driven Supervised Learning
by: Kataoka, Hirokatsu, et al.
Published: (2022)

Pre-trained Visual Dynamics Representations for Efficient Policy Learning
by: Luo, Hao, et al.
Published: (2024)

Minimax Optimal Two-Stage Algorithm For Moment Estimation Under Covariate Shift
by: Zhang, Zhen, et al.
Published: (2025)

Two Approaches to Direct Estimation of Riesz Representers
by: Bruns-Smith, David
Published: (2026)

Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetries
by: Urbano, Alonso, et al.
Published: (2023)

COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning
by: Meng, Chuizheng, et al.
Published: (2023)

AdaMR: Adaptable Molecular Representation for Unified Pre-training Strategy
by: Ding, Yan, et al.
Published: (2023)

Self-Supervised Graph Representation Learning via Global Context Prediction
by: Peng, Zhen, et al.
Published: (2020)

Learning Representation for Multitask learning through Self Supervised Auxiliary learning
by: Shin, Seokwon, et al.
Published: (2024)

Adv-SSL: Adversarial Self-Supervised Representation Learning with Theoretical Guarantees
by: Duan, Chenguang, et al.
Published: (2024)