Cover Image

Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Ibrahim, Mark, Klindt, David, Balestriero, Randall
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research
by: Reizinger, Patrik, et al.
Published: (2025)

LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
by: Balestriero, Randall, et al.
Published: (2025)

Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learning
by: Van Assel, Hugues, et al.
Published: (2025)

In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
by: Deora, Puneesh, et al.
Published: (2025)

No Location Left Behind: Measuring and Improving the Fairness of Implicit Representations for Earth Data
by: Cai, Daniel, et al.
Published: (2025)

ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws
by: Huang, Hai, et al.
Published: (2024)

GPS-SSL: Guided Positive Sampling to Inject Prior Into Self-Supervised Learning
by: Feizi, Aarash, et al.
Published: (2024)

Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum
by: Lu, Wenquan, et al.
Published: (2025)

Self-Supervised Anomaly Detection in the Wild: Favor Joint Embeddings Methods
by: Otero, Daniel, et al.
Published: (2024)

Cross-Entropy Is All You Need To Invert the Data Generating Process
by: Reizinger, Patrik, et al.
Published: (2024)

Data Whitening Improves Sparse Autoencoder Learning
by: Saraswatula, Ashwin, et al.
Published: (2025)

Task Priors: Enhancing Model Evaluation by Considering the Entire Space of Downstream Tasks
by: Patel, Niket, et al.
Published: (2025)

SAFE: A Novel Approach to AI Weather Evaluation through Stratified Assessments of Forecasts over Earth
by: Masi, Nick, et al.
Published: (2025)

Learning by Reconstruction Produces Uninformative Features For Perception
by: Balestriero, Randall, et al.
Published: (2024)

Fast and Exact Enumeration of Deep Networks Partitions Regions
by: Balestriero, Randall, et al.
Published: (2024)

A Geometric Modeling of Occam's Razor in Deep Learning
by: Sun, Ke, et al.
Published: (2019)

On the Geometry of Deep Learning
by: Balestriero, Randall, et al.
Published: (2024)

DIET-CP: Lightweight and Data Efficient Self Supervised Continued Pretraining
by: Rodas, Bryan, et al.
Published: (2025)

Occam's Razor is Only as Sharp as Your ELBO
by: Harvey, Ethan, et al.
Published: (2026)

The Fair Language Model Paradox
by: Pinto, Andrea, et al.
Published: (2024)

Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation
by: Balestriero, Randall, et al.
Published: (2023)

In-context learning and Occam's razor
by: Elmoznino, Eric, et al.
Published: (2024)

Understanding Representation Learnability of Nonlinear Self-Supervised Learning
by: Yang, Ruofeng, et al.
Published: (2024)

Data-Driven Self-Supervised Graph Representation Learning
by: Samy, Ahmed E., et al.
Published: (2024)

The Impact of Semantic Pairs on Self-Supervised Representation Learning
by: Alkhalefi, Mohammad, et al.
Published: (2025)

Quantifying Representation Reliability in Self-Supervised Learning Models
by: Park, Young-Jin, et al.
Published: (2023)

Seeking the Sufficiency and Necessity Causal Features in Multimodal Representation Learning
by: Chen, Boyu, et al.
Published: (2024)

Self-Supervised Representation Learning for Geospatial Objects: A Survey
by: Chen, Yile, et al.
Published: (2024)

Self-Supervised Learning of Disentangled Representations for Multivariate Time-Series
by: Chang, Ching, et al.
Published: (2024)

Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density
by: Balestriero, Randall, et al.
Published: (2025)

Why and How Auxiliary Tasks Improve JEPA Representations
by: Yu, Jiacan, et al.
Published: (2025)

From Alignment to Prediction: A Study of Self-Supervised Learning and Predictive Representation Learning
by: Dutta, Mintu, et al.
Published: (2026)

Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations
by: Mialon, Grégoire, et al.
Published: (2022)

DeepCell: Self-Supervised Multiview Fusion for Circuit Representation Learning
by: Shi, Zhengyuan, et al.
Published: (2025)

Subgraph Gaussian Embedding Contrast for Self-Supervised Graph Representation Learning
by: Xie, Shifeng, et al.
Published: (2025)

Diffusion-Assisted Distillation for Self-Supervised Graph Representation Learning with MLPs
by: Ahn, Seong Jin, et al.
Published: (2025)

UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
by: Al-Tahan, Haider, et al.
Published: (2024)

Domain Knowledge is Power: Leveraging Physiological Priors for Self Supervised Representation Learning in Electrocardiography
by: Maghsoodi, Nooshin, et al.
Published: (2025)

RL-BioAug: Label-Efficient Reinforcement Learning for Self-Supervised EEG Representation Learning
by: Lee, Cheol-Hui, et al.
Published: (2026)

Action-Sufficient Goal Representations
by: Hyeon, Jinu, et al.
Published: (2026)