Cover Image

Information theoretic underpinning of self-supervised learning by clustering

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kittler, Josef, Atito, Sara, Awais, Muhammad
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Information Theory
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DailyMAE: Towards Pretraining Masked Autoencoders in One Day
by: Wu, Jiantao, et al.
Published: (2024)

Fast Rate Information-theoretic Bounds on Generalization Errors
by: Wu, Xuetong, et al.
Published: (2023)

Information-theoretic signatures of causality in Bayesian networks and hypergraphs
by: Chiang, Sung En, et al.
Published: (2025)

Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method
by: Wang, Siwei, et al.
Published: (2023)

Information-theoretic generalization bounds for learning from quantum data
by: Caro, Matthias, et al.
Published: (2023)

Rethinking Positive Pairs in Contrastive Learning
by: Wu, Jiantao, et al.
Published: (2024)

Information-theoretic Bayesian Optimization: Survey and Tutorial
by: Garrido-Merchán, Eduardo C.
Published: (2025)

Channel-Aware Probing for Multi-Channel Imaging
by: Marikkar, Umar, et al.
Published: (2026)

XAI for Self-supervised Clustering of Wireless Spectrum Activity
by: Milosheski, Ljupcho, et al.
Published: (2023)

WaterSIC: information-theoretically (near) optimal linear layer quantization
by: Lifar, Egor, et al.
Published: (2026)

An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding
by: Hu, Dou, et al.
Published: (2025)

Self-supervised Latent Space Optimization with Nebula Variational Coding
by: Wang, Yida, et al.
Published: (2025)

Some theoretical improvements on the tightness of PAC-Bayes risk certificates for neural networks
by: García-Pérez, Diego, et al.
Published: (2025)

Graph self-supervised learning based on frequency corruption
by: Li, Haojie, et al.
Published: (2026)

Information-theoretic Limits of Learning and Estimation
by: Gamal, Abbas El, et al.
Published: (2026)

C3R: Channel Conditioned Cell Representations for unified evaluation in microscopy imaging
by: Marikkar, Umar, et al.
Published: (2025)

Robust Semi-supervised Learning via $f$-Divergence and $α$-Rényi Divergence
by: Aminian, Gholamali, et al.
Published: (2024)

Investigating Self-Supervised Methods for Label-Efficient Learning
by: Nandam, Srinivasa Rao, et al.
Published: (2024)

ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification
by: Atito, Sara, et al.
Published: (2022)

Pseudo Labelling for Enhanced Masked Autoencoders
by: Nandam, Srinivasa Rao, et al.
Published: (2024)

PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation
by: Nazarieh, Fatemeh, et al.
Published: (2024)

Semi-supervised Batch Learning From Logged Data
by: Aminian, Gholamali, et al.
Published: (2022)

LZ Penalty: An information-theoretic repetition penalty for autoregressive language models
by: Ginart, Antonio A., et al.
Published: (2025)

Asymptotic Analysis of Sample-averaged Q-learning
by: Panda, Saunak Kumar, et al.
Published: (2024)

A representation-learning game for classes of prediction tasks
by: Uzan, Neria, et al.
Published: (2024)

Generalizing Supervised Contrastive learning: A Projection Perspective
by: Jeong, Minoh, et al.
Published: (2025)

Category learning in deep neural networks: Information content and geometry of internal representations
by: Bonnasse-Gahot, Laurent, et al.
Published: (2025)

Directed Information $γ$-covering: An Information-Theoretic Framework for Context Engineering
by: Huang, Hai
Published: (2025)

A unified framework for learning with nonlinear model classes from arbitrary linear samples
by: Adcock, Ben, et al.
Published: (2023)

Information-Theoretic Discrete Diffusion
by: Jeon, Moongyu, et al.
Published: (2025)

Separable Computation of Information Measures
by: Xu, Xiangxiang, et al.
Published: (2025)

Mutual Information Multinomial Estimation
by: Chen, Yanzhi, et al.
Published: (2024)

Information-theoretic reduction of deep neural networks to linear models in the overparametrized proportional regime
by: Camilli, Francesco, et al.
Published: (2025)

Information Theoretic Perspective on Representation Learning
by: Pereg, Deborah, et al.
Published: (2026)

Information-Theoretic Proofs for Diffusion Sampling
by: Reeves, Galen, et al.
Published: (2025)

An Information Theoretic Perspective on Conformal Prediction
by: Correia, Alvaro H. C., et al.
Published: (2024)

An Information-Theoretic Analysis of Temporal GNNs
by: Farzaneh, Amirmohammad
Published: (2024)

An Information-Theoretic Analysis of In-Context Learning
by: Jeon, Hong Jun, et al.
Published: (2024)

Measures of Information Reflect Memorization Patterns
by: Bansal, Rachit, et al.
Published: (2022)

Structure Learning via Mutual Information
by: Nixon, Jeremy
Published: (2024)