Imagen de Portada

An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Shwartz-Ziv, Ravid, Balestriero, Randall, Kawaguchi, Kenji, Rudner, Tim G. J., LeCun, Yann
Formato:	Preprint
Publicado:	2023
Materias:	Information Theory Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Variance-Covariance Regularization Improves Representation Learning
por: Zhu, Jiachen, et al.
Publicado: (2023)

From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
por: Shani, Chen, et al.
Publicado: (2025)

Video Representation Learning with Joint-Embedding Predictive Architectures
por: Drozdov, Katrina, et al.
Publicado: (2024)

Fast and Exact Enumeration of Deep Networks Partitions Regions
por: Balestriero, Randall, et al.
Publicado: (2024)

AI Must Embrace Specialization via Superhuman Adaptable Intelligence
por: Goldfeder, Judah, et al.
Publicado: (2026)

LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics
por: Balestriero, Randall, et al.
Publicado: (2025)

Learning by Reconstruction Produces Uninformative Features For Perception
por: Balestriero, Randall, et al.
Publicado: (2024)

Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations
por: Mialon, Grégoire, et al.
Publicado: (2022)

LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures
por: Huang, Hai, et al.
Publicado: (2025)

Learning to Compress: Local Rank and Information Compression in Deep Neural Networks
por: Patel, Niket, et al.
Publicado: (2024)

Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density
por: Balestriero, Randall, et al.
Publicado: (2025)

Causal-JEPA: Learning World Models through Object-Level Latent Masking
por: Nam, Heejeong, et al.
Publicado: (2026)

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
por: Maes, Lucas, et al.
Publicado: (2026)

Beyond the Loss Curve: Scaling Laws, Active Learning, and the Limits of Learning from Exact Posteriors
por: Khorasani, Arian, et al.
Publicado: (2026)

Improving Pre-trained Self-Supervised Embeddings Through Effective Entropy Maximization
por: Chakraborty, Deep, et al.
Publicado: (2024)

JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention
por: Ioannides, Georgios, et al.
Publicado: (2025)

Layer by Layer: Uncovering Hidden Representations in Language Models
por: Skean, Oscar, et al.
Publicado: (2025)

Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
por: Queipo-de-Llano, Enrique, et al.
Publicado: (2025)

The Entropy Enigma: Success and Failure of Entropy Minimization
por: Press, Ori, et al.
Publicado: (2024)

Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
por: Ioannides, Georgios, et al.
Publicado: (2026)

stable-worldmodel-v1: Reproducible World Modeling Research and Evaluation
por: Maes, Lucas, et al.
Publicado: (2026)

Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
por: Kuang, Yilun, et al.
Publicado: (2026)

Does Representation Matter? Exploring Intermediate Layers in Large Language Models
por: Skean, Oscar, et al.
Publicado: (2024)

Semantic Tube Prediction: Beating LLM Data Efficiency with JEPA
por: Huang, Hai, et al.
Publicado: (2026)

Machine Unlearning via Information Theoretic Regularization
por: Xu, Shizhou, et al.
Publicado: (2025)

On Training in Imagination
por: Timor, Nadav, et al.
Publicado: (2026)

Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive science
por: Dupoux, Emmanuel, et al.
Publicado: (2026)

Fairness Overfitting in Machine Learning: An Information-Theoretic Perspective
por: Laakom, Firas, et al.
Publicado: (2025)

Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
por: Sobal, Vlad, et al.
Publicado: (2025)

Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation
por: Zeevi, Tal, et al.
Publicado: (2024)

Uncertainty Quantification and Data Efficiency in AI: An Information-Theoretic Perspective
por: Simeone, Osvaldo, et al.
Publicado: (2025)

An Information Theoretic Perspective on Agentic System Design
por: He, Shizhe, et al.
Publicado: (2025)

Intensional Inheritance Between Concepts: An Information-Theoretic Interpretation
por: Goertzel, Ben
Publicado: (2025)

Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
por: Kuang, Yilun, et al.
Publicado: (2026)

Neural Polar Decoders for Deletion Channels
por: Aharoni, Ziv, et al.
Publicado: (2025)

DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning
por: Zhou, Gaoyue, et al.
Publicado: (2024)

VISReg: Variance-Invariance-Sketching Regularization for JEPA training
por: Wu, Haiyu, et al.
Publicado: (2026)

Generalization and Informativeness of Weighted Conformal Risk Control Under Covariate Shift
por: Zecchin, Matteo, et al.
Publicado: (2025)

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models
por: Sanyal, Sunny, et al.
Publicado: (2024)

Temporal Straightening for Latent Planning
por: Wang, Ying, et al.
Publicado: (2026)