Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Kun, Tan, Zhiquan, Zou, Bochao, Chen, Jiansheng, Ma, Huimin, Huang, Weiran
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unveiling the Dynamics of Information Interplay in Supervised Learning
by: Song, Kun, et al.
Published: (2024)

Information-Theoretic Perspectives on Optimizers
by: Tan, Zhiquan, et al.
Published: (2025)

Understanding Grokking Through A Robustness Viewpoint
by: Tan, Zhiquan, et al.
Published: (2023)

Matrix Information Theory for Self-Supervised Learning
by: Zhang, Yifan, et al.
Published: (2023)

The Information of Large Language Model Geometry
by: Tan, Zhiquan, et al.
Published: (2024)

Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models
by: Wei, Lai, et al.
Published: (2024)

RhythmFormer: Extracting Patterned rPPG Signals based on Periodic Sparse Attention
by: Zou, Bochao, et al.
Published: (2024)

Self-Supervised On-Policy Distillation for Reasoning Language Models
by: Tan, Zhiquan, et al.
Published: (2026)

A Theoretical Lens for RL-Tuned Language Models via Energy-Based Models
by: Tan, Zhiquan, et al.
Published: (2025)

ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy
by: Li, Hong, et al.
Published: (2024)

PAINT: Partial-Solution Adaptive Interpolated Training for Self-Distilled Reasoners
by: Tan, Zhiquan, et al.
Published: (2026)

Can I understand what I create? Self-Knowledge Evaluation of Large Language Models
by: Tan, Zhiquan, et al.
Published: (2024)

Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse
by: He, Kun, et al.
Published: (2025)

Provable Contrastive Continual Learning
by: Wen, Yichen, et al.
Published: (2024)

Self-Supervised Learning for Neural Topic Models with Variance-Invariance-Covariance Regularization
by: Xu, Weiran, et al.
Published: (2025)

From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Models
by: Li, Xinyang, et al.
Published: (2025)

OTMatch: Improving Semi-Supervised Learning with Optimal Transport
by: Tan, Zhiquan, et al.
Published: (2023)

A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning
by: Lee, Chungpa, et al.
Published: (2025)

A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops
by: Fu, Shi, et al.
Published: (2025)

On the Robustness of Neural Collapse and the Neural Collapse of Robustness
by: Su, Jingtong, et al.
Published: (2023)

General Information Metrics for Improving AI Model Training Efficiency
by: Xu, Jianfeng, et al.
Published: (2025)

An Information Theoretic Evaluation Metric For Strong Unlearning
by: Jeon, Dongjae, et al.
Published: (2024)

Monitoring Neural Training with Topology: A Footprint-Predictable Collapse Index
by: Kalinowski, Alexander
Published: (2026)

DRIK: Distribution-Robust Inductive Kriging without Information Leakage
by: Yang, Chen, et al.
Published: (2025)

Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning
by: Xiao, Jia-Hao, et al.
Published: (2024)

Information Flow in Self-Supervised Learning
by: Tan, Zhiquan, et al.
Published: (2023)

How Label Imbalance Shapes Geometry: A General Spectral Analysis of Multi-Label Neural Collapse
by: Ma, Xiaoxuan, et al.
Published: (2026)

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
by: Qiu, Shikai, et al.
Published: (2025)

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse
by: Jacot, Arthur, et al.
Published: (2024)

Why Prototypes Collapse: Diagnosing and Preventing Partial Collapse in Prototypical Self-Supervised Learning
by: Arteaga, Gabriel Y., et al.
Published: (2025)

Provable Training Data Identification for Large Language Models
by: Liu, Zhenlong, et al.
Published: (2025)

Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning
by: Luthra, Achleshwar, et al.
Published: (2026)

Explaining Grokking and Information Bottleneck through Neural Collapse Emergence
by: Sakamoto, Keitaro, et al.
Published: (2025)

Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization
by: He, Junlin, et al.
Published: (2024)

Learning Expressive Random Feature Models via Parametrized Activations
by: Ma, Zailin, et al.
Published: (2024)

On the Generalization Properties of Learning the Random Feature Models with Learnable Activation Functions
by: Ma, Zailin, et al.
Published: (2025)

Graph Coarsening via Supervised Granular-Ball for Scalable Graph Neural Network Training
by: Xia, Shuyin, et al.
Published: (2024)

Preventing Model Collapse via Contraction-Conditioned Neural Filters
by: Han, Zongjian, et al.
Published: (2025)

Inference-Cost-Aware Dynamic Tree Construction for Efficient Inference in Large Language Models
by: Hong, Yinrong, et al.
Published: (2025)

Collapse-Proof Non-Contrastive Self-Supervised Learning
by: Sansone, Emanuele, et al.
Published: (2024)