Imagen de Portada

These Are Not All the Features You Are Looking For: A Fundamental Bottleneck in Supervised Pretraining

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yang, Xingyu Alice, Zhang, Jianyu, Bottou, Léon
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Memory Mosaics
por: Zhang, Jianyu, et al.
Publicado: (2024)

Memory Mosaics at scale
por: Zhang, Jianyu, et al.
Publicado: (2025)

Fine-tuning with Very Large Dropout
por: Zhang, Jianyu, et al.
Publicado: (2024)

XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
por: Myara, Alexandre, et al.
Publicado: (2026)

Cross-Entropy Is All You Need To Invert the Data Generating Process
por: Reizinger, Patrik, et al.
Publicado: (2024)

Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
por: Liu, Yuxing, et al.
Publicado: (2026)

Simulation as Supervision: Mechanistic Pretraining for Scientific Discovery
por: Dudley, Carson, et al.
Publicado: (2025)

Context is All You Need
por: Delanois, Jean Erik, et al.
Publicado: (2026)

Attention is All You Need Until You Need Retention
por: Yaslioglu, M. Murat
Publicado: (2025)

Exploitation Is All You Need... for Exploration
por: Rentschler, Micah, et al.
Publicado: (2025)

Vanishing Feature: Diagnosing Model Merging and Beyond
por: Qu, Xingyu, et al.
Publicado: (2024)

Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets
por: Jacot, Arthur, et al.
Publicado: (2024)

Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoff
por: Jacot, Arthur
Publicado: (2023)

Glocal Information Bottleneck for Time Series Imputation
por: Yang, Jie, et al.
Publicado: (2025)

All You Need Is Synthetic Task Augmentation
por: Godin, Guillaume
Publicado: (2025)

Element-wise Attention Is All You Need
por: Feng, Guoxin
Publicado: (2025)

Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning
por: Sun, Ao, et al.
Publicado: (2024)

Tensor Product Attention Is All You Need
por: Zhang, Yifan, et al.
Publicado: (2025)

AI for the Open-World: the Learning Principles
por: Zhang, Jianyu
Publicado: (2025)

Incremental Residual Concept Bottleneck Models
por: Shang, Chenming, et al.
Publicado: (2024)

Supervised Pretraining for Material Property Prediction
por: Rahman, Chowdhury Mohammad Abid, et al.
Publicado: (2025)

Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning
por: Wen, Yuxiao, et al.
Publicado: (2024)

The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference
por: Qasim, Kaleem Ullah, et al.
Publicado: (2026)

TransMLA: Multi-Head Latent Attention Is All You Need
por: Meng, Fanxu, et al.
Publicado: (2025)

More Agents Is All You Need
por: Li, Junyou, et al.
Publicado: (2024)

Transduction is All You Need for Structured Data Workflows
por: Gliozzo, Alfio, et al.
Publicado: (2025)

AllMatch: Exploiting All Unlabeled Data for Semi-Supervised Learning
por: Wu, Zhiyu, et al.
Publicado: (2024)

Fair Streaming Feature Selection
por: Duan, Zhangling, et al.
Publicado: (2024)

A Probabilistic Model Behind Self-Supervised Learning
por: Bizeul, Alice, et al.
Publicado: (2024)

Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining
por: Ariguib, Boshra, et al.
Publicado: (2025)

Capabilities Ain't All You Need: Measuring Propensities in AI
por: Romero-Alvarado, Daniel, et al.
Publicado: (2026)

HDL-GPT: High-Quality HDL is All You Need
por: Kumar, Bhuvnesh, et al.
Publicado: (2024)

A Single Character can Make or Break Your LLM Evals
por: Su, Jingtong, et al.
Publicado: (2025)

From Basic to Extra Features: Hypergraph Transformer Pretrain-then-Finetuning for Balanced Clinical Predictions on EHR
por: Xu, Ran, et al.
Publicado: (2024)

Adaptive Spatio-Temporal Graphs with Self-Supervised Pretraining for Multi-Horizon Weather Forecasting
por: Liu, Yao
Publicado: (2025)

All Random Features Representations are Equivalent
por: Sernau, Luke, et al.
Publicado: (2024)

SL-CBM: Enhancing Concept Bottleneck Models with Semantic Locality for Better Interpretability
por: Zhang, Hanwei, et al.
Publicado: (2026)

DTization: A New Method for Supervised Feature Scaling
por: Islam, Niful
Publicado: (2024)

Context-Selective State Space Models: Feedback is All You Need
por: Zattra, Riccardo, et al.
Publicado: (2025)

Efficient Deep Learning Board: Training Feedback Is Not All You Need
por: Gong, Lina, et al.
Publicado: (2024)