Cover Image

Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xing, Yue, Lin, Xiaofeng, Song, Qifan, Xu, Yi, Zeng, Belinda, Cheng, Guang
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Data
by: Xing, Yue, et al.
Published: (2024)

Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility
by: Haldar, Rajdeep, et al.
Published: (2024)

Task-tailored Pre-processing: Fair Downstream Supervised Learning
by: Sohn, Jinwon, et al.
Published: (2026)

Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability
by: Haldar, Rajdeep, et al.
Published: (2024)

Lower Difficulty and Better Robustness: A Bregman Divergence Perspective for Adversarial Training
by: Wu, Zihui, et al.
Published: (2022)

f-GRPO and Beyond: Divergence-Based Reinforcement Learning Algorithms for General LLM Alignment
by: Haldar, Rajdeep, et al.
Published: (2026)

How to Enhance Downstream Adversarial Robustness (almost) without Touching the Pre-Trained Foundation Model?
by: Liu, Meiqi, et al.
Published: (2025)

On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics
by: Li, Binghui, et al.
Published: (2023)

Fair Supervised Learning with A Simple Random Sampler of Sensitive Attributes
by: Sohn, Jinwon, et al.
Published: (2023)

Revisiting the Relationship between Adversarial and Clean Training: Why Clean Training Can Make Adversarial Training Better
by: Zhou, MingWei, et al.
Published: (2025)

LLM Safety Alignment is Divergence Estimation in Disguise
by: Haldar, Rajdeep, et al.
Published: (2025)

Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoders
by: Lei, Zhuxin, et al.
Published: (2026)

Parallelly Tempered Generative Adversarial Nets: Toward Stabilized Gradients
by: Sohn, Jinwon, et al.
Published: (2024)

Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models
by: Javanmard, Adel, et al.
Published: (2026)

Pre-Trained AI Model Assisted Online Decision-Making under Missing Covariates: A Theoretical Perspective
by: Hu, Haichen, et al.
Published: (2025)

Improving Clean Accuracy via a Tangent-Space Perspective on Adversarial Training
by: Yi, Bongsoo, et al.
Published: (2024)

HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training
by: Xu, Yuanjian, et al.
Published: (2025)

When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
by: Yan, Jun, et al.
Published: (2026)

Stability and Generalization in Free Adversarial Training
by: Cheng, Xiwei, et al.
Published: (2024)

Latent Adversarial Training Improves the Representation of Refusal
by: Abbas, Alexandra, et al.
Published: (2025)

CTSyn: A Foundation Model for Cross Tabular Data Generation
by: Lin, Xiaofeng, et al.
Published: (2024)

Cross-Receiver Generalization for RF Fingerprint Identification via Feature Disentanglement and Adversarial Training
by: Pan, Yuhao, et al.
Published: (2025)

Information-Theoretic Policy Pre-Training with Empowerment
by: Schneider, Moritz, et al.
Published: (2025)

Better Training Data Attribution via Better Inverse Hessian-Vector Products
by: Wang, Andrew, et al.
Published: (2025)

OptiMer: Optimal Distribution Vector Merging Is Better than Data Mixing for Continual Pre-Training
by: Song, Haiyue, et al.
Published: (2026)

Information Theoretic Adversarial Training of Large Language Models
by: Zhang, Yiwei, et al.
Published: (2026)

Rethinking Pre-Training in Tabular Data: A Neighborhood Embedding Perspective
by: Ye, Han-Jia, et al.
Published: (2023)

Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awareness
by: Zhang, Guibin, et al.
Published: (2024)

HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters
by: Mo, Yujie, et al.
Published: (2024)

Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective
by: Chen, Yang, et al.
Published: (2024)

Towards a Better Theoretical Understanding of Independent Subnetwork Training
by: Shulgin, Egor, et al.
Published: (2023)

FairRR: Pre-Processing for Group Fairness through Randomized Response
by: Zeng, Xianli, et al.
Published: (2024)

Adversarial Training from Mean Field Perspective
by: Kumano, Soichiro, et al.
Published: (2025)

Scaling Adversarial Training via Data Selection
by: Ye, Youran, et al.
Published: (2025)

SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
by: Wang, Ziyi, et al.
Published: (2025)

Representation Learning of Multivariate Time Series using Attention and Adversarial Training
by: Scharwächter, Leon, et al.
Published: (2024)

Adjustment for Confounding using Pre-Trained Representations
by: Schulte, Rickmer, et al.
Published: (2025)

An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learning
by: Yoon, Jaesik, et al.
Published: (2023)

Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization
by: Liu, Ganlin, et al.
Published: (2024)

Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets
by: Lu, Haoye, et al.
Published: (2025)