:: Library Catalog

Imagen de Portada

Guardado en:

Detalles Bibliográficos
Autores principales:	Mahdavi, Sadegh, Liao, Renjie, Thrampoulidis, Christos
Formato:	Preprint
Publicado:	2023
Materias:	Machine Learning
Acceso en línea:	https://arxiv.org/abs/2306.02010
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradients
por: Thrampoulidis, Christos, et al.
Publicado: (2025)

Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluation
por: Mahdavi, Sadegh, et al.
Publicado: (2025)

On the Optimization and Generalization of Multi-head Attention
por: Deora, Puneesh, et al.
Publicado: (2023)

Implicit Optimization Bias of Next-Token Prediction in Linear Models
por: Thrampoulidis, Christos
Publicado: (2024)

From Graph Diffusion to Graph Classification
por: Xian, Jia Jun Cheng, et al.
Publicado: (2024)

Why Loss Re-weighting Works If You Stop Early: Training Dynamics of Unconstrained Features
por: Zhao, Yize, et al.
Publicado: (2026)

Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features
por: Behnia, Tina, et al.
Publicado: (2024)

On the Optimal Memorization Capacity of Transformers
por: Kajitsuka, Tokio, et al.
Publicado: (2024)

Thumb on the Scale: Optimal Loss Weighting in Last Layer Retraining
por: Stromberg, Nathan, et al.
Publicado: (2025)

Memory capacity of two layer neural networks with smooth activations
por: Madden, Liam, et al.
Publicado: (2023)

In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
por: Deora, Puneesh, et al.
Publicado: (2025)

Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
por: Behnia, Tina, et al.
Publicado: (2025)

Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods
por: Taheri, Hossein, et al.
Publicado: (2024)

Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data
por: Fan, Chen, et al.
Publicado: (2025)

Implicit Bias and Fast Convergence Rates for Self-attention
por: Vasudeva, Bhavya, et al.
Publicado: (2024)

Transformers as Support Vector Machines
por: Tarzanagh, Davoud Ataee, et al.
Publicado: (2023)

Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning
por: Deng, Wenlong, et al.
Publicado: (2023)

Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge
por: Vasudeva, Bhavya, et al.
Publicado: (2026)

Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
por: Garrod, Connall, et al.
Publicado: (2025)

A Capacity-Based Rationale for Multi-Head Attention
por: Adler, Micah
Publicado: (2025)

Next-token prediction capacity: general upper bounds and a lower bound for transformers
por: Madden, Liam, et al.
Publicado: (2024)

Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations
por: Zhao, Yize, et al.
Publicado: (2024)

Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models
por: Mahdavi, Sadegh, et al.
Publicado: (2024)

Scaling Generative Verifiers For Natural Language Mathematical Proof Verification And Selection
por: Mahdavi, Sadegh, et al.
Publicado: (2025)

How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data
por: Vasudeva, Bhavya, et al.
Publicado: (2025)

Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective
por: Zhang, Xuechen, et al.
Publicado: (2024)

Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers
por: Barron, Joshua, et al.
Publicado: (2025)

Improving Transformers with Dynamically Composable Multi-Head Attention
por: Xiao, Da, et al.
Publicado: (2024)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
por: Musat, Tiberiu
Publicado: (2024)

Adaptive Head Budgeting for Efficient Multi-Head Attention
por: Faye, Bilal, et al.
Publicado: (2026)

Prediction of Herd Life in Dairy Cows Using Multi-Head Attention Transformers
por: Saki, Mahdi, et al.
Publicado: (2025)

Multi-Head Attention as a Source of Catastrophic Forgetting in MoE Transformers
por: Chen, Anrui, et al.
Publicado: (2026)

The Effect of Attention Head Count on Transformer Approximation
por: Yu, Penghao, et al.
Publicado: (2025)

Multi-Head Low-Rank Attention
por: Liu, Songtao, et al.
Publicado: (2026)

Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks
por: Sohn, Jy-yong, et al.
Publicado: (2024)

On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization
por: Deng, Wenlong, et al.
Publicado: (2025)

DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models
por: Deng, Wenlong, et al.
Publicado: (2024)

MoH: Multi-Head Attention as Mixture-of-Head Attention
por: Jin, Peng, et al.
Publicado: (2024)

Directional Alignment Mitigates Reward Hacking in Reinforcement Learning for Language Models
por: Deng, Wenlong, et al.
Publicado: (2026)

LLM-Assisted Content Conditional Debiasing for Fair Text Embedding
por: Deng, Wenlong, et al.
Publicado: (2024)