Cover Image

When Losses Align: Gradient-Based Composite Loss Weighting for Efficient Pretraining

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Karpukhin, Ivan, Savchenko, Andrey
Format:	Preprint
Published:	2026
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Detecting the Future: All-at-Once Event Sequence Forecasting with Horizon Matching
by: Karpukhin, Ivan, et al.
Published: (2024)

HT-Transformer: Event Sequences Classification by Accumulating Prefix Information with History Tokens
by: Karpukhin, Ivan, et al.
Published: (2025)

Multimodal Banking Dataset: Understanding Client Needs through Event Sequences
by: Mollaev, Dzhambulat, et al.
Published: (2024)

Gradient Aligned Regression via Pairwise Losses
by: Zhu, Dixian, et al.
Published: (2024)

HoTPP Benchmark: Are We Good at the Long Horizon Events Forecasting?
by: Karpukhin, Ivan, et al.
Published: (2024)

Dynamic Loss-Based Sample Reweighting for Improved Large Language Model Pretraining
by: Sow, Daouda, et al.
Published: (2025)

HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting
by: Savchenko, Andrey, et al.
Published: (2025)

Sampling and Loss Weights in Multi-Domain Training
by: Salmani, Mahdi, et al.
Published: (2025)

Adaptive Weighted Loss for Sequential Recommendations on Sparse Domains
by: Mittal, Akshay, et al.
Published: (2025)

Loss- and Reward-Weighting for Efficient Distributed Reinforcement Learning
by: Holen, Martin, et al.
Published: (2023)

Simplicial SMOTE: Oversampling Solution to the Imbalanced Learning Problem
by: Kachan, Oleg, et al.
Published: (2025)

Beyond Isolated Clients: Integrating Graph-Based Embeddings into Event Sequence Models
by: Proshian, Harry, et al.
Published: (2026)

Mitigating Long-Tailed Anomaly Score Distributions with Importance-Weighted Loss
by: Lee, Jungi, et al.
Published: (2026)

Latent Factorization of Tensors with Threshold Distance Weighted Loss for Traffic Data Estimation
by: Yang, Lei
Published: (2025)

Analytical Uncertainty-Based Loss Weighting in Multi-Task Learning
by: Kirchdorfer, Lukas, et al.
Published: (2024)

AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting
by: Ganguly, Shreyan, et al.
Published: (2024)

CP Loss: Channel-wise Perceptual Loss for Time Series Forecasting
by: Zha, Yaohua, et al.
Published: (2026)

Elliptic Loss Regularization
by: Hasan, Ali, et al.
Published: (2025)

A new membership inference attack that spots memorization in generative and predictive models: Loss-Based with Reference Model algorithm (LBRM)
by: Taleb, Faiz, et al.
Published: (2025)

Diversifying Deep Ensembles: A Saliency Map Approach for Enhanced OOD Detection, Calibration, and Accuracy
by: Dereka, Stanislav, et al.
Published: (2023)

XGrad: Boosting Gradient-Based Optimizers With Weight Prediction
by: Guan, Lei, et al.
Published: (2023)

Investigating the Histogram Loss in Regression
by: Imani, Ehsan, et al.
Published: (2024)

There is a Singularity in the Loss Landscape
by: Lowell, Mark
Published: (2022)

When to Stop Reusing: Dynamic Gradient Gating for Sample-Efficient RLVR
by: Miao, Yuchun, et al.
Published: (2026)

Meta-GCN: A Dynamically Weighted Loss Minimization Method for Dealing with the Data Imbalance in Graph Neural Networks
by: Mohammadizadeh, Mahdi, et al.
Published: (2024)

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training
by: Huang, Jia-Hong, et al.
Published: (2024)

When Distance Distracts: Representation Distance Bias in BT-Loss for Reward Models
by: Xie, Tong, et al.
Published: (2025)

Loss-to-Loss Prediction: Scaling Laws for All Datasets
by: Brandfonbrener, David, et al.
Published: (2024)

Weight-Entanglement Meets Gradient-Based Neural Architecture Search
by: Sukthanker, Rhea Sanjay, et al.
Published: (2023)

Barycentric Neural Networks and Length-Weighted Persistent Entropy Loss: A Green Geometric and Topological Framework for Function Approximation
by: Toscano-Duran, Victor, et al.
Published: (2025)

Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
by: Zheng, Ruijie, et al.
Published: (2024)

Rethinking Losses for Diffusion Bridge Samplers
by: Sanokowski, Sebastian, et al.
Published: (2025)

Global Minimizers of Sigmoid Contrastive Loss
by: Bangachev, Kiril, et al.
Published: (2025)

Deep Metric Loss for Multimodal Learning
by: Moon, Sehwan, et al.
Published: (2023)

Meta-Learning Adaptive Loss Functions
by: Raymond, Christian, et al.
Published: (2023)

Neural Network Plasticity and Loss Sharpness
by: Koster, Max, et al.
Published: (2024)

LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
by: Mayilvahanan, Prasanna, et al.
Published: (2025)

Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss
by: Getu, Tilahun M., et al.
Published: (2023)

Holdout-Loss-Based Data Selection for LLM Finetuning via In-Context Learning
by: Zhang, Ling, et al.
Published: (2025)

Decoupled Split Learning via Auxiliary Loss
by: Zihad, Anower, et al.
Published: (2026)