Image de couverture de livre

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Lee, Hyunwoo, Choi, Hayoung, Kim, Hyunju
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
par: Lee, Hyunwoo, et autres
Publié: (2024)

Progressive Weight Loading: Accelerating Initial Inference and Gradually Boosting Performance on Resource-Constrained Environments
par: Kim, Hyunwoo, et autres
Publié: (2025)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
par: Lee, Hyungu, et autres
Publié: (2025)

PINNet: a deep neural network with pathway prior knowledge for Alzheimer's disease
par: Kim, Yeojin, et autres
Publié: (2022)

Deep Metric Loss for Multimodal Learning
par: Moon, Sehwan, et autres
Publié: (2023)

Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks
par: Bencomo, Gianluca, et autres
Publié: (2025)

Fast Training of Sinusoidal Neural Fields via Scaling Initialization
par: Yeom, Taesun, et autres
Publié: (2024)

Frictional Q-Learning
par: Kim, Hyunwoo, et autres
Publié: (2025)

Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
par: Jang, Sooyoung, et autres
Publié: (2021)

Spatio-Temporal Graphs Beyond Grids: Benchmark for Maritime Anomaly Detection
par: Kim, Jeehong, et autres
Publié: (2025)

FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer
par: Hwang, Dongyeong, et autres
Publié: (2024)

Principal Components for Neural Network Initialization
par: Phan, Nhan, et autres
Publié: (2025)

LION-DG: Layer-Informed Initialization with Deep Gradient Protocols for Accelerated Neural Network Training
par: Kim, Hyunjun
Publié: (2026)

Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function
par: Abohwo, Jason, et autres
Publié: (2025)

Revisiting Weight Averaging for Model Merging
par: Choi, Jiho, et autres
Publié: (2024)

Global Minimizers of Sigmoid Contrastive Loss
par: Bangachev, Kiril, et autres
Publié: (2025)

Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
par: Kim, Hyunjun
Publié: (2026)

Achieving the Tightest Relaxation of Sigmoids for Formal Verification
par: Chevalier, Samuel, et autres
Publié: (2024)

Revisiting Glorot Initialization for Long-Range Linear Recurrences
par: Bar, Noga, et autres
Publié: (2025)

When Bias Meets Trainability: Connecting Theories of Initialization
par: Bassi, Alberto, et autres
Publié: (2025)

The Initialization Determines Whether In-Context Learning Is Gradient Descent
par: Xie, Shifeng, et autres
Publié: (2025)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
par: Abuduweili, Abulikemu, et autres
Publié: (2024)

Initializing Services in Interactive ML Systems for Diverse Users
par: Bose, Avinandan, et autres
Publié: (2023)

NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces
par: Kim, Jiwoo, et autres
Publié: (2026)

World Models for Autonomous Driving: An Initial Survey
par: Guan, Yanchen, et autres
Publié: (2024)

Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness
par: Joo, Hanseon, et autres
Publié: (2025)

UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models
par: Kang, Hyunju, et autres
Publié: (2026)

Random Initialization of Gated Sparse Adapters
par: Retault, Vi, et autres
Publié: (2025)

NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
par: Li, Zeming, et autres
Publié: (2025)

IDInit: A Universal and Stable Initialization Method for Neural Network Training
par: Pan, Yu, et autres
Publié: (2025)

Randomly Initialized Networks Can Learn from Peer-to-Peer Consensus
par: Rodríguez-Betancourt, Esteban, et autres
Publié: (2026)

Bit-Identical Medical Deep Learning via Structured Orthogonal Initialization
par: Shkolnikov, Yakov Pyotr
Publié: (2026)

Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization
par: Eccles, Bailey J., et autres
Publié: (2024)

No More Adam: Learning Rate Scaling at Initialization is All You Need
par: Xu, Minghao, et autres
Publié: (2024)

Inversion-based Latent Bayesian Optimization
par: Chu, Jaewon, et autres
Publié: (2024)

Mimetic Initialization of MLPs
par: Trockman, Asher, et autres
Publié: (2026)

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA
par: Kang, Hyunju, et autres
Publié: (2026)

Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models
par: Xia, Shi-Yu, et autres
Publié: (2024)

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
par: Choi, Moonseok, et autres
Publié: (2023)

The Impact of Initialization on LoRA Finetuning Dynamics
par: Hayou, Soufiane, et autres
Publié: (2024)