Imagen de Portada

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Lee, Hyunwoo, Choi, Hayoung, Kim, Hyunju
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
por: Lee, Hyunwoo, et al.
Publicado: (2024)

Progressive Weight Loading: Accelerating Initial Inference and Gradually Boosting Performance on Resource-Constrained Environments
por: Kim, Hyunwoo, et al.
Publicado: (2025)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
por: Lee, Hyungu, et al.
Publicado: (2025)

PINNet: a deep neural network with pathway prior knowledge for Alzheimer's disease
por: Kim, Yeojin, et al.
Publicado: (2022)

Deep Metric Loss for Multimodal Learning
por: Moon, Sehwan, et al.
Publicado: (2023)

Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks
por: Bencomo, Gianluca, et al.
Publicado: (2025)

Fast Training of Sinusoidal Neural Fields via Scaling Initialization
por: Yeom, Taesun, et al.
Publicado: (2024)

Frictional Q-Learning
por: Kim, Hyunwoo, et al.
Publicado: (2025)

Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
por: Jang, Sooyoung, et al.
Publicado: (2021)

Spatio-Temporal Graphs Beyond Grids: Benchmark for Maritime Anomaly Detection
por: Kim, Jeehong, et al.
Publicado: (2025)

FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer
por: Hwang, Dongyeong, et al.
Publicado: (2024)

Principal Components for Neural Network Initialization
por: Phan, Nhan, et al.
Publicado: (2025)

LION-DG: Layer-Informed Initialization with Deep Gradient Protocols for Accelerated Neural Network Training
por: Kim, Hyunjun
Publicado: (2026)

Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function
por: Abohwo, Jason, et al.
Publicado: (2025)

Revisiting Weight Averaging for Model Merging
por: Choi, Jiho, et al.
Publicado: (2024)

Global Minimizers of Sigmoid Contrastive Loss
por: Bangachev, Kiril, et al.
Publicado: (2025)

Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
por: Kim, Hyunjun
Publicado: (2026)

Achieving the Tightest Relaxation of Sigmoids for Formal Verification
por: Chevalier, Samuel, et al.
Publicado: (2024)

Revisiting Glorot Initialization for Long-Range Linear Recurrences
por: Bar, Noga, et al.
Publicado: (2025)

When Bias Meets Trainability: Connecting Theories of Initialization
por: Bassi, Alberto, et al.
Publicado: (2025)

The Initialization Determines Whether In-Context Learning Is Gradient Descent
por: Xie, Shifeng, et al.
Publicado: (2025)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
por: Abuduweili, Abulikemu, et al.
Publicado: (2024)

Initializing Services in Interactive ML Systems for Diverse Users
por: Bose, Avinandan, et al.
Publicado: (2023)

NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces
por: Kim, Jiwoo, et al.
Publicado: (2026)

World Models for Autonomous Driving: An Initial Survey
por: Guan, Yanchen, et al.
Publicado: (2024)

Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness
por: Joo, Hanseon, et al.
Publicado: (2025)

UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models
por: Kang, Hyunju, et al.
Publicado: (2026)

Random Initialization of Gated Sparse Adapters
por: Retault, Vi, et al.
Publicado: (2025)

NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
por: Li, Zeming, et al.
Publicado: (2025)

IDInit: A Universal and Stable Initialization Method for Neural Network Training
por: Pan, Yu, et al.
Publicado: (2025)

Randomly Initialized Networks Can Learn from Peer-to-Peer Consensus
por: Rodríguez-Betancourt, Esteban, et al.
Publicado: (2026)

Bit-Identical Medical Deep Learning via Structured Orthogonal Initialization
por: Shkolnikov, Yakov Pyotr
Publicado: (2026)

Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization
por: Eccles, Bailey J., et al.
Publicado: (2024)

No More Adam: Learning Rate Scaling at Initialization is All You Need
por: Xu, Minghao, et al.
Publicado: (2024)

Inversion-based Latent Bayesian Optimization
por: Chu, Jaewon, et al.
Publicado: (2024)

Mimetic Initialization of MLPs
por: Trockman, Asher, et al.
Publicado: (2026)

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA
por: Kang, Hyunju, et al.
Publicado: (2026)

Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models
por: Xia, Shi-Yu, et al.
Publicado: (2024)

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
por: Choi, Moonseok, et al.
Publicado: (2023)

The Impact of Initialization on LoRA Finetuning Dynamics
por: Hayou, Soufiane, et al.
Publicado: (2024)