Copertina

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Lee, Hyunwoo, Choi, Hayoung, Kim, Hyunju
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis
di: Lee, Hyunwoo, et al.
Pubblicazione: (2024)

Progressive Weight Loading: Accelerating Initial Inference and Gradually Boosting Performance on Resource-Constrained Environments
di: Kim, Hyunwoo, et al.
Pubblicazione: (2025)

Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
di: Lee, Hyungu, et al.
Pubblicazione: (2025)

PINNet: a deep neural network with pathway prior knowledge for Alzheimer's disease
di: Kim, Yeojin, et al.
Pubblicazione: (2022)

Deep Metric Loss for Multimodal Learning
di: Moon, Sehwan, et al.
Pubblicazione: (2023)

Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks
di: Bencomo, Gianluca, et al.
Pubblicazione: (2025)

Fast Training of Sinusoidal Neural Fields via Scaling Initialization
di: Yeom, Taesun, et al.
Pubblicazione: (2024)

Frictional Q-Learning
di: Kim, Hyunwoo, et al.
Pubblicazione: (2025)

Entropy-Aware Model Initialization for Effective Exploration in Deep Reinforcement Learning
di: Jang, Sooyoung, et al.
Pubblicazione: (2021)

Spatio-Temporal Graphs Beyond Grids: Benchmark for Maritime Anomaly Detection
di: Kim, Jeehong, et al.
Pubblicazione: (2025)

FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer
di: Hwang, Dongyeong, et al.
Pubblicazione: (2024)

Principal Components for Neural Network Initialization
di: Phan, Nhan, et al.
Pubblicazione: (2025)

LION-DG: Layer-Informed Initialization with Deep Gradient Protocols for Accelerated Neural Network Training
di: Kim, Hyunjun
Pubblicazione: (2026)

Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function
di: Abohwo, Jason, et al.
Pubblicazione: (2025)

Revisiting Weight Averaging for Model Merging
di: Choi, Jiho, et al.
Pubblicazione: (2024)

Global Minimizers of Sigmoid Contrastive Loss
di: Bangachev, Kiril, et al.
Pubblicazione: (2025)

Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
di: Kim, Hyunjun
Pubblicazione: (2026)

Achieving the Tightest Relaxation of Sigmoids for Formal Verification
di: Chevalier, Samuel, et al.
Pubblicazione: (2024)

Revisiting Glorot Initialization for Long-Range Linear Recurrences
di: Bar, Noga, et al.
Pubblicazione: (2025)

When Bias Meets Trainability: Connecting Theories of Initialization
di: Bassi, Alberto, et al.
Pubblicazione: (2025)

The Initialization Determines Whether In-Context Learning Is Gradient Descent
di: Xie, Shifeng, et al.
Pubblicazione: (2025)

Revisiting the Initial Steps in Adaptive Gradient Descent Optimization
di: Abuduweili, Abulikemu, et al.
Pubblicazione: (2024)

Initializing Services in Interactive ML Systems for Diverse Users
di: Bose, Avinandan, et al.
Pubblicazione: (2023)

NNiT: Width-Agnostic Neural Network Generation with Structurally Aligned Weight Spaces
di: Kim, Jiwoo, et al.
Pubblicazione: (2026)

World Models for Autonomous Driving: An Initial Survey
di: Guan, Yanchen, et al.
Pubblicazione: (2024)

Bridging KAN and MLP: MJKAN, a Hybrid Architecture with Both Efficiency and Expressiveness
di: Joo, Hanseon, et al.
Pubblicazione: (2025)

UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models
di: Kang, Hyunju, et al.
Pubblicazione: (2026)

Random Initialization of Gated Sparse Adapters
di: Retault, Vi, et al.
Pubblicazione: (2025)

NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
di: Li, Zeming, et al.
Pubblicazione: (2025)

IDInit: A Universal and Stable Initialization Method for Neural Network Training
di: Pan, Yu, et al.
Pubblicazione: (2025)

Randomly Initialized Networks Can Learn from Peer-to-Peer Consensus
di: Rodríguez-Betancourt, Esteban, et al.
Pubblicazione: (2026)

Bit-Identical Medical Deep Learning via Structured Orthogonal Initialization
di: Shkolnikov, Yakov Pyotr
Pubblicazione: (2026)

Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initialization
di: Eccles, Bailey J., et al.
Pubblicazione: (2024)

No More Adam: Learning Rate Scaling at Initialization is All You Need
di: Xu, Minghao, et al.
Pubblicazione: (2024)

Inversion-based Latent Bayesian Optimization
di: Chu, Jaewon, et al.
Pubblicazione: (2024)

Mimetic Initialization of MLPs
di: Trockman, Asher, et al.
Pubblicazione: (2026)

JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA
di: Kang, Hyunju, et al.
Pubblicazione: (2026)

Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models
di: Xia, Shi-Yu, et al.
Pubblicazione: (2024)

Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
di: Choi, Moonseok, et al.
Pubblicazione: (2023)

The Impact of Initialization on LoRA Finetuning Dynamics
di: Hayou, Soufiane, et al.
Pubblicazione: (2024)