Cover Image

SmartMixed: A Two-Phase Training Strategy for Adaptive Activation Function Learning in Neural Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Omidvar, Amin
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks
by: Patty, William H
Published: (2025)

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
by: Omidvar, Hamed, et al.
Published: (2026)

A Convexity-dependent Two-Phase Training Algorithm for Deep Neural Networks
by: Hrycej, Tomas, et al.
Published: (2025)

ANAct: Adaptive Normalization for Activation Functions
by: Peiwen, Yuan, et al.
Published: (2022)

Learning from Historical Activations in Graph Neural Networks
by: Galron, Yaniv, et al.
Published: (2026)

MetaSymNet: A Tree-like Symbol Network with Adaptive Architecture and Activation Functions
by: Li, Yanjie, et al.
Published: (2023)

More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations
by: Wang, Mingze, et al.
Published: (2026)

Spectral Edge Dynamics: An Analytical-Empirical Study of Phase Transitions in Neural Network Training
by: Xu, Yongzhong
Published: (2026)

GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration
by: Qiao, Tong, et al.
Published: (2024)

Global Convergence in Neural ODEs: Impact of Activation Functions
by: Gao, Tianxiang, et al.
Published: (2025)

Ray-Tracing for Conditionally Activated Neural Networks
by: Gallicchio, Claudio, et al.
Published: (2025)

Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies
by: Yang, Yuan-Chih, et al.
Published: (2025)

Adaptive Width Neural Networks
by: Errica, Federico, et al.
Published: (2025)

Learning Adaptive Distribution Alignment with Neural Characteristic Function for Graph Domain Adaptation
by: Chen, Wei, et al.
Published: (2026)

GAC: Noise-Aware Adaptive Mixing for Hybrid SFT-RL Post-Training
by: Hu, Yuelin, et al.
Published: (2026)

Eau De $Q$-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning
by: Vincent, Théo, et al.
Published: (2025)

Efficient Neural Networks with Discrete Cosine Transform Activations
by: Martinez-Gost, Marc, et al.
Published: (2025)

Self-Abstraction Learning for Effective and Stable Training of Deep Neural Networks
by: Cho, Wonyong, et al.
Published: (2026)

Adaptive Data Harvesting for Efficient Neural Network Learning with Universal Constraints
by: Kang, Siteng, et al.
Published: (2026)

APEX: Probing Neural Networks via Activation Perturbation
by: Ren, Tao, et al.
Published: (2026)

PRISM: Distribution-free Adaptive Computation of Matrix Functions for Accelerating Neural Network Training
by: Yang, Shenghao, et al.
Published: (2026)

Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training
by: Varshney, Ayush K., et al.
Published: (2026)

DynaMoE: Dynamic Token-Level Expert Activation with Layer-Wise Adaptive Capacity for Mixture-of-Experts Neural Networks
by: Gülmez, Gökdeniz
Published: (2026)

Adaptive Physics-informed Neural Networks: A Survey
by: Torres, Edgar, et al.
Published: (2025)

GNN-XAR: A Graph Neural Network for Explainable Activity Recognition in Smart Homes
by: Fiori, Michele, et al.
Published: (2025)

BEND: Bagging Deep Learning Training Based on Efficient Neural Network Diffusion
by: Wei, Jia, et al.
Published: (2024)

BugSweeper: Function-Level Detection of Smart Contract Vulnerabilities Using Graph Neural Networks
by: Lee, Uisang, et al.
Published: (2025)

Breaking the Conventional Forward-Backward Tie in Neural Networks: Activation Functions
by: Troiano, Luigi, et al.
Published: (2025)

Limitations of Physics-Informed Neural Networks: a Study on Smart Grid Surrogation
by: Cestero, Julen, et al.
Published: (2025)

Z-Error Loss for Training Neural Networks
by: Godin, Guillaume
Published: (2025)

Energy Consumption in Parallel Neural Network Training
by: Huber, Philipp, et al.
Published: (2025)

Training Neural Networks for Modularity aids Interpretability
by: Golechha, Satvik, et al.
Published: (2024)

Automatic Stability and Recovery for Neural Network Training
by: Or, Barak
Published: (2026)

Gradient-Free Training of Quantized Neural Networks
by: Cohen, Noa, et al.
Published: (2024)

GRANOLA: Adaptive Normalization for Graph Neural Networks
by: Eliasof, Moshe, et al.
Published: (2024)

Toward Improving fNIRS Classification: A Study on Activation Functions in Deep Neural Architectures
by: Adeli, Behtom, et al.
Published: (2025)

A Functional Perspective on Knowledge Distillation in Neural Networks
by: Mason-Williams, Israel, et al.
Published: (2025)

Meta-Learning Adaptive Loss Functions
by: Raymond, Christian, et al.
Published: (2023)

Probing Graph Neural Network Activation Patterns Through Graph Topology
by: Tori, Floriano, et al.
Published: (2026)

Graph Neural Networks Need Cluster-Normalize-Activate Modules
by: Skryagin, Arseny, et al.
Published: (2024)