Image de couverture de livre

Preserving Bilinear Weight Spectra with a Signed and Shrunk Quadratic Activation Function

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Abohwo, Jason, Mosen, Thomas
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Weight-based Decomposition: A Case for Bilinear MLPs
par: Pearce, Michael T., et autres
Publié: (2024)

Beyond Gaussian Initializations: Signal Preserving Weight Initialization for Odd-Sigmoid Activations
par: Lee, Hyunwoo, et autres
Publié: (2025)

Network-Aware Bilinear Tokenization for Brain Functional Connectivity Representation Learning
par: Milecki, Leo, et autres
Publié: (2026)

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning
par: Kim, Hyo Seo, et autres
Publié: (2024)

Bilinear Convolution Decomposition for Causal RL Interpretability
par: Oozeer, Narmeen, et autres
Publié: (2024)

Learning Equivariant Functions via Quadratic Forms
par: Karjol, Pavan, et autres
Publié: (2025)

Beyond Johnson-Lindenstrauss: Uniform Bounds for Sketched Bilinear Forms
par: Deb, Rohan, et autres
Publié: (2025)

Bilinear representation mitigates reversal curse and enables consistent model editing
par: Kim, Dong-Kyum, et autres
Publié: (2025)

Geometric Regularization in Mixture-of-Experts: The Disconnect Between Weights and Activations
par: Kim, Hyunjun
Publié: (2026)

Mining Generalizable Activation Functions
par: Vitvitskyi, Alex, et autres
Publié: (2026)

BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learning
par: Lin, Haohong, et autres
Publié: (2024)

Spectra-to-Structure and Structure-to-Spectra Inference Across the Periodic Table
par: Wang, Yufeng, et autres
Publié: (2025)

Towards a More Complete Theory of Function Preserving Transforms
par: Painter, Michael
Publié: (2024)

MIBP-Cert: Certified Training against Data Perturbations with Mixed-Integer Bilinear Programs
par: Lorenz, Tobias, et autres
Publié: (2024)

ANAct: Adaptive Normalization for Activation Functions
par: Peiwen, Yuan, et autres
Publié: (2022)

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs
par: Cheng, Wenhua, et autres
Publié: (2023)

WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
par: Chen, Sihan, et autres
Publié: (2025)

A Method on Searching Better Activation Functions
par: Sun, Haoyuan, et autres
Publié: (2024)

ScoresActivation: A New Activation Function for Model Agnostic Global Explainability by Design
par: Covaci, Emanuel, et autres
Publié: (2025)

WiSparse: Boosting LLM Inference Efficiency with Weight-Aware Mixed Activation Sparsity
par: Chen, Lei, et autres
Publié: (2026)

Intrinsic Structure as a Proxy for Saliency: SVD-Based Weight Preservation for Mixed-Precision Quantization in Large Language Models
par: Landge, Shashank, et autres
Publié: (2025)

Global Convergence in Neural ODEs: Impact of Activation Functions
par: Gao, Tianxiang, et autres
Publié: (2025)

Efficient Search for Customized Activation Functions with Gradient Descent
par: Strack, Lukas, et autres
Publié: (2024)

Beyond Expression Similarity: Contrastive Learning Recovers Functional Gene Associations from Protein Interaction Structure
par: Dury, Jason
Publié: (2026)

Find A Winning Sign: Sign Is All We Need to Win the Lottery
par: Oh, Junghun, et autres
Publié: (2025)

Unsupervised Learning for Quadratic Assignment
par: Min, Yimeng, et autres
Publié: (2025)

Analytical Solution of a Three-layer Network with a Matrix Exponential Activation Function
par: Gai, Kuo, et autres
Publié: (2024)

Non-Interfering Weight Fields: Treating Model Parameters as a Continuously Extensible Function
par: Chaudhry, Sarim
Publié: (2026)

Certified Signed Graph Unlearning
par: Zhao, Junpeng, et autres
Publié: (2025)

Towards Trustworthy Vital Sign Forecasting: Leveraging Uncertainty for Prediction Intervals
par: Wang, Li Rong, et autres
Publié: (2025)

FLAIN: Mitigating Backdoor Attacks in Federated Learning via Flipping Weight Updates of Low-Activation Input Neurons
par: Ding, Binbin, et autres
Publié: (2024)

Hallucination Detection via Activations of Open-Weight Proxy Analyzers
par: Singh, Akshita, et autres
Publié: (2026)

Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model
par: Mitra, Sinjini, et autres
Publié: (2026)

Beyond Weighted Summation: Learnable Nonlinear Aggregation Functions for Robust Artificial Neurons
par: Bozyigit, Berke Deniz
Publié: (2026)

Measuring What Matters: Intrinsic Distance Preservation as a Robust Metric for Embedding Quality
par: Hart, Steven N., et autres
Publié: (2024)

Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy
par: Huang, Zhendong, et autres
Publié: (2026)

CauchyNet: Compact and Data-Efficient Learning using Holomorphic Activation Functions
par: Zhang, Hong-Kun, et autres
Publié: (2025)

Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks
par: Patty, William H
Publié: (2025)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
par: Zhang, Zhengyan, et autres
Publié: (2024)

Catapult Dynamics and Phase Transitions in Quadratic Nets
par: Meltzer, David, et autres
Publié: (2023)