Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximation
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912557589069824 |
|---|---|
| author | Xie, Pengcheng Zhou, Zihao Zhou, Zijian |
| author_facet | Xie, Pengcheng Zhou, Zihao Zhou, Zijian |
| contents | This paper introduce a novel metric of an objective function f, we say VC (value change) to measure the difficulty and approximation affection when conducting an neural network approximation task, and it numerically supports characterizing the local performance and behavior of neural network approximation. Neural networks often suffer from unpredictable local performance, which can hinder their reliability in critical applications. VC addresses this issue by providing a quantifiable measure of local value changes in network behavior, offering insights into the stability and performance for achieving the neural-network approximation. We investigate some fundamental theoretical properties of VC and identified two intriguing phenomena in neural network approximation: the VC-tendency and the minority-tendency. These trends respectively characterize how pointwise errors evolve in relation to the distribution of VC during the approximation process.In addition, we propose a novel metric based on VC, which measures the distance between two functions from the perspective of variation. Building upon this metric, we further propose a new preprocessing framework for neural network approximation. Numerical results including the real-world experiment and the PDE-related scientific problem support our discovery and pre-processing acceleration method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_20290 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximation Xie, Pengcheng Zhou, Zihao Zhou, Zijian Machine Learning Artificial Intelligence Numerical Analysis Optimization and Control 68T07, 65K05, 65D15, 90C30 This paper introduce a novel metric of an objective function f, we say VC (value change) to measure the difficulty and approximation affection when conducting an neural network approximation task, and it numerically supports characterizing the local performance and behavior of neural network approximation. Neural networks often suffer from unpredictable local performance, which can hinder their reliability in critical applications. VC addresses this issue by providing a quantifiable measure of local value changes in network behavior, offering insights into the stability and performance for achieving the neural-network approximation. We investigate some fundamental theoretical properties of VC and identified two intriguing phenomena in neural network approximation: the VC-tendency and the minority-tendency. These trends respectively characterize how pointwise errors evolve in relation to the distribution of VC during the approximation process.In addition, we propose a novel metric based on VC, which measures the distance between two functions from the perspective of variation. Building upon this metric, we further propose a new preprocessing framework for neural network approximation. Numerical results including the real-world experiment and the PDE-related scientific problem support our discovery and pre-processing acceleration method. |
| title | Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximation |
| topic | Machine Learning Artificial Intelligence Numerical Analysis Optimization and Control 68T07, 65K05, 65D15, 90C30 |
| url | https://arxiv.org/abs/2508.20290 |