Benign Overfitting for Regression with Trained Two-Layer ReLU Networks

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Park, Junhyung, Bloebaum, Patrick, Kasiviswanathan, Shiva Prasad
Format:	Preprint
Veröffentlicht:	2024
Schlagworte:	Machine Learning Artificial Intelligence
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

A Classical View on Benign Overfitting: The Role of Sample Size
von: Park, Junhyung, et al.
Veröffentlicht: (2025)

Initialization Matters: On the Benign Overfitting of Two-Layer ReLU CNN with Fully Trainable Layers
von: Shang, Shuning, et al.
Veröffentlicht: (2024)

Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes
von: Qiao, Dan, et al.
Veröffentlicht: (2024)

A Quantitative Characterization of Forgetting in Post-Training
von: Balasubramanian, Krishnakumar, et al.
Veröffentlicht: (2026)

From Tempered to Benign Overfitting in ReLU Neural Networks
von: Kornowski, Guy, et al.
Veröffentlicht: (2023)

N-ReLU: Zero-Mean Stochastic Extension of ReLU
von: Manik, Md Motaleb Hossen, et al.
Veröffentlicht: (2025)

The Geometry of ReLU Networks through the ReLU Transition Graph
von: Dhayalkar, Sahil Rajesh
Veröffentlicht: (2025)

The Resurrection of the ReLU
von: Horuz, Coşku Can, et al.
Veröffentlicht: (2025)

Does Flatness imply Generalization for Logistic Loss in Univariate Two-Layer ReLU Network?
von: Qiao, Dan, et al.
Veröffentlicht: (2025)

Expressive Power of ReLU and Step Networks under Floating-Point Operations
von: Park, Yeachan, et al.
Veröffentlicht: (2024)

Pathwise Explanation of ReLU Neural Networks
von: Lim, Seongwoo, et al.
Veröffentlicht: (2025)

Unveiling the Training Dynamics of ReLU Networks through a Linear Lens
von: Ye, Longqing
Veröffentlicht: (2025)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
von: Jia, Sheng, et al.
Veröffentlicht: (2025)

A Lower Bound for the Number of Linear Regions of Ternary ReLU Regression Neural Networks
von: Nakahara, Yuta, et al.
Veröffentlicht: (2025)

Uncovering Layer-Dependent Activation Sparsity Patterns in ReLU Transformers
von: Wild, Cody, et al.
Veröffentlicht: (2024)

Three Quantization Regimes for ReLU Networks
von: Ou, Weigutian, et al.
Veröffentlicht: (2024)

Activation-Descent Regularization for Input Optimization of ReLU Networks
von: Yu, Hongzhan, et al.
Veröffentlicht: (2024)

Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks
von: Schoots, Nandi, et al.
Veröffentlicht: (2025)

Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training
von: Milkert, Max, et al.
Veröffentlicht: (2023)

Benign Overfitting in Adversarial Training for Vision Transformers
von: Zhang, Jiaming, et al.
Veröffentlicht: (2026)

ReLU Networks for Exact Generation of Similar Graphs
von: Ghafoor, Mamoona, et al.
Veröffentlicht: (2026)

Beyond ReLU: Chebyshev-DQN for Enhanced Deep Q-Networks
von: Yazdannik, Saman, et al.
Veröffentlicht: (2025)

Training a Two Layer ReLU Network Analytically
von: Barbu, Adrian
Veröffentlicht: (2023)

Convergence of Shallow ReLU Networks on Weakly Interacting Data
von: Dana, Léo, et al.
Veröffentlicht: (2025)

Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
von: Harzli, Ouns El, et al.
Veröffentlicht: (2026)

Is ReLU Adversarially Robust?
von: Sooksatra, Korn, et al.
Veröffentlicht: (2024)

RePO: Understanding Preference Learning Through ReLU-Based Optimization
von: Wu, Junkang, et al.
Veröffentlicht: (2025)

$λ$-GELU: Learning Gating Hardness for Controlled ReLU-ization in Deep Networks
von: Pérez-Corral, Cristian, et al.
Veröffentlicht: (2026)

Risk Phase Transitions in Spiked Regression: Alignment Driven Benign and Catastrophic Overfitting
von: Li, Jiping, et al.
Veröffentlicht: (2025)

Precise Verification of Transformers through ReLU-Catalyzed Abstraction Refinement
von: Liu, Hengjie, et al.
Veröffentlicht: (2026)

Detecting Invariant Manifolds in ReLU-Based RNNs
von: Eisenmann, Lukas, et al.
Veröffentlicht: (2025)

Topological Signatures of ReLU Neural Network Activation Patterns
von: Bosca, Vicente, et al.
Veröffentlicht: (2025)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
von: Zhang, Zhengyan, et al.
Veröffentlicht: (2024)

ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models
von: Jha, Nandan Kumar, et al.
Veröffentlicht: (2024)

Deep ReLU Networks Have Surprisingly Simple Polytopes
von: Fan, Feng-Lei, et al.
Veröffentlicht: (2023)

The Median is Easier than it Looks: Approximation with a Constant-Depth, Linear-Width ReLU Network
von: Dutta, Abhigyan, et al.
Veröffentlicht: (2026)

Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape
von: Bantzis, Ioannis, et al.
Veröffentlicht: (2025)

Convex Formulations for Training Two-Layer ReLU Neural Networks
von: Prakhya, Karthik, et al.
Veröffentlicht: (2024)

Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks
von: Hashimoto, Ichiro
Veröffentlicht: (2025)

On the Principles of ReLU Networks with One Hidden Layer
von: Huang, Changcun
Veröffentlicht: (2024)

if (filemtime($filename) + $maxlifetime < time()) { unlink($filename); $count++; } } return $count; } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ protected function saveSession($sessId, $data): bool { $sessFile = $this->path . '/sess_' . $sessId; if ($handle = fopen($sessFile, 'w')) { $return = false; // Lock the file for exclusive access to avoid issues with multiple // processes writing session simultaneously: if (flock($handle, LOCK_EX)) { $return = fwrite($handle, $data); // Make sure that there's no trailing data by truncating the file to // the correct length: ftruncate($handle, strlen($data)); fflush($handle); flock($handle, LOCK_UN); } fclose($handle); if ($return !== false) { return true; } } // If we got this far, something went wrong with the file output... // It is tempting to throw an exception here, but this code is called // outside of the context of exception handling, so all we can do is // echo a message.

// Anecdotal testing Today and Yesterday seems to indicate destroy() // is called by the garbage collector and everything is good. // Something to keep in mind though. return 0; } /** * Write function that is called when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ public function write($sessId, $data): bool { if ($this->writesDisabled) { return true; } return $this->saveSession($sessId, $data); } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ abstract protected function saveSession($sessId, $data): bool; }

* * @return void */ public function writeClose() { // The assumption is that we're using PHP's ext/session. // session_write_close() will actually overwrite $_SESSION with an // empty array on completion -- which leads to a mismatch between what // is in the storage object and $_SESSION. To get around this, we // temporarily reset $_SESSION to an array, and then re-link it to // the storage object. // // Additionally, while you _can_ write to $_SESSION following a // session_write_close() operation, no changes made to it will be // flushed to the session handler. As such, we now mark the storage // object isImmutable. $storage = $this->getStorage(); if (! $storage->isImmutable()) { $_SESSION = $storage->toArray(true); session_write_close(); $storage->fromArray($_SESSION); $storage->markImmutable(); } } /** * Attempt to set the session name * * If the session has already been started, or if the name provided fails * validation, an exception will be raised. * * @param string $name * @return SessionManager * @throws Exception\InvalidArgumentException */ public function setName($name) { if ($this->sessionExists()) { throw new Exception\InvalidArgumentException( 'Cannot set session name after a session has already started'

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
VUFIND_SESSION	"50f9068104f4da7fa4410d22a59099b3"
ui	"standard"
language	"de"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1791267948.2525 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "50f9068104f4da7fa4410d22a59099b3" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1433}
SessionHelper	Laminas\Stdlib\ArrayObject {#1759}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"VUFIND_SESSION=50f9068104f4da7fa4410d22a59099b3; ui=standard; language=de"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.216.28"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"9507"
REDIRECT_URL	""" /Record/\n arxiv_https___arxiv_org_abs_2410_06191/Similar """
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/%0Aarxiv_https___arxiv_org_abs_2410_06191/Similar"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1791267948.245
REQUEST_TIME	1791267948

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler