Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2412.13321 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913616585818112 |
|---|---|
| author | Xie, Tiankai Chen, Jiaqing Yang, Yaoqing Geniesse, Caleb Shi, Ge Chaudhari, Ajinkya Cava, John Kevin Mahoney, Michael W. Perciano, Talita Weber, Gunther H. Maciejewski, Ross |
| author_facet | Xie, Tiankai Chen, Jiaqing Yang, Yaoqing Geniesse, Caleb Shi, Ge Chaudhari, Ajinkya Cava, John Kevin Mahoney, Michael W. Perciano, Talita Weber, Gunther H. Maciejewski, Ross |
| contents | Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. We demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_13321 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics Xie, Tiankai Chen, Jiaqing Yang, Yaoqing Geniesse, Caleb Shi, Ge Chaudhari, Ajinkya Cava, John Kevin Mahoney, Michael W. Perciano, Talita Weber, Gunther H. Maciejewski, Ross Machine Learning Modern machine learning often relies on optimizing a neural network's parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network's parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens, a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. We demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem. |
| title | LossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.13321 |