ActNAS : Generating Efficient YOLO Models using Activation NAS
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913579047845888 |
|---|---|
| author | Sah, Sudhakar Kumar, Ravish Ganji, Darshan C. Saboori, Ehsan |
| author_facet | Sah, Sudhakar Kumar, Ravish Ganji, Darshan C. Saboori, Ehsan |
| contents | Activation functions introduce non-linearity into Neural Networks, enabling them to learn complex patterns. Different activation functions vary in speed and accuracy, ranging from faster but less accurate options like ReLU to slower but more accurate functions like SiLU or SELU. Typically, same activation function is used throughout an entire model architecture. In this paper, we conduct a comprehensive study on the effects of using mixed activation functions in YOLO-based models, evaluating their impact on latency, memory usage, and accuracy across CPU, NPU, and GPU edge devices. We also propose a novel approach that leverages Neural Architecture Search (NAS) to design YOLO models with optimized mixed activation functions.The best model generated through this method demonstrates a slight improvement in mean Average Precision (mAP) compared to baseline model (SiLU), while it is 22.28% faster and consumes 64.15% less memory on the reference NPU device. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_10887 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ActNAS : Generating Efficient YOLO Models using Activation NAS Sah, Sudhakar Kumar, Ravish Ganji, Darshan C. Saboori, Ehsan Machine Learning Neural and Evolutionary Computing Activation functions introduce non-linearity into Neural Networks, enabling them to learn complex patterns. Different activation functions vary in speed and accuracy, ranging from faster but less accurate options like ReLU to slower but more accurate functions like SiLU or SELU. Typically, same activation function is used throughout an entire model architecture. In this paper, we conduct a comprehensive study on the effects of using mixed activation functions in YOLO-based models, evaluating their impact on latency, memory usage, and accuracy across CPU, NPU, and GPU edge devices. We also propose a novel approach that leverages Neural Architecture Search (NAS) to design YOLO models with optimized mixed activation functions.The best model generated through this method demonstrates a slight improvement in mean Average Precision (mAP) compared to baseline model (SiLU), while it is 22.28% faster and consumes 64.15% less memory on the reference NPU device. |
| title | ActNAS : Generating Efficient YOLO Models using Activation NAS |
| topic | Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2410.10887 |