Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916013494239232 |
|---|---|
| author | Chen, Yuzhu Qin, Tian Tian, Xinmei He, Fengxiang Tao, Dacheng |
| author_facet | Chen, Yuzhu Qin, Tian Tian, Xinmei He, Fengxiang Tao, Dacheng |
| contents | Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to their expressive power remains premature. Focusing on 2-layer ReLU networks, this paper investigates the impact of enforcing equivariance constraints on the expressive power. By examining the boundary hyperplanes and the channel vectors, we constructively demonstrate that enforcing equivariance constraints could undermine the expressive power. Naturally, this drawback can be compensated for by enlarging the model size -- we further prove upper bounds on the required enlargement for compensation. Surprisingly, we show that the enlarged neural architectures have reduced hypothesis space dimensionality, implying even better generalizability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_09673 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power Chen, Yuzhu Qin, Tian Tian, Xinmei He, Fengxiang Tao, Dacheng Machine Learning Artificial Intelligence Neural and Evolutionary Computing Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to their expressive power remains premature. Focusing on 2-layer ReLU networks, this paper investigates the impact of enforcing equivariance constraints on the expressive power. By examining the boundary hyperplanes and the channel vectors, we constructively demonstrate that enforcing equivariance constraints could undermine the expressive power. Naturally, this drawback can be compensated for by enlarging the model size -- we further prove upper bounds on the required enlargement for compensation. Surprisingly, we show that the enlarged neural architectures have reduced hypothesis space dimensionality, implying even better generalizability. |
| title | Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power |
| topic | Machine Learning Artificial Intelligence Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2512.09673 |