Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909614250917888 |
|---|---|
| author | Kumar, Akarsh Clune, Jeff Lehman, Joel Stanley, Kenneth O. |
| author_facet | Kumar, Akarsh Clune, Jeff Lehman, Joel Stanley, Kenneth O. |
| contents | Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance. But does better performance necessarily imply better internal representations? While the representational optimist assumes it must, this position paper challenges that view. We compare neural networks evolved through an open-ended search process to networks trained via conventional stochastic gradient descent (SGD) on the simple task of generating a single image. This minimal setup offers a unique advantage: each hidden neuron's full functional behavior can be easily visualized as an image, thus revealing how the network's output behavior is internally constructed neuron by neuron. The result is striking: while both networks produce the same output behavior, their internal representations differ dramatically. The SGD-trained networks exhibit a form of disorganization that we term fractured entangled representation (FER). Interestingly, the evolved networks largely lack FER, even approaching a unified factored representation (UFR). In large models, FER may be degrading core model capacities like generalization, creativity, and (continual) learning. Therefore, understanding and mitigating FER could be critical to the future of representation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11581 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis Kumar, Akarsh Clune, Jeff Lehman, Joel Stanley, Kenneth O. Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing Much of the excitement in modern AI is driven by the observation that scaling up existing systems leads to better performance. But does better performance necessarily imply better internal representations? While the representational optimist assumes it must, this position paper challenges that view. We compare neural networks evolved through an open-ended search process to networks trained via conventional stochastic gradient descent (SGD) on the simple task of generating a single image. This minimal setup offers a unique advantage: each hidden neuron's full functional behavior can be easily visualized as an image, thus revealing how the network's output behavior is internally constructed neuron by neuron. The result is striking: while both networks produce the same output behavior, their internal representations differ dramatically. The SGD-trained networks exhibit a form of disorganization that we term fractured entangled representation (FER). Interestingly, the evolved networks largely lack FER, even approaching a unified factored representation (UFR). In large models, FER may be degrading core model capacities like generalization, creativity, and (continual) learning. Therefore, understanding and mitigating FER could be critical to the future of representation learning. |
| title | Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis |
| topic | Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2505.11581 |