Multiscale nonlinear integration drives accurate encoding of input information
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909394545934336 |
|---|---|
| author | Nicoletti, Giorgio Busiello, Daniel M. |
| author_facet | Nicoletti, Giorgio Busiello, Daniel M. |
| contents | Biological and artificial systems encode information through several complex nonlinear operations, making their exact study a formidable challenge. These internal mechanisms often take place across multiple timescales and process external signals to enable functional output responses. In this work, we focus on two widely implemented paradigms: nonlinear summation, where signals are first processed independently and then combined; and nonlinear integration, where they are combined first and then processed. We study a general model where the input signal is propagated to an output unit through a processing layer via nonlinear activation functions. Further, we distinguish between the two cases of fast and slow processing timescales. We demonstrate that integration and fast-processing capabilities systematically enhance input-output mutual information over a wide range of parameters and system sizes, while simultaneously enabling tunable input discrimination. Moreover, we reveal that high-dimensional embeddings and low-dimensional projections emerge naturally as optimal competing strategies. Our results uncover the foundational features of nonlinear information processing with profound implications for both biological and artificial systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_11710 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Multiscale nonlinear integration drives accurate encoding of input information Nicoletti, Giorgio Busiello, Daniel M. Statistical Mechanics Biological and artificial systems encode information through several complex nonlinear operations, making their exact study a formidable challenge. These internal mechanisms often take place across multiple timescales and process external signals to enable functional output responses. In this work, we focus on two widely implemented paradigms: nonlinear summation, where signals are first processed independently and then combined; and nonlinear integration, where they are combined first and then processed. We study a general model where the input signal is propagated to an output unit through a processing layer via nonlinear activation functions. Further, we distinguish between the two cases of fast and slow processing timescales. We demonstrate that integration and fast-processing capabilities systematically enhance input-output mutual information over a wide range of parameters and system sizes, while simultaneously enabling tunable input discrimination. Moreover, we reveal that high-dimensional embeddings and low-dimensional projections emerge naturally as optimal competing strategies. Our results uncover the foundational features of nonlinear information processing with profound implications for both biological and artificial systems. |
| title | Multiscale nonlinear integration drives accurate encoding of input information |
| topic | Statistical Mechanics |
| url | https://arxiv.org/abs/2411.11710 |