Zapisane w:
| Główni autorzy: | , |
|---|---|
| Format: | Preprint |
| Wydane: |
2001
|
| Hasła przedmiotowe: | |
| Dostęp online: | https://arxiv.org/abs/math/0104083 |
| Etykiety: |
Dodaj etykietę
Nie ma etykietki, Dołącz pierwszą etykiete!
|
| _version_ | 1866917022517952512 |
|---|---|
| author | Benichou, Bertrand Saito, Naoki |
| author_facet | Benichou, Bertrand Saito, Naoki |
| contents | Finding a basis/coordinate system that can efficiently represent an input data stream by viewing them as realizations of a stochastic process is of tremendous importance in many fields including data compression and computational neuroscience. Two popular measures of such efficiency of a basis are sparsity (measured by the expected $\ell^p$ norm, $0 < p \leq 1$) and statistical independence (measured by the mutual information). Gaining deeper understanding of their intricate relationship, however, remains elusive. Therefore, we chose to study a simple synthetic stochastic process called the spike process, which puts a unit impulse at a random location in an $n$-dimensional vector for each realization. For this process, we obtained the following results: 1) The standard basis is the best both in terms of sparsity and statistical independence if $n \geq 5$ and the search of basis is restricted within all possible orthonormal bases in $R^n$; 2) If we extend our basis search in all possible invertible linear transformations in $R^n$, then the best basis in statistical independence differs from the one in sparsity; 3) In either of the above, the best basis in statistical independence is not unique, and there even exist those which make the inputs completely dense; 4) There is no linear invertible transformation that achieves the true statistical independence for $n > 2$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_math_0104083 |
| institution | arXiv |
| publishDate | 2001 |
| record_format | arxiv |
| spellingShingle | Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process Benichou, Bertrand Saito, Naoki Numerical Analysis Statistics Theory Quantitative Biology 65T60, 65C60, 62M99, 94A17 Finding a basis/coordinate system that can efficiently represent an input data stream by viewing them as realizations of a stochastic process is of tremendous importance in many fields including data compression and computational neuroscience. Two popular measures of such efficiency of a basis are sparsity (measured by the expected $\ell^p$ norm, $0 < p \leq 1$) and statistical independence (measured by the mutual information). Gaining deeper understanding of their intricate relationship, however, remains elusive. Therefore, we chose to study a simple synthetic stochastic process called the spike process, which puts a unit impulse at a random location in an $n$-dimensional vector for each realization. For this process, we obtained the following results: 1) The standard basis is the best both in terms of sparsity and statistical independence if $n \geq 5$ and the search of basis is restricted within all possible orthonormal bases in $R^n$; 2) If we extend our basis search in all possible invertible linear transformations in $R^n$, then the best basis in statistical independence differs from the one in sparsity; 3) In either of the above, the best basis in statistical independence is not unique, and there even exist those which make the inputs completely dense; 4) There is no linear invertible transformation that achieves the true statistical independence for $n > 2$. |
| title | Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process |
| topic | Numerical Analysis Statistics Theory Quantitative Biology 65T60, 65C60, 62M99, 94A17 |
| url | https://arxiv.org/abs/math/0104083 |