A Sample-Deficient Analysis of the Leading Generalized Eigenvalue for the Detection of Signals in Colored Gaussian Noise

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Dharmawansa, Prathapasinghe, Atapattu, Saman, Evans, Jamie, Sithamparanathan, Kandeepan
Format: Preprint
Published: 2023
Subjects:
Online Access:
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866909180725559296
author Dharmawansa, Prathapasinghe
Atapattu, Saman
Evans, Jamie
Sithamparanathan, Kandeepan
author_facet Dharmawansa, Prathapasinghe
Atapattu, Saman
Evans, Jamie
Sithamparanathan, Kandeepan
contents This paper investigates the signal detection problem in colored Gaussian noise with an unknown covariance matrix. To be specific, we consider a sample deficient scenario in which the number of signal bearing samples ($n$) is strictly smaller than the dimensionality of the signal space ($m$). Our test statistic is the leading generalized eigenvalue of the whitened sample covariance matrix (a.k.a. $F$-matrix) which is constructed by whitening the signal bearing sample covariance matrix with noise-only sample covariance matrix. The whitening operation along with the observation model induces a single spiked covariance structure on the $F$-matrix. Moreover, the sample deficiency (i.e., $m>n$) in turn makes this $F$-matrix rank deficient, thereby {\it singular}. Therefore, a simple exact statistical characterization of the leading generalized eigenvalue (l.g.e.) of a complex correlated {\it singular} $F$-matrix with a single spiked associated covariance is of paramount importance to assess the performance of the detector (i.e., the receiver operating characteristics (ROC)). To this end, we adopt the powerful orthogonal polynomial technique in random matrix theory to derive a new finite dimensional c.d.f. expression for the l.g.e. of this particular $F$-matrix. It turns out that when the noise only sample covariance matrix is nearly rank deficient and the signal-to-noise ratio is $O(m)$, the ROC profile converges to a remarkably simple limiting profile.
format Preprint
id arxiv_https___arxiv_org_abs_2306_11214
institution arXiv
publishDate 2023
record_format arxiv
spellingShingle A Sample-Deficient Analysis of the Leading Generalized Eigenvalue for the Detection of Signals in Colored Gaussian Noise
Dharmawansa, Prathapasinghe
Atapattu, Saman
Evans, Jamie
Sithamparanathan, Kandeepan
Signal Processing
This paper investigates the signal detection problem in colored Gaussian noise with an unknown covariance matrix. To be specific, we consider a sample deficient scenario in which the number of signal bearing samples ($n$) is strictly smaller than the dimensionality of the signal space ($m$). Our test statistic is the leading generalized eigenvalue of the whitened sample covariance matrix (a.k.a. $F$-matrix) which is constructed by whitening the signal bearing sample covariance matrix with noise-only sample covariance matrix. The whitening operation along with the observation model induces a single spiked covariance structure on the $F$-matrix. Moreover, the sample deficiency (i.e., $m>n$) in turn makes this $F$-matrix rank deficient, thereby {\it singular}. Therefore, a simple exact statistical characterization of the leading generalized eigenvalue (l.g.e.) of a complex correlated {\it singular} $F$-matrix with a single spiked associated covariance is of paramount importance to assess the performance of the detector (i.e., the receiver operating characteristics (ROC)). To this end, we adopt the powerful orthogonal polynomial technique in random matrix theory to derive a new finite dimensional c.d.f. expression for the l.g.e. of this particular $F$-matrix. It turns out that when the noise only sample covariance matrix is nearly rank deficient and the signal-to-noise ratio is $O(m)$, the ROC profile converges to a remarkably simple limiting profile.
title A Sample-Deficient Analysis of the Leading Generalized Eigenvalue for the Detection of Signals in Colored Gaussian Noise
topic Signal Processing
url https://arxiv.org/abs/2306.11214