Extreme eigenvalues of large-dimensional spiked Fisher matrices with application

Abstract

Consider two pp-variate populations, not necessarily Gaussian, with covariance matrices Σ1Σ1 and Σ2Σ2, respectively. Let S1S1 and S2S2 be the corresponding sample covariance matrices with degrees of freedom mm and nn. When the difference ΔΔ between Σ1Σ1 and Σ2Σ2 is of small rank compared to p,mp,m and nn, the Fisher matrix S:=S−12S1S:=S2−1S1 is called a spiked Fisher matrix. When p,mp,m and nn grow to infinity proportionally, we establish a phase transition for the extreme eigenvalues of the Fisher matrix: a displacement formula showing that when the eigenvalues of ΔΔ (spikes) are above (or under) a critical value, the associated extreme eigenvalues of SS will converge to some point outside the support of the global limit (LSD) of other eigenvalues (become outliers); otherwise, they will converge to the edge points of the LSD. Furthermore, we derive central limit theorems for those outlier eigenvalues of SS. The limiting distributions are found to be Gaussian if and only if the corresponding population spike eigenvalues in ΔΔ are simple. Two applications are introduced. The first application uses the largest eigenvalue of the Fisher matrix to test the equality between two high-dimensional covariance matrices, and explicit power function is found under the spiked alternative. The second application is in the field of signal detection, where an estimator for the number of signals is proposed while the covariance structure of the noise is arbitrary.