Resursive local estimation: algorithm, performance and applications

Abstract

Adaptive filters are frequently employed in many applications, such as, system identification, adaptive echo cancellation (AEC), active noise control (ANC), adaptive beamforming, speech signal processing and other related problems, in which the statistic of the underlying signals is either unknown a priori, or slowly-varying.

Given the observed signals under study, we shall consider, in this dissertation, the time-varying linear model with Gaussian or contaminated Gaussian (CG) noises. In particular, we focus on recursive local estimation and its applications in linear systems. We base our development on the concept of local likelihood function (LLF) and local posterior probability for parameter estimation, which lead to efficient adaptive filtering algorithms. We also study the convergence performance of these algorithms and their applications by theoretical analyses. As for applications, another important one is to utilize adaptive filters to obtain recursive hypothesis testing and model order selection methods.

It is known that the maximum likelihood estimate (MLE) may lead to large variance or ill-conditioning problems when the number of observations is limited. An effective approach to address these problems is to employ various form of regularization in order to reduce the variance at the expense of slightly increased bias. In general, this can be viewed as adopting the Bayesian estimation, where the regularization can be viewed as providing a certain prior density of the parameters to be estimated. By adopting different prior densities in the LLF, we derive the variable regularized QR decomposition-based recursive least squares (VR-QRRLS) and recursive least M-estimate (VR-QRRLM) algorithms. An improved state-regularized variable forgetting factor QRRLS (SR-VFF-QRRLS) algorithm is also proposed. By approximating the covariance matrix in the RLS, new variable regularized and variable step-size transform domain normalized least mean square (VR-TDNLMS and VSS-TDNLMS) algorithms are proposed. Convergence behaviors of these algorithms are studied to characterize their performance and provide useful guidelines for selecting appropriate parameters in practical applications.

Based on the local Bayesian estimation framework for linear model parameters developed previously, the resulting estimate can be utilized for recursive nonstationarity detection. This can be cast under the problem of hypothesis testing, as the hypotheses can be viewed as two competitive models between stationary and nonstationary to be selected. In this dissertation, we develop new regularized and recursive generalized likelihood ratio test (GLRT), Rao’s and Wald tests, which can be implemented recursively in a QRRLS-type adaptive filtering algorithm with low computational complexity. Another issue to be addressed in nonstationarity detection is the selection of various models or model orders. In particular, we derive a recursive method for model order selection from the Bayesian Information Criterion (BIC) based on recursive local estimation.

In general, the algorithms proposed in this dissertation have addressed some of the important problems in estimation and detection under the local and recursive Bayesian estimation framework. They are intrinsically connected together and can potentially be utilized for various applications. In this dissertation, their applications to adaptive beamforming, ANC system and speech signal processing, e.g. adaptive frequency estimation and nonstationarity detection, have been studied. For adaptive beamforming, the difficulties in determining the regularization or loading factor have been explored by automatically selecting the regularization parameter. For ANC systems, to combat uncertainties in the secondary path estimation, regularization techniques can be employed. Consequently, a new filtered-x VR-QRRLM (Fx-VR-QRRLM) algorithm is proposed and the theoretical analysis helps to address challenging problems in the design of ANC systems. On the other hand, for ANC systems with online secondary-path modeling, the coupling effect of the ANC controller and the secondary path estimator is thoroughly studied by analyzing the Fx-LMS algorithm. For speech signal processing, new approaches for recursive nonstationarity detection with automatic model order selection are proposed, which provides online time-varying autoregressive (TVAR) parameter estimation and the corresponding stationary intervals with low complexity.