Monte Carlo methods for variance reduction and log-rank test

Abstract

Monte Carlo methods are frequently used in Bayesian analysis to use randomness to solve problems. This work consists of two parts concerning two fundamental areas of statistics, estimation and hypothesis testing.

The first part is about variance reduction. Taking advantage of the information wasted by nonparametric importance sampling (NIS), a novel estimation algorithm called NIS-CV is constructed to achieve small variance. NIS-CV has at least three advantages over NIS. Firstly, the kernel density estimator in NIS is mixed with a heavy-tailed distribution for robust estimation. Secondly, the computational burden is reduced by limiting the number of proposal components or kernels via resampling. Thirdly, the estimation accuracy is improved utilizing stratification and the control variates (CV) method. Self-normalized version of NIS-CV is also built to handle un-normalized Bayesian problems. Through simulations and a real example, it is shown that NIS-CV outperforms traditional NIS. In addition, the support points method and cluster analysis can be further integrated into our algorithm. They lead to promising estimation results in specific problems.

The second part is about survival curve comparison, a fundamental problem in survival analysis. Although abundant frequentist methods have been developed for comparing survival functions, inference procedures from the Bayesian perspective are rather limited. In this article, we extract the quantity of interest from the classic log-rank test and propose its Bayesian counterpart. Monte Carlo methods, including a Gibbs sampler and a sequential importance sampling procedure, are developed to draw posterior samples and a decision rule of hypothesis testing is constructed for making inference. Via simulations and real data analysis, the proposed Bayesian log-rank test is shown to be asymptotically equivalent to the classic one in the case of using noninformative prior distributions, providing a Bayesian interpretation of the log-rank test. In addition, when using the correct prior information from historical data, the Bayesian log-rank test is shown to outperform the classic one in terms of power.