Sequential combination of weighted and nonparametric bagging for classification

Abstract

We propose a simple sequential procedure for bagged classification, which modifies nonparametric bagging by randomizing class labels of resampled data points. The random labelling feature of the procedure also enables us to undertake unsupervised classification with the benefit of supervised learning. Theoretical properties are given for the nearest neighbour classifier in the case of supervised learning and a hard-thresholding indicator in the case of unsupervised learning, showing that sequential bagging accelerates convergence of the bagged predictor to the Bayes rule. Simulation results are provided in support of the proposed method.