Robust visual learning under imperfection : navigating limited supervision and label uncertainty

Abstract

This doctoral dissertation explores effective strategies for robust visual learning in the presence of imperfect data sources, with a particular focus on three key areas: Semi-supervised Domain Adaptation (SSDA), Learning with Noisy Labels (LNL), and Federated Learning with Noisy Labels (F-LNL). Each area presents distinct challenges stemming from limited supervision and label uncertainty, necessitating methods to alleviate these challenges and enhance model adaptability and accuracy. In the realm of SSDA, this research introduces two state-of-the-art methods: Cross-domain Adaptive Clustering (CDAC) and Graph-based Adaptive Betweenness Clustering (G-ABC). These methodologies harness the presence of a few target labels to facilitate precise feature adaptation between source and target domains. Specifically, CDAC employs a proposed version of the adversarial adaptive clustering loss, as well as an adapted version of pseudo labeling, to enhance feature clustering across domains, thereby improving both inter-domain adaptation and intra-domain generalization. Similarly, G-ABC, constructed upon a refined graph structure, proposes adaptive betweenness clustering to align semantic features across domains by establishing connections based on semantic consistency and feature similarity. Both methods enhance feature alignment between domains, thereby bolstering model generalization to the target domain. Empirical evaluations conducted on diverse datasets such as DomainNet, Office-Home, and Office-31 demonstrate the superior performance of these methods over existing state-of-the-art SSDA algorithms. To tackle the pervasive issue of label noise in LNL tasks, this study proposes a novel algorithm called Neighborhood Collective Estimation (NCE), comprising two steps: 1) Neighborhood Collective Noise Verification, which categorizes all training samples into either a clean or noisy subset, and 2) Neighborhood Collective Label Correction, which corrects the labels of noisy samples. To this end, NCE enhances predictive reliability by contrasting candidate samples against their feature-space nearest neighbors, enriching predictive information and mitigating biases in noisy label identification and correction. The efficacy of NCE is demonstrated through superior performance on benchmarks including CIFAR-10, CIFAR-100, and Clothing1M. In the domain of F-LNL, this thesis introduces the FedDiv framework to address challenges originating from data heterogeneity and noise heterogeneity. Leveraging complementary knowledge learned from all clients, this FedDiv succeeds in decreasing the adverse effects of label noise across local clients while preserving data privacy. To be specific, FedDiv proposes global noise filtering and predictive consistency-based sampling to enhance the robustness and stability of learning in the decentralized scenarios. The effectiveness of FedDiv has been demonstrated by empirical evaluations on benchmark datasets such as CIFAR-10, CIFAR-100, and Clothing1M under various label noise settings for both IID and non-IID data partitions. In conclusion, this dissertation contributes robust methodologies that advance the frontier of machine learning by enabling models to learn effectively from imperfect data. These methodologies, inspired by the adaptability of the human visual system, demonstrate significant progress in handling real-world data complexities. Extensive experimental validations set new benchmarks for robust learning in the presence of data imperfections.