Imbalanced data problems in digital investigation

Abstract

More and more digital investigation tasks employ machine learning models to address problems as artificial intelligence advances. However, one issue that frequently arises in digital investigation tasks is the imbalanced data issue, which has a significant impact on how well a machine learning model performs. The common solutions against imbalanced data problems can be improved in various digital investigation scenarios. In this thesis, we discuss imbalanced data problem solutions for three different scenarios in digital investigation. These three scenarios have different data imbalance severities. Corresponding solutions are proposed for improving the performance. The first imbalanced scenario is caused by label shortage for minority class. In some digital investigation tasks, the cost of manually labeling data is high and the speed is slow, so it may cause the data imbalance problem in training data. The task of automated public opinion analysis of social media is discussed as first sample scenario. The location data in the corpus is the minor class, and obtaining the location's high quality training label is challenging. The data with location label is highly imbalanced in the training data. On the basis of a location dictionary, we suggest a semi-supervised learning technique to automatically label more location in the dataset to improve the location label ratio in the dataset, which can decrease the effect of the imbalance problem. The second imbalanced scenario is relative imbalance problem. Relatively imbalance refers to the imbalanced scenario where the samples of the minority class are not scarce. In digital investigation tasks, relative imbalanced scenario is common. Android malware classification is the sample scenario for relative imbalanced task, since malware samples for training are not rare. However, existing solutions to the relative imbalance problem cannot achieve low false positive rate and high malware identification rate at the same time. We demonstrate the imbalance tuning method to better fitting the relative imbalanced scenario of malware classification task. The method trains with balanced data and tunes with the imbalanced ratio in the real world, which could achieve better performance. The third imbalanced scenario is absolute imbalance problem. Absolute imbalance refers to the imbalance scenario where the minority class samples are extremely scarce. In this scenario, unsupervised learning is utilized, as only negative data is available for model training, with no positive data present. In digital investigation, some scenarios are hard to get the malicious data. Anomaly detection on water treatment system is discussed as sample scenario. We propose Traceable Time-series Prediction Anomaly Detection (TTPAD) method which performs unsupervised real-time anomaly detection by time series data prediction. This method addresses the problem of absolute imbalance since it only uses data from the water treatment system's regular operation. In digital investigation, there is no one-size-fits-all solution to the imbalance issues, as the appropriate solution may vary depending on the specific problem. The experiment results show that for different imbalanced scenarios, the corresponding solution we propose can effectively improve the performance.