Application of metaheuristics and machine learning techniques in computational finance

Abstract

Computational finance (CF) applies advanced computational methods such as computational intelligence to various practical problems in finance. Currently, the development and application of appropriate computational techniques for the financial market have been still insufficient due to the much complex nature of the market. In recent years, metaheuristics and machine learning (ML) techniques have attracted extensive attention in CF. This thesis aims to investigate both techniques for a few important problems including portfolio optimization, market prediction and devising trading strategies in CF.

First, although some metaheuristics exist, due to the nature of the market, the determination of the attributes of many financial optimization problems is difficult. Thus, selecting the most appropriate metaheuristic to solve each financial problem is challenging. This motivates us to propose the Adaptive Multi-Population Optimization (AMPO) algorithm with a higher level of adaptivity and effectiveness for tackling various optimization problems in CF. The AMPO hybridizes yet modifies several useful designs like mutation and memory retention from evolutionary algorithms and swarm intelligence approaches in a multi-population manner. Moreover, the diverse control on multiple populations, solution cloning mechanism and reset operation are designed. Compared with other metaheuristics, the AMPO can attain an adaptive balance between the capabilities of exploration and exploitation. The algorithm is critically evaluated on a series of well-known continuous benchmark problems and portfolio optimization problems. The experimental results demonstrate that the AMPO achieves a better performance over some selected and state-of-the-art metaheuristics including IEEE CEC winning algorithms in such problems.

Second, stock market prediction and devising profitable trading strategies are significant yet considerably challenging. For market predictions, a real-time wavelet denoising approach combined with various ML approaches, namely the Long Short-Term Memory (LSTM) neural network and Support Vector Machine (SVM), is proposed to predict stock prices. The empirical results show that the proposed models avoid the look-ahead bias issue that exists in previous studies, and they also produce significant improvements in the prediction compared to the original ML models. In addition, a creative idea as supported by an interesting financial phenomenon is provided to develop trading strategies by using ML techniques for forecasting momentum and reversal effects in the market. The results reveal that it is possible to devise effective trading strategies in this way.

Last, regarding the market prediction and momentum-reversal trading models, selecting hyper-parameters of the algorithms and input features dramatically affects their ultimate performance. Therefore, a methodology that integrates both metaheuristics and ML techniques is proposed to overcome these issues. For such SVM-based market prediction and trading models, a metaheuristic optimizer is applied for both hyper-parameter tuning and feature selection. The results indicate that the proposed integrative frameworks outperform the existing methods as well as other advanced Deep Learning methods such as the Bidirectional LSTM in terms of prediction errors and profitability. In essence, the AMPO as the proposed optimizer shows its excellent optimization capability on these problems over other optimization algorithms for comparison.