Accelerating machine learning applications with FPGAs

Abstract

As the demand for a more power efficient device to handle machine learning tasks rises, FPGAs serve as a platform with great potential, due to their flexibility to adapt to novel algorithms and designs. However, FPGA users face many unique difficulties when implementing such algorithms, for instances the lack of algorithmic libraries, and the lack of custom-precision math operator libraries, or the lack of reusable common interfaces for host-software integration. In this thesis, we start by presenting a case study on accelerating the Support Vector Machine (SVM) training on an Apache Spark cluster equipped with FPGAs, demonstrating a consistent speed-up of about 1.6x against CPUs as the cluster size increases up to 8-nodes. Then, to address the problem of the lack of math operators, we propose an open source function generator, NnCore, for floating-point non-linear operator cores built using fixed-point piecewise polynomial segments. The proposed work takes advantage of properties such as oddness/evenness and intercept-at-origin, often found in numerical functions commonly used in machine learning applications, and applies an improved segmentation algorithm that specifically handles ``outlier'' segments to reduce memory size. Experiments show that at single-precision, generated cores use up to 65% fewer BRAMs and runs at up to 2.2x the clock speed, compared with cores generated from a previous generic function generator. At half-precision, cores can run at 1.2x higher clock speed while requiring more resource, or use a comparable number of resource but run at 12% to 45% lower clock speed. At last, we present hDNN, a software-hardware integrated research platform for deep learning, which can serve as both the building blocks for further research, or as a baseline comparison target for benchmarks. It consists of a collection of hardware IP modules, written in HLS C++ for the Xilinx SDAccel platform, and a modified Caffe that enables support of quantized arithmetic in individual layers with user-defined quantization scheme, that can target to run on CPUs/GPUs/FPGAs. The hardware designs include a 32x32 systolic array that runs at 200 MHz for 16-bit integer/fixed-point types on a Virtex-7 FPGA, which translates to a 409.6 GOPS theoretical throughput. Together with an ``Im2col'' module provided, convolution operations can also be achieved. Experiments show that speed-up in the range of 1.8x to 32.7x is achieved against an optimized CPU implementation, for the matrix multiplication part of convolution layers in networks like Lenet, Cifar10 and Caffenet.