Learning local and global context from sequence and matrix inputs

Abstract

Learning local and global context from sequence and matrix inputs plays an extremely significate role for bioinformatics and computer vision problems. By taking advantage of big data and appealing data-driven methods, in this thesis, we propose novel pipelines for learning local, global context and integrated local-global context.

At first, a novel deep learning pipeline is proposed for protein secondary structure prediction. Specifically, we propose an end-to-end deep network that predicts protein secondary structures from integrated local and global contextual features. Our deep architecture leverages convolutional neural networks with different kernel sizes to extract multiscale local contextual features. In addition, considering long-range dependencies existing in amino acid sequences, we set up a bidirectional neural network consisting of gated recurrent unit to capture global contextual features. Furthermore, multi-task learning is utilized to predict secondary structure labels and amino-acid solvent accessibility simultaneously. Our proposed deep network demonstrates its effectiveness by achieving state-of-the-art performance.

Inspired by the success of previous sequence context learning, a new proposed base-caller, WaveNano, are presented to improve the Oxford MinION nanopore basecalling. We further show that the indel (insertions and deletions, mainly cause the high error rate) issue can be significantly reduced via accurate labeling of nucleotide and move labels directly from the raw signal. Our bi-directional WaveNet model with residual blocks and skip connections is able to capture the extremely long dependency in the raw sequential signal. Taking the predicted move as the segmentation guidance, we employ the Viterbi decoding to obtain the final basecalling results from the smoothed nucleotide probability matrix. Our proposed base-caller, WaveNano, achieves state-of-the-art performance on real MinION sequencing data from Lambda phage.

Though protein contacts contain key information for protein structure understanding, the predicted contacts based on existing methods learning context form matrix inputs are still of low quality, especially for membrane proteins (MPs) with lack of sufficient solved structures. A low-cost, high-throughput deep transfer learning method is proposed to first predict MP contacts by learning from non-membrane proteins (non-MPs) using integrated local and global context from amino acid sequential and matrix co-evolutional features, and then predict 3D structure models using predicted contacts as distance restraints. Tested on 510 non-redundant MPs, our method has much better contact prediction accuracy than existing ones. A rigorous blind test in CAMEO and human multi-pass MPs test verify the priority of our method.

Finally, we address the RGB-D scene labeling problem, which generates pixel-wise and fine-grained label maps from simultaneously sensed photometric (RGB) and depth channels. Our proposed pipeline solves this problem by i) developing a novel Long Short-Term Memorized Context Fusion (LSTM-CF) Model that captures and fuses contextual information from multiple channels of photometric and depth data, and ii) incorporating this model into deep convolutional neural networks (CNNs) for end-to-end training. At last, the fused contextual representation is concatenated with the local convolutional features extracted from the photometric channels in order to improve the accuracy of fine-scale semantic labeling. Our proposed model has set a new state-of-the-art on three main datasets.