A novel full-convolution UNet-transformer for medical image segmentation

Abstract

The Transformer-based methods are still unable to effectively model local contexts although they make up for the deficiency of remote information dependencies for approaches based on small kernel CNNs. To overcome such a shortage, this paper proposes a novel full-convolution UNet Transformer model, FC-UNETTR, for medical image segmentation. First, a novel global–local attention module is proposed by utilizing multiple small kernels of different sizes for depth-wise convolutions to expand the receptive field of the network model, increase the remote dependence of semantic information in the encoder stage, and also improve the feature extraction capability of the network for fuzzy edges. Then, a reparametrized feedforward network is developed to further improve the local information extraction and mitigate the coupling between feature maps such that the relationship between feature map channels can be better revealed. Furthermore, the skip connection and decoder are redesigned by constructing a dense multiscale module instead of traditional ResNet modules to mitigate semantic bias. Benefiting from the above improvements, the constructed FC-UNETTR without pre-training demonstrates a strong capability to extract local features and capture long-range dependencies of images in medical image segmentation. Experiments show that FC-UNETTR achieves an excellent performance of 85.67% for DSC and 7.82 for HD metrics on the Synapse dataset with fewer model parameters compared with state-of-the-art networks. Furthermore, DSC reaches 92.46% and 94.76% on the ACDC dataset and the private dataset of oral graft bone, respectively, outperforming some of the latest medical image segmentation models.