|Table of Contents|

Attention-Guided Multimodal Cardiac Segmentation(PDF)

《南京师大学报（自然科学版）》[ISSN:1001-4616/CN:32-1239/N]

Issue:
2019年03期
Page:
27-31
Research Field:
·全国机器学习会议论文专栏·
Publishing date:

Info

Title:
Attention-Guided Multimodal Cardiac Segmentation
Author(s):
Yang Wanqi1; Zhou Ziqi1; Guo Xinna1
School of Computer Science and Technology,Nanjing Normal University,Nanjing 210023,China
Keywords:
attention; multimodal cardiac segmentation; semi-siamese network; cross-modal image generation
PACS:
TP391.4
DOI:
10.3969/j.issn.1001-4616.2019.03.004
Abstract:
With the goal of leveraging the modal-shareable and modal-specific information during cross-modal segmentation,we propose a novel cross-modal attention-guided semi-Siamese network for joint cardiac segmentation from MR and CT images. In particular,we first employed the cycle-consistency generative adversarial networks to complete the bidirectional image generation(i.e.,MR to CT,CT to MR)to help reduce the modal-level inconsistency. Then,with the generated and original MR and CT images,a novel semi-Siamese network is utilized where 1)two encoders learn modal-specific features separately and 2)a common decoder makes full use of modal-shareable information from different modalities for a final consistent segmentation. Also,we implement the cross-modal attention to incorporate these shareable and specific information,and our model can be trained in an end-to-end manner. With extensive evaluation on the unpaired CT and MR cardiac images,our method outperforms the baselines in terms of the segmentation performance.

References:

[1] Benjamin E J,Muntner P,Alonso A,et al. Heart disease and stroke statistics 2019 update:a report from the American heart association[J]. Circulation,2019,139(10):56-528.
[2]Ruan Y,Guo Y,Zheng Y,et al. Cardiovascular disease(CVD)and associated risk factors among older adults in six low-and middle-income countries:results from SAGE wave 1[J]. BMC public health,2018,18(1):778.
[3]Zhang Z,Xie Y,Xing F,et al. MDNet:a semantically and visually interpretable medical image diagnosis network[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),IEEE,Honolulu,HI,2017:3549-3557.
[4]Zhu J,Park T,Isola P,et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//IEEE International Conference on Computer Vision(ICCV),IEEE,Venice,2017:2242-2251.
[5]Zagoruyko S,Komodakis N. Paying more attention to attention:improving the performance of convolutional neural networks via attention transfer[J]. arXiv preprint arXiv:1612.03928,2016.
[6]Zhuang X,Shen J. Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI[J]. Medical image analysis,2016,31:77-87.
[7]Dou Q,Ouyang C,Chen C,et al. Unsupervised cross-modality domain adaptation of ConvNets for biomedical image segmentations with adversarial loss[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. AAAI Press,Stockholm,2018:691-697.
[8]Zhang Z,Yang L,Zheng Y. Translating and segmenting multimodal medical volumes with cycle-and shape-consistency generative adversarial network[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),IEEE,Salt Lake City,UT,2018:9242-9251.
[9]Ronneberger O,Fischer P,Brox T. U-Net:convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer Assisted Intervention(MICCAI),Springer,Cham,Munich,2015:234-241.
[10]Hong S,Oh J,Han B,et al. Learning transferrable knowledge for semantic segmentation with deep convolutional neural network[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),IEEE,Las Vegas,NV,2016:3204-3212.
[11]Kingma D P,Ba J. Adam:a method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980,2014.
[12]http://www.itksnap.org/pmwiki/pmwiki.php.
[13]Shelhamer E,Long J,Darrell T. Fully convolutional networks for semantic segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR),IEEE,Boston,MA,2015:3431-3440.

Memo

Memo:
-
Last Update: 2019-09-30