[1]吴水淼,陈 强.基于Transformer和CNN的双流网络眼底图像可变形配准方法[J].南京师大学报(自然科学版),2025,48(04):118-127.[doi:10.3969/j.issn.1001-4616.2025.04.012]
　Wu Shuimiao,Chen Qiang.Deformable Registration Method for Retinal Images Based on a Dual-Stream Network of Transformer and CNN[J].Journal of Nanjing Normal University(Natural Science Edition),2025,48(04):118-127.[doi:10.3969/j.issn.1001-4616.2025.04.012]
点击复制

基于Transformer和CNN的双流网络眼底图像可变形配准方法()

《南京师大学报（自然科学版）》[ISSN:1001-4616/CN:32-1239/N]

卷:
48
期数:
2025年04期
页码:
118-127
栏目:
计算机科学与技术
出版日期:
2025-08-20

文章信息/Info

Title:
Deformable Registration Method for Retinal Images Based on a Dual-Stream Network of Transformer and CNN
文章编号:
1001-4616(2025)04-0118-10
作者:
吴水淼; 陈 强
(南京理工大学计算机科学与工程学院,江苏 南京 210094)
Author(s):
Wu Shuimiao; Chen Qiang
(School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China)
关键词:
眼底图像配准; 无监督学习; transformer; CNN
Keywords:
retinal image registration; unsupervised learning; transformer; CNN
分类号:
TP391
DOI:
10.3969/j.issn.1001-4616.2025.04.012
文献标志码:
A
摘要:
视网膜图像配准有助于医生全面了解视网膜结构,但由于缺乏带真实标签的数据,增加了配准难度. 传统配准方法效率低,且依赖固定模型和手工设计特征,难以处理复杂变形,而深度学习方法尽管高效,但多为单流结构,该结构会对特征融合产生干扰. 针对现有方法在特征提取上的不足,本文提出一种双流网络,通过Transformer和CNN分别提取全局和局部特征,在多个尺度上进行特征匹配,并引入血管信息辅助训练. 实验结果表明,该方法在彩色眼底数据集上显著提升了配准精度,验证了其在可变形医学图像配准中的有效性.
Abstract:
Retinal image registration helps doctors to comprehensively understand the structure of the retina,but the lack of data with real labels increases the difficulty of registration. Traditional registration methods are inefficient and rely on fixed models and manually designed features,making it difficult to handle complex deformations,while deep learning methods,although efficient,are mostly single stream structures that can interfere with feature fusion. In response to the shortcomings of existing methods in feature extraction,this paper proposes a dual stream network that extracts global and local features through Transformer and CNN respectively,performs feature matching at multiple scales,and introduces vascular information to assist in training. The experimental results show that this method significantly improves the registration accuracy on color fundus datasets,verifying its effectiveness in deformable medical image registration.

参考文献/References:

[1]LIM L S,MITCHELL P,SEDDON J M,et al. Age-related macular degeneration[J]. The lancet,2012,379(9827):1728-1738.
[2]VASWANI A,SHAZEER N,PARMAR N,et al. Attention is all you need[J]. Advances in neural information processing systems,2017,30:5998-6008.
[3]石磊,籍庆余,陈清威,等. 视觉Transformer在医学图像分析中的应用研究综述[J]. 计算机工程与应用,2023,59(8):41-55. DOI:10.3778/j.issn.1002-8331.2206-0022.
[4]LOWEKAMP B C,CHEN D T,IBÁÑEZ L,et al. The design of SimpleITK[J]. Frontiers in neuroinformatics,2013,7:45.
[5]BYRD R H,LU P,NOCEDAL J,et al. A limited memory algorithm for bound constrained optimization[J]. SIAM journal on scientific computing,1995,16(5):1190-1208.
[6]GUTIERREZ-BECKER B,MATEUS D,PETER L,et al. Guiding multimodal registration with learned optimization updates[J]. Medical image analysis,2017,41:2-17.
[7]OTIRAS A,DAVATZIKOS C,PARAGIOS N. Deformable medical image registration:A survey[J]. IEEE transactions on medical imaging,2013,32(7):1153-1190.
[8]BALAKRISHNAN G,ZHAO A,SABUNCU M R,et al. Voxelmorph:a learning framework for deformable medical image registration[J]. IEEE transactions on medical imaging,2019,38(8):1788-1800.
[9]JADERBERG M,SIMONYAN K,ZISSERMAN A. Spatial transformer networks[J]. Advances in neural information processing systems,2015,28:2017-2025.
[10]CHEN J,FREY E C,HE Y,et al. Transmorph:Transformer for unsupervised medical image registration[J]. Medical image analysis,2022,82:102615.
[11]LIU Z,LIN Y,CAO Y,et al. Swin transformer:Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal,QC,Canada,2021:10012-10022.
[12]CHEN Z,ZHENG Y,GEE J C. Transmatch:A transformer-based multilevel dual-stream feature matching network for unsupervised deformable image registration[J]. IEEE transactions on medical imaging,2024,43(1):15-27.
[13]SUN J,SHEN Z,WANG Y,et al. LoFTR:Detector-free local feature matching with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville,TN,USA:2021:8922-8931.
[14]HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas,NV,USA:2016:770-778.
[15]DICE L R. Measures of the amount of ecologic association between species[J]. Ecology,1945,26(3):297-302.
[16]HERNANDEZ-MATAS C,ZABULIS X,TRIANTAFYLLOU A,et al. FIRE:fundus image registration dataset[J]. Modeling and artificial intelligence in ophthalmology,2017,1(4):16-28.
[17]吴玲玉,蓝洋,夏海英. 基于卷积神经网络的眼底图像配准研究[J]. 广西师范大学学报(自然科学版),2021,39(5):122-133.
[18]RONNEBERGER O,FISCHER P,BROX T. U-net:Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015:18th International Conference. Munich,Germany:Springer International Publishing,2015:234-241.
[19]KINGMA D P,BA J. Adam:A method for stochastic optimization[J]. arxiv preprint arxiv:1412.6980,2014.
[20]WANG Z,BOVIK A C,SHEIKH H R,et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE transactions on image processing,2004,13(4):600-612.

相似文献/References:

[1]张 帅,谢志华,牛杰一,等.基于对抗判别域适应的近红外与可见光异质人脸识别[J].南京师大学报(自然科学版),2020,43(04):95.[doi:10.3969/j.issn.1001-4616.2020.04.014]
　Zhang Shuai,Xie Zhihua,Niu Jieyi,et al.Near-infrared and Visible-light Image Heterogeneous Face RecognitionBased on Adversarial Domain Adaptation Learning[J].Journal of Nanjing Normal University(Natural Science Edition),2020,43(04):95.[doi:10.3969/j.issn.1001-4616.2020.04.014]

备注/Memo

备注/Memo:
收稿日期:2024-09-24.
基金项目:国家自然科学基金项目(92370109、6217223)、中央高校基本科研基金项目(30921013105).
通讯作者:陈强,博士,教授,研究方向:图像分析与机器学习. E-mail:chen2qiang@njust.edu.cn
更新日期/Last Update: 2025-08-20