[1]李锦峰,裴 伟,朱永英,等.基于主题色板的图像上色方法研究[J].南京师大学报(自然科学版),2022,45(03):116-122.[doi:10.3969/j.issn.1001-4616.2022.03.015]
 Li Jinfeng,Pei Wei,Zhu Yongying,et al.Research on Image Coloring Method Based on Theme Palette[J].Journal of Nanjing Normal University(Natural Science Edition),2022,45(03):116-122.[doi:10.3969/j.issn.1001-4616.2022.03.015]
点击复制

基于主题色板的图像上色方法研究()
分享到:

《南京师大学报(自然科学版)》[ISSN:1001-4616/CN:32-1239/N]

卷:
第45卷
期数:
2022年03期
页码:
116-122
栏目:
计算机科学与技术
出版日期:
2022-09-15

文章信息/Info

Title:
Research on Image Coloring Method Based on Theme Palette
文章编号:
1001-4616(2022)03-0116-07
作者:
李锦峰1裴 伟2朱永英3鲁明羽1宋 琳1
(1.大连海事大学信息科学技术学院,辽宁 大连 116026)(2.大连海事大学环境科学与工程学院,辽宁 大连 116026)(3.大连海洋大学海洋与土木工程学院,辽宁 大连 116023)
Author(s):
Li Jinfeng1Pei Wei2Zhu Yongying3Lu Mingyu1Song Lin1
(1.Information Science and Technology College,Dalian Maritime University,Dalian 116026,China)(2.College of Environmental Sciences and Engineering,Dalian Maritime University,Dalian 116026,China)(3.College of Ocean and Civil Engineering,Dalian Ocean University,Dalian 116023,China)
关键词:
图像上色图像分割主题色板目标色板
Keywords:
image coloringimage segmentationtheme palettetarget palette
分类号:
TP391
DOI:
10.3969/j.issn.1001-4616.2022.03.015
文献标志码:
A
摘要:
现有基于主题色板的图像上色方法存在着主题不准确、色彩不和谐、美感评价不客观等问题. 鉴于此,本文提出了一套上色解决方案,用Lasso回归模型对Mask R-CNN分割的前景目标提取主题色、WGAN_gp对主题色扩展、NIMA对主题上色方案评价. 在美感评价实验中,采用本文方案上色后LPIPS降低了37.5%,NIMA提高了6.6%,表明该方案可行有效.
Abstract:
There exist some problems in the image colorizing methods based on theme palette today,such as inaccurate themes,inharmonious colors and biased aesthetic evaluation. In this regard,this paper proposed a set of precise colorizing schemes,which adopts Lasso regression model to extract the theme color from the foreground object segmented by Mask R-CNN,extends the theme color by WGAN_gp,and uses NIMA to quantify the optimal scheme. In the aesthetic evaluation experiment,LPIPS decreased by 37.5%,and NIMA increased by 6.6% after coloring,indicating that the scheme is feasible and effective.

参考文献/References:

[1]VITORIA P,RAAD L,BALLESTER C. ChromaGAN:adversarial picture colorization with semantic class distribution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV),Snowmass,CO,USA,2020. Piscataway,NJ:IEEE,2020:2445-2454.
[2]ZHANG L,JI Y,LIN X,et al. Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier GAN[C]//2017 4th IAPR Asian Conference on Pattern Recognition(ACPR),Nanjing,China,2017. Piscataway,NJ:IEEE,2017:506-511.
[3]ANTIC J. A deep learning based project for colorizing and restoring old images(and video!)[EB/OL]. [2021-05-28]. https://github.com/jantic/DeOldify.
[4]YOO S,BAHNG H,CHUNG S,et al. Coloring with limited data:few-shot colorization via memory augmented networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Long Beach,CA,USA,2019. Piscataway,NJ:IEEE,2019:11283-11292.
[5]SU J,CHU H,HUANG J. Instance-aware image colorization[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Seattle,WA,USA,2020. Piscataway,NJ:IEEE,2020:7968-7977.
[6]贾玉福,胡胜红,刘文平,等. 使用条件生成对抗网络的自然图像增强方法[J]. 南京师大学报(自然科学版),2019,42(3):88-95.
[7]LEVIN A,LISCHINSKI D,WEISS Y. Colorization using optimization[C]//ACM SIGGRAPH 2004,Los Angeles,California,USA,2004. New York,NY:ACM,2004:689-694.
[8]CHANG H,FRIED O,LIU Y,et al. Palette-based photo recoloring[J]. ACM transactions on graphics,2015,34(4):131-139.
[9]TAN J,ECHEVARRIA J,GINGOLD Y. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry[J]. ACM transactions on graphics,2018,37(6):1-10.
[10]HE K,GKIOXARI G,DOLLAR P,et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV),Venice,Italy,2017. Piscataway,NJ:IEEE,2017:2961-2969.
[11]REN S,HE K,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems,2015,28:91-99.
[12]LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Boston,USA,2015. Piscataway,NJ:IEEE,2015:3431-3440.
[13]LIN S,HANRAHAN P. Modeling how people extract color themes from images[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems,New York,USA,2013. New York,NY:ACM,2013:3101-3110.
[14]ANWAR S,TAHIR M,LI C,et al. Image colorization:a survey and dataset[EB/OL]. [2020-11-03]. https://arxiv.org/abs/2008.10774.
[15]TALEBI H,MILANFAR P. NIMA:neural image assessment[J]. IEEE transactions on image processing,2018,27(8):3998-4011.
[16]GULRAJANI I,AHMED F,ARJOVSKY M,et al. Improved training of Wasserstein Gans[EB/OL]. [2017-12-25]. https://arxiv.org/abs/1704.00028.
[17]ZHANG R,ISOLA P,EFROS A A,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),San Diego,USA,2018. Piscataway,NJ:IEEE,2018:586-595.

相似文献/References:

[1]康家银,张文娟.用于图像分割的非局部空间约束的核FCM算法[J].南京师大学报(自然科学版),2019,42(03):122.[doi:10.3969/j.issn.1001-4616.2019.03.016]
 Kang Jiayin,Zhang Wenjuan.Kernelized FCM Algorithm with Non-Local SpatialConstraint for Image Segmentation[J].Journal of Nanjing Normal University(Natural Science Edition),2019,42(03):122.[doi:10.3969/j.issn.1001-4616.2019.03.016]
[2]班楷第,孙 宇,韩悦,等.乳腺癌超声图像的腋窝淋巴结特征指导分割网络[J].南京师大学报(自然科学版),2023,46(02):92.[doi:10.3969/j.issn.1001-4616.2023.02.012]
 Ban Kaidi,Sun Yu,Han Yue,et al.Features Guidance Axillary Lymph Nodes Network for Breast Cancer Segmentation[J].Journal of Nanjing Normal University(Natural Science Edition),2023,46(03):92.[doi:10.3969/j.issn.1001-4616.2023.02.012]

备注/Memo

备注/Memo:
收稿日期:2021-06-01.
基金项目:国家自然科学基金项目(61001158、 61272369、 61370070)、辽宁省自然科学基金项目(2014025003)、 辽宁省教育厅科学研究一般项目(L2012270)、大连市科技创新基金项目(2018J12GX043).
通讯作者:裴伟,博士,副教授,研究方向:模式识别、人工智能. E-mail:peiwei@dlmu.edu.cn
更新日期/Last Update: 2022-09-15