|Table of Contents|

Research on Image Coloring Method Based on Theme Palette(PDF)

《南京师大学报（自然科学版）》[ISSN:1001-4616/CN:32-1239/N]

Issue:
2022年03期
Page:
116-122
Research Field:
计算机科学与技术
Publishing date:

Info

Title:
Research on Image Coloring Method Based on Theme Palette
Author(s):
Li Jinfeng1; Pei Wei2; Zhu Yongying3; Lu Mingyu1; Song Lin1
(1.Information Science and Technology College,Dalian Maritime University,Dalian 116026,China)(2.College of Environmental Sciences and Engineering,Dalian Maritime University,Dalian 116026,China)(3.College of Ocean and Civil Engineering,Dalian Ocean University,Dalian 116023,China)
Keywords:
image coloring; image segmentation; theme palette; target palette
PACS:
TP391
DOI:
10.3969/j.issn.1001-4616.2022.03.015
Abstract:
There exist some problems in the image colorizing methods based on theme palette today,such as inaccurate themes,inharmonious colors and biased aesthetic evaluation. In this regard,this paper proposed a set of precise colorizing schemes,which adopts Lasso regression model to extract the theme color from the foreground object segmented by Mask R-CNN,extends the theme color by WGAN_gp,and uses NIMA to quantify the optimal scheme. In the aesthetic evaluation experiment,LPIPS decreased by 37.5%,and NIMA increased by 6.6% after coloring,indicating that the scheme is feasible and effective.

References:

[1]VITORIA P,RAAD L,BALLESTER C. ChromaGAN:adversarial picture colorization with semantic class distribution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision(WACV),Snowmass,CO,USA,2020. Piscataway,NJ:IEEE,2020:2445-2454.
[2]ZHANG L,JI Y,LIN X,et al. Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier GAN[C]//2017 4th IAPR Asian Conference on Pattern Recognition(ACPR),Nanjing,China,2017. Piscataway,NJ:IEEE,2017:506-511.
[3]ANTIC J. A deep learning based project for colorizing and restoring old images(and video!)[EB/OL]. [2021-05-28]. https://github.com/jantic/DeOldify.
[4]YOO S,BAHNG H,CHUNG S,et al. Coloring with limited data:few-shot colorization via memory augmented networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Long Beach,CA,USA,2019. Piscataway,NJ:IEEE,2019:11283-11292.
[5]SU J,CHU H,HUANG J. Instance-aware image colorization[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),Seattle,WA,USA,2020. Piscataway,NJ:IEEE,2020:7968-7977.
[6]贾玉福,胡胜红,刘文平,等. 使用条件生成对抗网络的自然图像增强方法[J]. 南京师大学报(自然科学版),2019,42(3):88-95.
[7]LEVIN A,LISCHINSKI D,WEISS Y. Colorization using optimization[C]//ACM SIGGRAPH 2004,Los Angeles,California,USA,2004. New York,NY:ACM,2004:689-694.
[8]CHANG H,FRIED O,LIU Y,et al. Palette-based photo recoloring[J]. ACM transactions on graphics,2015,34(4):131-139.
[9]TAN J,ECHEVARRIA J,GINGOLD Y. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry[J]. ACM transactions on graphics,2018,37(6):1-10.
[10]HE K,GKIOXARI G,DOLLAR P,et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision(ICCV),Venice,Italy,2017. Piscataway,NJ:IEEE,2017:2961-2969.
[11]REN S,HE K,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. Advances in neural information processing systems,2015,28:91-99.
[12]LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),Boston,USA,2015. Piscataway,NJ:IEEE,2015:3431-3440.
[13]LIN S,HANRAHAN P. Modeling how people extract color themes from images[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems,New York,USA,2013. New York,NY:ACM,2013:3101-3110.
[14]ANWAR S,TAHIR M,LI C,et al. Image colorization:a survey and dataset[EB/OL]. [2020-11-03]. https://arxiv.org/abs/2008.10774.
[15]TALEBI H,MILANFAR P. NIMA:neural image assessment[J]. IEEE transactions on image processing,2018,27(8):3998-4011.
[16]GULRAJANI I,AHMED F,ARJOVSKY M,et al. Improved training of Wasserstein Gans[EB/OL]. [2017-12-25]. https://arxiv.org/abs/1704.00028.
[17]ZHANG R,ISOLA P,EFROS A A,et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//The IEEE Conference on Computer Vision and Pattern Recognition(CVPR),San Diego,USA,2018. Piscataway,NJ:IEEE,2018:586-595.

Memo

Memo:
-
Last Update: 2022-09-15