Frontiers of Information Technology & Electronic Engineering
>> 2019,
Volume 20,
Issue 12
doi:
10.1631/FITEE.1900386
Orginal Article
SmartPaint: a co-creative drawing system based on generative adversarial networks
1. Key Laboratory of Design Intelligence and Digital Creativity of Zhejiang Province, Hangzhou 310027, China
2. State Key Lab of CAD&CG, Zhejiang University, Hangzhou 310058, China
3. Alibaba-Zhejiang University Joint Institute of Frontier Technologies, Hangzhou 310027, China |
Available online: 2020-03-05
Next
Previous
Abstract
Artificial intelligence (AI) has played a significant role in imitating and producing large-scale designs such as e-commerce banners. However, it is less successful at creative and collaborative design outputs. Most humans express their ideas as rough sketches, and lack the professional skills to complete pleasing paintings. Existing AI approaches have failed to convert varied user sketches into artistically beautiful paintings while preserving their semantic concepts. To bridge this gap, we have developed SmartPaint, a co-creative drawing system based on generative adversarial networks (GANs), enabling a machine and a human being to collaborate in cartoon landscape painting. SmartPaint trains a GAN using triples of cartoon images, their corresponding semantic label maps, and edge detection maps. The machine can then simultaneously understand the cartoon style and semantics, along with the spatial relationships among the objects in the landscape images. The trained system receives a sketch as a semantic label map input, and automatically synthesizes its edge map for stable handling of varied sketches. It then outputs a creative and fine painting with the appropriate style corresponding to the human’s sketch. Experiments confirmed that the proposed SmartPaint system successfully generates high-quality cartoon paintings.
京公网安备 11010502051620号
