The advent of artificial intelligence has helped liberate producers’ hands, allowing producers to focus more on creativity.
Editor’s note: This article is from WeChat public account The ability of machines (ID: almosthuman2017), by Xiong Lilang.
Comics and animations are an important way to express stories and ideas, and they are also an important pillar of the Japanese entertainment industry. By designing iterative iterations of drawing techniques, it allows artists to quickly present their ideas on paper or screen. The beginning of the creation is that artists give priority to how to express and record concepts and ideas as soon as possible, so they get rough drafts. Then, from the draft, you start to draw lines, color, even shoot, dub, and iteratively refine iteratively until the final comic or animation work is produced. The process of iterative refinement makes artists have a huge workload. The emergence of artificial intelligence has helped liberate producers’ hands, allowing producers to focus more on creativity.
I. ACG market scale in Japan
According to data provided by the Japan Animation Association in 2018, the Japanese ACG industry will continue to grow. The total market related to ACG was 212.7 billion yen (108.1% of the previous year). In 2017, the market size exceeded 2 trillion yen for the first time in history, the fifth consecutive year. The biggest growth factor comes from overseas markets, which have maintained rapid growth since 2015 (an increase of 129.6 percent over the previous year). It is estimated that works have a large share of sales in China’s major Internet platforms, Amazon and Netflix. Due to the worldwide popularity of animation, the overseas market has grown significantly, and the video distribution business has continued to grow. On the other hand, due to a lack of DVD and merchandise sales, the Japanese domestic market has fallen for the third consecutive year.
Second, artificial intelligence technology in animation
Deep learning: Deep learning analyzes various animation production materials, and automatically generates animations or sketch lines for transitions based on models and data. This technique is mainly used to reduce the time that the original artist or animation producer spends on repetitive work. The addition of deep learning technology can greatly improve the production efficiency of animation production.
Generation Adversarial Networks (GANs): At its core, GANs consist of two agents with competing goals that work through opposing goals. By enabling AI to learn a large number of real face photos and anime character faces, you can create brand new image samples. Different types of GANs can convert photos into two-dimensional style pictures, or reverse operations.
Image recognition technology: Through image recognition, the edges of an image / animation or sketch lines are correctly identified from anime originals. The identified content is used for further processing. Iteratively refine the sketch feedback until the final work is produced. For example, a convolution model can be used with a raster image to convert multiple rough sketch lines into a clean line.
Speech synthesis technology: a branch of natural language processing technology. Speech synthesis technology turns text into sound. Turn text messages into audible voice messages, that is, make virtual characters speak like humans. It mainly simulates human understanding of natural language-text regularization, word segmentation, grammatical analysis and semantic analysis, so that the computer can fully understand the input text, and give a variety of pronunciation prompts required by the latter two parts.
Third, the landing scene of artificial intelligence technology in the field of two-dimensional elements
Fourth, the application case of artificial intelligence technology in the two-dimensional creation
Automatic drafting of comic drafts: The automatic drafting technique of comic drafts developed by Waseda University uses AI to iteratively refine from the initial sketch according to feedback until the final work is produced. Allows creators to prioritize expressing concepts and ideas as quickly as possible, rather than exquisite details. The image is used as the input to the network, and the output is the same size as the input image. The network structure is very simple. The first part is used as an encoder to compress the image in space. The second part is to process and extract the basic lines in the image. The last part is a decoder that converts small or simpler features into a grayscale image with the same resolution as the input. This is all done with convolutions.
Automatic animation frame generation: Dwango’s technical team leveraged early 2016