domeet webmaster

Many of the younger generation are ACG (animation, comics, and games). On social networks, the avatars of anime characters are often seen, because for these users, the common hobby is an effective way for them to identify each other and improve communication.

However, there are two minor problems: First, there are so many popular anime works, and accidents that hit the head often occur. As well as the gender of the owner of the account, the avatar is a cute girl, and the time is long and unrecognizable. Every time I read the post on the Internet, it’s like watching a bunch of anime characters comment on each other…

We all know that neural network-based AI has developed very rapidly in recent years. One of the technologies is called style transfer. Simply put, picture A gets the style of picture B, but it still has obvious The characteristics of A.

With this technology, ACG enthusiasts can also make their avatars anime. Just the effect is still not very satisfactory, it looks like using a brush and color on the face photoStroke only:

This style of migration is actually a bit contrary to the original intention: Many people use anime avatars for cuteness, but the final output does not look cute at all.

However, ACG enthusiasts will not stop!

A group of Korean AI researchers recently published a paper showing their important progress in image-to-image translation.

The effect they achieved seemed to be more like finding a cartoonist, recreating the original photo seriously, and the effect exceeded all existing avatar cartoonization techniques:

The red frame is marked with the original image (a) and the output result (e). The output avatar is not only closer to the image we often see in the animation, but also the recognition of the original image is very high. .

Based on the unsupervised learning approach and the architecture of the Generated Confrontation Network (GAN), the researchers added a new attention module and invented a model called “Adaptive Layer-Instance Normalization” ( AdaLIN)’s normalization mechanism has developed a completely new neural network.

This study was named U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation [1]. It is precisely because of the new attention and refinement designed by researchers. The mechanism, we can see from the output, this neural network has different treatments for different features.

The magnified features, such as the eyes, are magnified; the reduced features, such as the nose and mouth, are also reduced; as for other features, such as hairstyles, hair color,Skin color, even facial shadows, also has a fairly accurate reduction.

The following figure shows the architecture of the generator and discriminator that generates the anti-network:

The researchers pointed out that the previous image style migration results, like the strokes, must map the line background of the original image to the output; and the attention and normalization mechanism they developed can The original image and the target style map are drawn with attention, and then the model is guided to different reconstructions of different regions and features.

In other words, this new model understands that anime avatars must have big eyes, more line-shaped hair, and more vibrant hair color and ochre. It will translate according to these principles (taken from the input reference avatar).

The researchers mentioned in the paper that their model has taken a new step on the basis of simple style migration, with object transfiguration.

The adaptive layer-instance normalization mechanism they invented, AdaLIN, has other advantages, such as adjustability. Researchers can adjust Layers and Instances separately to achieve different outputs. The degree of shape and material changes.

The following figure shows that the third to sixth from the horizontal is the different result of adjusting the normalization mechanism:

In text translation, the output should conform to the syntax of the object language, usage habits, and so on. The same is true in image-to-image translation. You can understand this: the previous style migration was just doing “literal translation”, and U-GAT-IT achieved a breakthrough in “Xinda Ya”.

Another great feature of the technology is the ability to perform accurate image translations of almost any type of photo (animal, pet face, landscape) without any adjustment to the parameters of the neural network:

The first author of the paper is Junho Kim:

It is worth mentioning that the three authors, including Kim, are from the Korean game company NCsoft, one of the “Gorge Shuangxiong” players in the game industry; the other author is from the Boeing Korea Engineering Technology Center.

NCsoft’s established online game “Paradise” (Lineage 1 & 2) was launched in 1998. It has been operating continuously for 21 years in many countries including China, as well as several well-known games such as “Aion”. But little is known, in fact, NCsoft is also one of the most radical companies in Korea for artificial intelligence research and development.

According to the Korea Times Report, NCsoft has established a dedicated research and development team in 2011. It currently operates two independent research institutes, the Artificial Intelligence Center and the Natural Language Processing Center. The former is responsible for game AI, speech recognition and computer vision, the latter is the main languageWord understanding and knowledge systems.

The heads of the two centers directly manage the company’s founder and CEO Jin Zechen. The total number of employees is about 160, and will increase to about 300 this year.

The main purpose of NCsoft’s deep learning is to introduce relevant technologies into games and services. Currently, the company is using AI to detect plug-ins and to customize services based on the player’s gaming habits and interest data.

The technology mentioned in this article will greatly improve the player’s gaming experience – perhaps without pinching your face, you can generate accurate cartoon character characters by uploading an avatar.

Boeing just established an engineering and technology center in South Korea last year. Its main research interests include automation, artificial intelligence, avionics and data analysis. The technical staff of the organization is mainly from well-known universities and technology companies in South Korea.

At the moment, there isn’t a demo available for this technology, but if you are familiar with TensorFlow, you can browse the code on GitHub [2] or read the original text and follow the instructions to run on your computer. ~

[1] U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image TranSlation https://arxiv.org/pdf/1907.10830.pdf

[2] https://github.com/taki0112/UGATIT