This article is from WeChat official account:Big Data Digest (ID: BigDataDigest), the original title “AI Generates Chinese Landscape Painting! Princeton girl graduate work, the line stroke cheat majority of human observers, “the authors: abstract bacteria, title figure from: Vision China

It seems that creating with GAN is nothing new.

In 2019, NVIDIA launched an artificial intelligence image generator “GauGAN” at the GTC conference. Users only need to simply outline a few lines and it will automatically generate beautiful scenery pictures.

The technology used by this AI is to generate a confrontation network (GAN), which is also a deep learning model, which is now widely used in image generation .

Including AI Portraits Ars, which was jointly released by MIT and IBM Watson Joint Laboratory last year. Users can transform their photos into a medieval optimized style online. This online tool was once so popular that the website went down.

You might say, “Isn’t this style transfer?”

No, the team members specifically emphasized that this is not a style transfer. It is created by AI itself. From lines to tones, they are all created by human artists like real people.

However, just like the huge cultural gap between the East and the West, in the field of combining art and technology, AI seems to be more biased towards the West. We have seen many AI generate works of realism, postmodernism, and even abstraction, but It is rare to see the performance of AI in traditional oriental art.

Finally, Alice Xue, an undergraduate student at Princeton University, set his sights on Chinese landscape painting.

In her graduation thesis, she developed an AI model called SAPGAN(Sketch-And-Paint GAN). The model can generate traditional Chinese landscape paintings, for which she also won the Princeton 2020 Excellent Graduation Thesis Award.


Link to the paper: https://arxiv.org/pdf/2011.05552.pdf

The paper mentioned that a Turing vision test study of 242 people showed that the paintings created by SAPGAN were mistaken for human artworks as high as 55%, which was significantly higher than the paintings created by the baseline GAN model.

Like human beings, first draw a sketch and then color

In the drawing process of traditional Chinese landscape painting, there are usually steps such as hooking, sculpting, dotting, and dyeing. As the name suggests, the outline is drawn first before rendering.

The end-to-end generation of unconditional input model for Chinese landscape paintings proposed by AIice also follows this step. In order to realize this process, AIice has constructed two models:

  • Stage I: SketchGAN

  • Stage II: PaintGAN

SketchGAN collects high-resolution edge maps from sample images, and PaintGAN is based on SketchGAN for “translation” creation, thereby generating a complete landscape painting.

Chinese people are more prone to misjudgment

The results of the experiment are also amazing.

At the time of the final evaluation, more than half of the paintings generated by the model (55%) of the 242 participants were mistaken for humans works.

The score distribution of the Visual Turing Test requires participants to determine whether the artwork is made by humans or computers.(Average = 70.5%)

In terms of “aesthetic pleasure”, “artistic composition”, “clearness” and “creativity”, the SAPGAN model consistently scored above the baseline in all art categories. The biggest difference between SAPGAN and human painting is “clearness”.

It is incredible that Chinese people may be more easily deceived by SAPGAN. As a native speaker of Chinese, people have seen several landscape paintings, but when judging whether a landscape painting is made by SAPGAN, the Chinese may be more likely to be deceived.

The author compared the results of participants whose mother tongue is Chinese and English to see if cultural contact can enable Chinese participants to correctly judge these paintings. However, the average score of Chinese-speaking participants was 49.2%, which was significantly lower than the 73.5% of English-speaking candidates.

In other words, 70% of Chinese speakers mistake SAPGAN’s paintings for people, and the overall level is 55%. Obviously, regardless of their familiarity with Chinese culture, it is difficult for participants to distinguish the source of the painting.

I collected more than two thousand landscape painting data by myself and made it public on GitHub

The model proposed in the article is trained on a new dataset of traditional Chinese landscape paintings. This dataset is not from Baidu or Google, but collected by the author himself.

AIice stated that the current landscape painting datasets have problems of non-uniqueness and insufficient image quality and quantity. In order to promote the development of this field, Alice herself has established a new dataset composed of 2192 high-quality traditional Chinese landscape paintings. The landscape painting comes from the collection of the Princeton Museum of Art.

At present, these valuable paintings have not been touched by generative creation research to a large extent, and the author has also released this data set on GitHub for public use.

Alice said in an interview with the school that the Princeton University Art Museum has an amazing open digital collection of Chinese paintings, which is very valuable to my data set, but unfortunately, most researchers did not make full use of it .

Data set link https://github.com/Alice x 2020/Chinese-landscape painting- Dataset

I have never taken a machine learning class before writing the paper, I am going to work on Facebook

Seeing this, you may think that Alice is a “senior programmer”. But she said, “I never took a machine learning course when I wrote this paper, so I was often overwhelmed by the question: What can a novice like me do for existing innovative research. But I find that there is always an interesting angle to deal with a problem, because a person’s interests and skills are unique to them.”

Speaking of advice to other students, Alice said that integrating digital humanities into your work is a natural thing. Find what you are interested in-whether it is 19th-century literature or jazz-there is always a way to collect data from it to analyze or produce technical tools related to it.

Speaking of her future plans, Alice said that she is going to work at Facebook and become a software engineer.


This article is from WeChat official account:Big Data Digest (ID: BigDataDigest), author : Abstract bacteria