Zhejiang Lab

What We Are Innovating ②: Mo Ran, An AI Painter Specializing in Traditional Chinese Painting

Is it possible to replicate human creativity? At Zhejiang Lab, there is a mysterious “artist” who can create a landscape scroll in milliseconds. It has a very poetic name in Chinese – Mo Ran (ink wash), which is taken from the conception of traditional Chinese painting. Born in October 2019, this “painter” is an artificial intelligent (AI) system with superb learning ability and creativity. On October 20, its first solo exhibition was unveiled at the China (Hangzhou) International Intelligent Products Expo, including 20 paintings of different types, such as traditional Chinese paintings and oil paintings.

With only about two days of learning from sample paintings, Mo Ran can create new paintings in an instant that should be calculated in milliseconds. How does it accurately learn and grasp different painting styles? How does it achieve the ability to complete a painting in milliseconds? This short video gives us a brief explanation.

Mo Ran is an AI system that focuses on traditional Chinese landscape paintings, especially long landscape scrolls. As demonstrated in the video, with equally sized images taken from the paintings of two different artists, an operator simply needs to draw a few lines in the blank space between the patches. Mo Ran can automatically complete the painting and create a new landscape scroll. It can also create a painting on its own, without the lines drawn by the operator. It is worth mentioning that the layout and strokes painted by Mo Ran can naturally fit in with the images on both sides, to form a vivid and smooth painting full of the visual effects produced by ink washing.

“Mo Ran mainly relies on deep neural networks and generative adversarial models for continuous learning and creation,” says Professor Tang Yongchuan, a leader of the Creative Intelligence Team at Zhejiang Lab’s Research Center for AI Algorithms and Platforms. In a recent Zhejiang Lab Seminar, he introduced the core technologies used in Mo Ran. He said that every painting created by Mo Ran needs the support of four key technologies, which include scroll synthesis, font design, super-resolution, and style transfer.

Scroll synthesis aims to imitate the three steps of Chinese landscape painting: drawing the key lines, adding shades and textures, and applying colors with light ink. Due to the constraints posed by loss functions designed for boundary reconstruction, edge reconstruction, and style consistency, Mo Ran can perform unsupervised training to generate complete paintings. In addition, there has been a tradition of adding an inscription when a Chinese landscape painting is completed. Similarly, through a deep neural network, Mo Ran can learn various fonts and text effects. By analyzing the structures and effects of different fonts, it can transform the text in a painting into an inscription with any style that the operator wants. Super-resolution is used to solve the problem of low resolution. Currently, Mo Ran can increase the resolution of paintings to 4K and even 8K. Style transfer means integrating the styles of other paintings into AI systems’ creative work through feature extraction and fusion. The combination of these four key technologies makes Mo Ran possible.

“The primary goal of Mo Ran is to promote traditional Chinese culture and achieve innovative digital outcomes for Chinese culture and art,” said Professor Tang. He pointed out that the deep learning system of Mo Ran can help non-professionals to participate in the creation of paintings and works of calligraphy in diversified ways, narrowing the distance between art and life. In the future, Mo Ran will also play an important role in the restoration of calligraphy and painting relics. Recently, the research team for Moran at Zhejiang Lab has tried to restore a damaged scroll called “Dwelling in the Fuchun Mountains”. “Our current AI technology for painting restoration relies on big data-based deep learning algorithms. Different data will lead to different results,” says Professor Tang. “AI is not capable of 100% restoration of damaged paintings, but it can provide a variety of repair resolutions. In the future, the restoration of calligraphy and painting relics will be achieved through the collaboration of people and machines. Only with experts’ experience and AI systems’ analysis can we further restore the missing parts of our traditional culture.”

Another goal of Mo Ran is to create digital creative software with Chinese features. “In the future, AI will provide infinite possibilities for artistic creation. It will not only be used for landscape paintings, but for other artistic forms such as seal carving and calligraphy. Mo Ran can learn and recreate any magnificent culture and artistic form,” says Professor Tang. We can take calligraphy as an example. Though there are only a few hundred characters left for Wang Xizhi’s calligraphy works, Mo Ran can still generate a complete font library of his calligraphy style after in-depth learning of his existing works, so that the derivative application scenarios of his fonts will be greatly enriched. The digital creative software currently developed by Professor Tang’s team is mainly based on the combination of big data and AI. “China has a natural advantage in developing the digital creative industry. We have large amounts of traditional cultural and artistic works, as well as a large number of designers, and have therefore accumulated massive data. In addition, China’s leading position in the AI industry has laid a solid foundation for relevant research,” says Professor Tang.

As the old saying goes, “We should follow the basic rules summarized by the ancients to create our own styles.” AI, as a cutting-edge technology, is taking a unique way to significantly influence the inheritance of traditional culture. It will also create a novel and distinctive environment for the development of digital creative arts. Mo Ran will use algorithms to develop creativity and promote the integrated development of art and science in accordance with the idea that “The more art develops, the more scientific it will be, just as science will become artistic.”