The underlying logic is to create a “water grid” in the virtual digital world.
In December 2018, the interactive movie “Black Mirror: Bandersnatch” was launched on Netflix, allowing players to replace the protagonist Choose between different events and choose different storylines.
The novel interactive mode has attracted a lot of interest from viewers, and the surge in user traffic even brought down the Netflix server for a time. Since 2019, Google, Amazon, and the BBC have announced the production and launch of their own interactive works. In China, Internet giants such as iQiyi, Tencent, and B Station have also begun to deploy interactive dramas.
rct studio is a new type of creative entertainment recently encountered The studio, established in 2018, is dedicated to the development of next-generation immersive interactive entertainment experiences through artificial intelligence. The company has been selected as the largest entrepreneurial incubator in the United States by Y combinator and is the first group of students at YC China.
From the perspective of interaction, Lu Yi, founder of rct studio, believes that holding the remote control to select the plot among the only few options is not the final form of interactive movies. This is important for the audience experience. , Is split .
In the concept of rct studio, The user needs to be fully immersed in the story environment. Natural language and motion become the connection points of the film and television screen. Participants can make decisions freely, and each action can be instantly drawn into the corresponding 3D view to advance the story. In terms of form, it cannot be specifically defined as a movie or a game. RCT Studio collectively refers to it as immersive full interactive entertainment.
So how to do it?
rct studio independently developed an AI-based Morpheus engine with core capabilities, machine learning capabilities, natural language programs (NLP), and real-time text to 3D image programs (Text-to-3D render ) And Motion prediction generation system based on deep neural network calculations.
Through the study of a large number of scripts, Morpheus can expand the input of a single story plot into an open world, understand the plot from the text,The inner logic and relationship between the characters, according to these, enrich each character’s way of thinking and action. It is equivalent to having a unique AI model for each character. These algorithms will understand what they should do in this storyline, and then they will cross-validate between characters and finally visualize all the possible ending.
The value that this can provide is that after the audience participates in the plot, as long as each choice made is consistent with the logic of the story environment, other NPCs will start to respond accordingly. In other words, AI can automatically complete the construction of the story line, which solves a large number of story branches of artificial screenwriters in the early stage of interactive dramas.
Story branch
By inputting text, real-time text-to-3D image programs can quickly generate physical expressions and motion relationships between people and objects in real time.
In the way of real-time streaming, the algorithm predicts and pushes out the pre-rendering result of the next frame in each frame of the character’s movement, achieving a process that can be real-time and dynamic, without the need to operate locally or in the The computer does extra rendering.
Taking walking as an example, using motion prediction to generate a path that can be arbitrarily planned, and it can also be arbitrarily adjusted during the walking process. Assuming a stone is suddenly placed in the path, people will avoid obstacles directly in the dynamic process, and then Plan your path.
The left side can avoid obstacles automatically
This is a new digital content production method, Lu said that it can significantly reduce the production cycle. Take walking as an example. In the traditional production process, you need to do 3D modeling, bone binding, and dynamic adjustment first. With this program, the work that might have taken from August to September can be completed in 3-4 days.
Also, this program can support the kinematic relationship between multiple objects, taking into account the physical meaning. For example, if the cup is thrown on the sofa and on the table, the latter will break, the former will not. In an interactive entertainment environment, this can provide physical support for immediate character feedback.
In general, using the Morpheus engine to create an interactive entertainment product, you only need to enter a standard Hollywood script or describe a scene that is sufficiently clear. The machine can automatically expand the story nodes, improve the character decision model, and generate The physical performance of the scene and the relationship between people and objects.
The user wears VR glasses with a microphone and somatosensory equipment and plays his own role, then he can start voice dialogue and interaction with the characters in the play.
Lu Ye said that what the core of rct studio hopes to solve is how to quickly generate text into an open world with internal logic and how to turn it into real-time actions. Both Morpheus and other technical means Only in order to ultimately improve the production efficiency of virtual digital content, rct studio hopes to be the “water grid” of the virtual digital world.
Cooperate with Nanjing University’s Artificial Intelligence Innovation Institute to launch user-level experience products
According to Ma Xinjie, co-founder of rct studio, rct studio has now reached a strategic cooperation with the Institute of Artificial Intelligence and Innovation of Nanjing University. The two parties will conduct research cooperation on Reinforcement Learning (RL) and research results. Will help rct studio optimize the character decision model.
In 2020, rct studio will launch a user-level entertainment experience product, with bank robbery as the theme, mainly for overseas markets.
In terms of content, Lu Yan said that rct studio would prefer to cooperate with mature IP first. At present, it has reached a cooperation with the future affairs bureau of science fiction content publishers. The number of writers that the latter has signed up to about 200 People, including Liu Cixin, author of Three Body, and the two may collaborate to launch interactive movies.
In terms of business model, Lu Yan said that rct studio is more interested in the C-end market than providing tools directly to the B-end. In addition to overall product sales, value-added services during the experience are also in the imagination space.
At present, there are more than ten people in the rct studio team, headquartered in Los Angeles, USA. Founder Lu Yan graduated from the University of Liverpool in the United Kingdom. He has founded the artificial intelligence intelligent operating system Raven Technology. After being acquired by Baidu, he served as Baidu. General Manager of Smart Home Hardware reports to Lu Qi, President and Chief Operating Officer of Baidu Group. rct studio has received multi-million dollar seed round funding from Xinghan Capital and Y Combinator, as well as a tens of millions of dollars in Series A investment from the Makers Fund.
Image source in text: rct studio