NVIDIA real-time ray tracing, AI painting, motion capture, VR theater, experience showroom, immersion paradise, emerging technologies bring endless possibilities for future entertainment experiences.
Editor’s note: This article is from WeChat public account “NeXTSCENE” (ID: theNeXTSCENE) , author Li Ya.
The quality immersive entertainment experience is inseparable from the perfect combination of art and technology. As a professional media with an industrial research perspective, NeXT SCENE has always paid attention to the introduction and reporting of immersive experience projects from the artistic and commercial level. The forefront of the development of immersive entertainment technology.
SIGGRAPH is the Top Annual Computer Graphics Conference organized by ACM SIGGRAPH (Computer Graphics Professional Group), the world’s most influential, largest and most Authoritative computer graphics conference. At this time of the year, the world’s top technology companies, researchers and institutions of higher learning will gather to share their research results. In recent years, SIGGRAPH has also highlighted the influence of the Immersive Experience, with VR Theater and Immersive Pavilion every year. NeXT SCENE was invited as an industry research media to participate in the entire SIGGRAPH event. Come and follow our researchers to see what the latest cutting edge of immersive entertainment technology is.
SIGGRAPH Full name: SpecialInterestGroup on Computer< Strong>GRAPHics and Interactive Techniques
Image from ACM SIGGRAPH Media
Immersive @ SIGGRAPH
15 VR theater experiences, 29 immersive venue experiences
Image from Muso
As a science, art, and business CG showcase and symposium, SIGGRAPH gave me the first impression of “Geek” (geek). Those who are most interested in computer graphics are undoubtedly computer engineers and graphic designers who pay close attention to the latest developments in technology. At the venue, the lectures were packed. They respected knowledge, remained curious about new things, and tirelessly explored the problem. This spirit gave me a lot of shock.
The warm corner of “Geek Bar” is constantly charging participants for a day
Image from ACM SIGGRAPH Media
In addition to seeing academic bulls, there will be a lot of “Real Geek” in the venue
pictureFrom Muso Instagram
News at the show
Every day at SIGGAPH, you can hear heavy news about computer graphics.
NVIDIA
NVIDIA announced at the seminar that more than 35 professional creative applications now support its RTX GPU ray tracing technology (Raytracing) ), and many of the world’s leading software manufacturers have launched applications that use RTX technology. This GPU not only maximizes graphics rendering tasks, improves real-time ray tracing in the entertainment experience, but also supports deep learning and advanced artificial intelligence.
Programs supporting RTX include: Adobe Substance Designer, Autodesk Flame, Blender Cycles, Daz 3D Studios, Foundry Modo, Dimension 5 D5 Fusion, and Luxion Keyshot, etc.
Image from author Lia
At the NVIDIA show, the personalized “moon landing” experience project (Moon Landing) attracted everyone’s attention. At the 50th anniversary of the Apollo 11 moon landing, NVIDIA recreated a real-time moon landing experience with the RTX GPU. After capturing the pose through the camera, the experiencer will match the 3D rendered astronaut character and render the details through real-time ray tracing. It is then combined with the development collaboration platform Omniverse to instantly generate realistic 3D images.
Let players revisit historical moments
Image from Nvidia
The special feature of this technique is that each pixel on the screen is generated in real time by tracking the beam into the camera. In this process, changes in characters in the scene trigger the interaction of light, intelligently adjust the picture and generate new images. For creators, this technology allows them to instantly and accurately see the reflection of light, the softness of shadows, global illumination, and other visual effects to better tweak the design. In the past, it took many days for special effects renderers to achieve the same level of fidelity.
Speaking of NVIDIA, you can’t help but mention the AI painting web applicationGauGAN. The app, named after the painter Gauguin, won the Siggraph’s Real-Time Live! Best in Show and the Audience Choice awards in the conference. Whether it’s a professional computer artist or an amateur painter, you can use the live demos to create high-level paintings in minutes.
I tried it for the first time, I feel that AI is still not with me
Image from Lia
Khronos Group
In Augmented Reality (AR) and Virtual Reality (VR), Khronos Group released the OpenXR 1.0 Specification, open source technology standards for royalty-free. The OpenXR Working Group is designed to help AR/VR developers create cross-platform mixed reality (XR) experiences without the hassle of switching between different software development kits.
PIXAR
Since the announcement of Pixar in 2015, open source animation software Universal Scene Description (USD), it has gradually become the standard in the animation and visual effects industry. This software helps creators to fully integrate the various applications in animation into other applications, effectively reducing data baking. So different creative people can work together in the same design scenario without waiting for the other party to complete. In addition to The Foundry, Unity, Epic Games (Unreal Engine), Pixar, and Animal Logic, both Autodesk and Nvidia have used and supported USD.
Speaking of this, I have to mention Pixar’s Renderman software. The latest release of “Toy Story 4” has been applied to this powerful rendering engine.
Special exhibition hall medium-high statue
Image from Lia
If you’ve seen a movie, you must remember that Hu Di and the shepherdess are in the antique store with a chandelier scene. This scene uses about 17,000 bulbs, all of which are presented with the support of Renderman!
In addition to the above booths, the author also found that there are many booths in the exhibition site: Optitrack and Qualisys , Rokoko showing the jumpsuit without an external camera, even the Noitom from China that I occasionally talked about.
Amazing Optitrack Motion Capture Show
Image from ACM SIGGRAPH Media
Noitom booth
Image from Lia
We can see that the commercialization of mobile technology is becoming more and more mature, assisted by the promotion of 5G technology, and everyone may be free to create later.
Experience Center Sharing
In the SIGGRAPH venue, the author’s favorite place is not the Experience Center (Experience Hall). This is the big stage for SIGGRAPH creators and researchers, and the perfect combination of art and technology.
Experience Hall
Image from ACM SIGGRAPH Media
In addition to the crowded VR Theatre (VR Theatre), the Experience Center also includes Art Gallery (Art Gallery) and Emerging Technology Showroom(Emerging Technologies),Experience Presentation, Immersion Park (immersive pavil)Ion) and studio (studio).
Art Gallery Booth Recommendation
This year’s art exhibition aims to explore how people face a future full of uncertainty. The works exhibited not only show the artistic achievements we have achieved in the present, but also raise the problems we have found in the globalized society and possible solutions.
Growing possibilities: the future of art and design conjecture
Image from Lia
Here, artists use Journey to mix different art forms and digital production methods to combine science, architecture, design, and virtual and enhanced platforms to think about social issues.
RUSHI: If yes
Color and Chinese metaphysical landscape combination
Image from ACM SIGGRAPH Media
John Wong from Hong Kong, China, was inspired by China’s Feng Shui Theory and Birthday CharactersFortune-telling Method, creating “If”, A algorithm interactive device surgery. This experience full of artistic beauty is behind some seemingly unconventional and wonderfully integrated elements: big data and artificial intelligence, destiny and superstition. Participants only need to enter their date of birth, and the screen will display visual data from different audiences, which will eventually be reflected in the color changes.
The author mentioned that data scientists and fortune tellersTo some extent, the uncertain world is quantified into understandable numbers or answers. So, is big data and artificial intelligence a new superstition?
Robotic Voice Activated Word Kicking Machine
This real “word game” makes the author play a lot of fun.
Image from Lia
Anything the participant said in the microphone will be instantly converted into text. If the text falls into the earpiece in the lower right corner, it will be read. If the text unfortunately hits the foot in the upper right corner, it will be “kicked back”. This interesting installation explores the interaction between virtual and reality, which shows me another interesting application of sound visualization. If Zhou Xingchi uses this piece in the scene of practicing the mouth of the sea in the “Nine Pieces of Sesame Officials”, the scene will be more spectacular!
NoiseAquarium
Arts about the ocean always give a warm and restful feeling. This seemingly ordinary virtual reality experience is actually not simple. Few people know that artificially generated underwater noise has a devastating effect on marine life. Not only do high-level creatures such as whales and dolphins, which are connected by sound, suffer from human noise, but the living space of a large number of plankton is also increasingly threatened.
Image from Lia
This plankton-filled interactive device uses 3D scanning technology to monitor the survival of microorganisms under the influence of noise, combined with data analysis, through animationThe experiencer presents a magnificent sight. Life that is invisible to the naked eye is also a life worthy of being respected, not to mention that these lives are closely related to the marine ecosystem. In fact, this is not only a problem of plankton, but also a problem faced by us human beings.
Emerging Technology Showroom Booth Recommendation
360-Degree Transparent Holographic Screen Display
Image from Lia
This 360-degree transparent holographic screen from Sony Japan Design Research Center provides participants with a unique “existence” reality. In fact, behind the transparent cylinder is a combination of 2D imagery and high-speed camera sensing technology.
Arque: Artificial Biomimicry-Inspired Tail for Extending Innate Body Functions
What if humans have a tail? Do you know how long it feels to have a tail? This research work, also from Japan, attracted a lot of attention on the spot.
Artificial bionic mechanical tail
Image from Lia
In view of the importance of the tail to most vertebrates, the designers proposed a personalized tail based on artificial bionics, which allows the experiencer to By changing the tail to affect the body’s momentum, strengthen the feedback and application of touch.
Matching Visual Acuity and PrescrIption: Towards AR for Humans
New wearable augmented reality device
Image from Lia
For the majority of eyewear wearers, wearing extra glasses is both cumbersome and detrimental to the experience. Researchers from the NVIDIA research team designed two new augmented reality devices: one that integrates degree correction into a 5 mm thick image synthesizer so that you can see without extra lenses Augmented reality images; one is to adapt the user’s field of view by adjusting the resolution and depth of focus.
Experience Showroom Booth Recommendation
OZ Immersive
OZ Game System
Image from Lia
This company from France can be described as my unexpected gain. High-level VR gaming images, fun game design and social elements, and an immersive experience, the OZ immersive gaming system is eye-catching.
game principle diagram
Image from Twitter @BlackLight
Mary and the Monster
Mary and the Monster is a perfect combination of drama and mixed reality.
Image from Parallux
The mother of science fiction Mary Shelley My life is as wonderful as the story she wrote, Frankenstein. This Augmented Reality/Virtual Reality experience brings readers back to the monster through the perspective of Young Mary. It’s unique in that it combines enhanced/the advantages of virtual reality with the viewing experience of the stage play, allowing viewers to More immersed in the hero’s unique inner world, and more free choice of different perspectives to experience the birth of the monster.
Wolves in the Walls: Chapter 2, It’s All Over.
Image from ACMSIGGRAPH Blog
Prefaced at the two major film festivals in Sundance and Tribeca, adapted from the virtual reality experience of Neil Gaiman’s novel “strong” “Wolf in the wall” The work also came to this SIGGRAPH. This is not just a fairy tale, it is an unparalleled journey of the soul.
Children playing in childhood, andThe interaction of virtual friends, the fear of darkness and unknownness, and the need for family trust, are all perfectly integrated into this thrilling VR adventure. In this experience, I am no longer just a bystander, but a storyparticipant. I have witnessed the horrible wolves coming to me; I am also graffiti and talking with the little girl in the darkness; I am looking for the truth in a big house with ghosts; the last stop is not only for the girls’ family. Destiny is the heart of my entanglement.
VR Theatre Recommended Works
Inside the VR Theater
Image from ictus_audio instagram
Small note: The 3D acoustic system of the entire VR Theater was designed and implemented by Ictus Audio in Vancouver. It is proud that the two founders of Ictus Audio, Yao Wang and Sherry Li, are Chinese and NSR research team. A friend who is in charge of Muso teacher.
KaiJu Confidential
Image from Lia
This five-minute VR movie is undoubtedly a surprise for this VR show. As a special film enthusiast, I first discovered that it was such a fun thing to see the destroyed town from the perspective of the monster.
Kate
Image from ACM SIGGRAPH Media
Speaking of VR movies, you have to mention the second VR short film “Kite” (Kate) in Disney Animation. Looking at the Chinese kite dragon kite ups and downs, I can’t help but recall the days when I was flying a kite. The kite here is human and energetic and does not require any words. In fact, simple things are often the most touching.
Computer Animation Festival Recommended Works
This Computer Animation Festival at the Electronic Theatre is a huge event. In the nearly three hours, dozens of animation masterpieces took turns: some pictures have reached the highest level in the industry, some of the stories are thought-provoking, and some are examples of the development of computer graphics technology. I have to say that Siggraph’s animation is superior in the industry.
Stuffed
Image from ACMSIGGRAPH Media
This animation of getting the Best Student Work Award is unbelievable. Among the many CG additions, Stuffed not only completed the production of stop motion animation at a high level, but also told a thought-provoking story. Although the animation has no lines, after reading it, I can feel the kind of loss, anxiety and helplessness that the author wants to express. The story that conveys emotions and moods is impressive.
Passage
Image from ACMSIGGRAPH Media
Animators from Serbia showed the audience what Art and Humanity. This character is made up of geometry, and the story without too much speech is deeply shocked.
War and violence, civilization and family, destiny and reincarnation, I think different audiences will have different experiences. Seeing the war machines that pointed the guns at the villagers, I couldn’t help but think of the chilling history of humiliation. Even more frightening is that these history may be reincarnation, perhaps in a corner of the world.
In view of space limitations, the author I only chose some of the most interesting insights to share with everyone. This year’s SIGGRAPH showcases the possibilities of combining cutting-edge computer graphics technology with art, and these possibilities also provide new spaces for the development of immersive experiences.
We’ve seen an immersive experience combined with games, and we’ve seen that immersive experiences are no longer limited to vision, but more to touch, temperature, and even pain. It is the continuous exploration and innovation of these computer engineers and graphic designers that makes more moving works from one idea to the other. It will be interesting to see how the future immersive experience will develop.