Picture: Qian Xuesen translated VR into “spiritual realm” in his letter
“It is a good thing that the concepts of virtual reality and the metaverse are being actively discussed at home and abroad, and it also promotes the progress of related disciplines. However, as a scientific research, I think the most important thing may not be the progress of a certain technology or algorithm itself, but the breakthrough of related technologies to the traditional scientific research thinking paradigm, and the impact on human life, sports, and Changes in the way of health management.” Sheng Bin, a professor at the Department of Computer Science and Engineering of Shanghai Jiao Tong University, said in an exclusive interview with News (www.thepapewr.cn).
Sheng Bin is the winner of the National “Ten Thousand Talents Program” for young top-notch talents, the director of the Artificial Intelligence Key Laboratory of the Ministry of Education and the smart medical technology of the Shanghai Institute of Artificial Intelligence. Both artificial intelligence and virtual reality (Virtual Reality, VR) fields have been studied. Based on the world’s largest community diabetic fundus image database led by Academician Jia Weiping of the Sixth People’s Hospital Affiliated to Shanghai Jiaotong University, Sheng Bin’s research group constructed the DeepDR system, an intelligent diagnosis system for the whole course of diabetic retinopathy. It can automatically diagnose the entire disease process from the proliferative stage, and can provide real-time feedback on the quality of fundus images and identification and segmentation of fundus lesions. The paper was selected as Editors’ Highlights by Nature Communications, a sub-journal of Nature, and is considered to be “one of the most important technological innovations and research achievements in the field of translational and clinical research” by the international academic community recently. 
https://www.nature.com/articles/s41467-021-23458-5
In the field of virtual reality, since 2019, Sheng Bin’s team has been deeply involved in the technology led by Shanghai Institute of Physical EducationIn the Ministry of Science and Technology Winter Olympics research project, a virtual reality-based snowmobile simulation training platform was built for Chinese athletes preparing for the 2022 Beijing Winter Olympics. Due to the high cost of snowmobile slides and extremely high technical requirements, there are currently 17 snowmobile slides distributed in very few countries in the world. The extreme demand for related venue resources greatly troubles Chinese athletes to prepare for the competition. For example, due to the lack of proper track support, the first China Snowmobile Championships in 2017 were all held in Calgary, Canada.
“With the strong support and help of Prof. Wang Ran and Prof. Gao Binghong from Shanghai Institute of Physical Education, we have repeatedly solicited opinions and feedback from Chinese national team leaders, coaches and athletes , the purpose is to face the technical challenges that plague China’s large-scale popularization of ice and snow sports, and then fundamentally promote the level of China’s ice and snow sports through scientific and technological strength. With the support of the Science and Technology Winter Olympics project, we modeled through realistic graphics. Interacting with rendering, physical simulation in ice and snow sliding, and highly immersive virtual reality, transforms snowmobile training in a virtual environment into a virtual simulation-assisted teaching and training system with high-level competitive experience, rather than games.” Sheng Bin said.
“Ice and snow sports should not only belong to Gu Ailing,” Sheng Bin said with a smile, “Virtual reality technology can allow more Chinese teenagers to experience the joy brought by ice and snow sports. “Since 2019, the ice and snow sports virtual simulation system and its teaching application developed by Sheng Bin have been selected by the Shanghai Municipal Education Commission as the Shanghai virtual simulation teaching project and the first-class course of Shanghai Jiaotong University.
In January 2022, on the eve of the Beijing Winter Olympics, led by Vice President Wang Jihong of Shanghai Institute of Physical Education, Shanghai Jiaotong University, Baoshan District Education Bureau and Shanghai Artificial Intelligence Research With the support of the Ministry of Education, the “Intelligent Sports and Education Application Practice Community” has been established. One of the tasks is to rely on the virtual simulation system of ice and snow sports developed by Sheng Bin’s team and promote it to Baoshan District, Shanghai. In the physical education and technology experience of various primary and secondary schools, help more Shanghai teenagers to understand and love ice and snow sports. Sheng Bin said, “It is equivalent to leaving a little spark in his heart.”
“Our country has not performed well in the Winter Olympics for a long time, so what is the reason for this? The reason is that, first of all, there are not enough ice and snow geographic resources, and secondly, there are shortcomings in our work to popularize the knowledge of young people’s ice and snow sports.” Sheng Bin said. Not only in China, but in fact few countries other than countries that have hosted the Winter Olympics will build such high-input and low-output professional ski trails – large ski resorts can also be converted into civilian leisure facilities after the competition. A professional track is really hard to have the value of follow-up use other than training competitions. andIn addition, the speed of snowmobiles can generally reach 130 kilometers per hour, and the world record has exceeded 200 kilometers per hour. It is known as “F1 on the snow”. For safety reasons, there are also high professional requirements for participants.
In this regard, Sheng Bin believes, “Whether it is a steel frame sled, skiing, or speed skating, these athletes have their physical conditions and hobbies behind their success. However, China has a large population and has abundant physical resources suitable for ice and snow sports. However, due to geographical and climatic constraints, not all of these people can easily come into contact with ice and snow sports in the enlightenment stage. A large number of ice and snow sports The genius may be buried here, then the simulation system of ice and snow sports in the virtual reality environment jointly developed by Shanghai Jiaotong University and Shanghai Institute of Physical Education may be able to help the future talent pool of China’s ice and snow sports to the greatest extent, and help them take the first step of ice and snow life “
Sheng Bin also combined the current hot “metaverse” concept to further explain the technical background behind the virtual simulation system of ice and snow sports, “Our ice and snow sports virtual The simulation system can be regarded as a metaverse application of ice and snow sports teaching, and the metaverse can be regarded as an expression form of human networked life in a virtual reality environment. The so-called virtual in virtual reality refers to the use of computer technology as the core, combined with In related scientific fields, a virtual digital environment is generated to provide people with visual, auditory, tactile, spatial information and other perceptions; the so-called reality refers to such a digital virtual environment and the various perceptual information provided to people, which can make people immersed Among them, a sense of realism is produced.”
The early maturity of VR: vertigo has not been completely solved
5G and even the post-5G era, the basic hardware that supports the large-scale development of VR, such as chips, sensors, Optical components, display screens, etc. all have mature development conditions. Roblox, a metaverse concept stock company that uses VR and other devices as an interface, will have a market value of US$50 billion after its listing in March 2021. After ByteDance’s acquisition of Pico, the first domestic VR manufacturer, once again attracted widespread attention. In addition, Goertek Group has been deploying the VR/AR industry since 2012, and currently occupies 70% of the global market share of mid-to-high-end virtual reality headsets, and took the lead in introducing the world’s leading 12-inch nano-imprint production line to build the world’s leading VR/AR High-end optical components industry base.
According to a report released by IDC in November 2021, shipments of VR headsets will increase from about 5 million units in 2020 to more than 5 million in 2025. 28 million units. Meta (former Facebook) CEO Mark Zuckerberg once said that about 10 million people need to use and purchase VR content on a platform in order for developers to continue to develop and make profits; once this threshold is exceeded, the content and ecosystem will be Realize leapfrog development.
On the domestic academic ecological construction of virtual reality, Virtual Reality & Intelligent Hardware (VRIH), an international journal in the field of virtual reality independently founded by Chinese academic circles, was founded less than three years ago , that is, at the end of January this year, it was included in EI Compendex, the world’s most extensive and complete engineering literature database, and will be held in October this year with Computer Graphics International (CGI2022), the world’s oldest and well-known international academic conference in virtual reality and graphics. As the editorial board member of the VRIH journal, Sheng Bin, who jointly organized the “Metaverse” album, believes that the above events all prove that China’s virtual reality academic and industrial achievements have an increasingly important international influence.
In the process of designing the virtual simulation system for ice and snow sports, Sheng Bin realized that the most subtle part is to overcome the limitations of the inherent knowledge field. Sports carry out realistic modeling and simulation, and is committed to using virtual reality and artificial intelligence technology to restore and reproduce the real experience and feelings of others about this activity to the greatest extent. “This is actually one of the charms of the Metaverse. To really do this well, in addition to tackling difficult problems in modeling, rendering or interaction technology, it is also important to integrate other disciplines, such as exercise physiology, The laws of movement of some equipment and equipment on ice, etc., are integrated in the system development process.”
The “Virtual Reality Industry Development White Paper 2021” released by CCID Research Institute shows that the current The key core technologies such as dizziness and interaction in the virtual reality industry have not yet broken through, and there is still a gap between the wearing comfort and visual comfort of VR, AR, MR and other terminal equipment and comfortable experience.
As for dizziness, Sheng Bin explained, “Essentially, dizziness actually comes from the discomfort caused by the brain’s processing of ‘unrealistic information’. Unrealistic visual perception, flickering pictures, low resolution, or abnormal light and shadow effects will all bring discomfort. There are many technologies for dealing with vertigo, but I don’t think this problem has been completely overcome. This is why many people talk about the metaverse, but the actual experience is still stuck on the device.”
“”The Ultimate Display”: The Virtual Reality Window of the Future Human Digital Society – The Metaverse
As early as 1965, Ivan Sutherland, the father of computer graphics He published a short article titled “The Ultimate Display”, proposing that the ultimate goal of display technology is to immerse people directly in a computer-controlled virtual environment, just like everyday life in the real world. The expectation of the ultimate state of this display technology is still the window of the Windows system. People can walk into Windows, just like Alice in Wonderland, they can move, listen, see, touch and smell. Sheng Bin said.
This “ultimate display” idea has continued to this day, and there is a “next-generation general-purpose computing platform”. In 2016, Goldman Sachs released a report “VR With AR: Interpretation of the Next Universal Computing Platform” (AR Augmented Reality, Augmented Reality). In February 2022, Guotai Junan released the report “MR: The Next Generation Universal Computing Platform Leading to the Metaverse” (MR Mixed Reality, Mixed Reality).
Sheng Bin revealed that he is currently working closely with the team of Academician Jia Weiping, Chair Professor of Shanghai Jiao Tong University, and working with experts from sports science, clinical medicine, psychology and brain science. A multidisciplinary team of experts collaborates to carry out scientific research and develop a “spiritual” system for the intelligent and refined management of people’s health in future life, hoping to use artificial intelligence and virtual reality to answer the current problems that have plagued the scientific community for many years: physical exercise in the real world and virtual world. (Metaverse) What impact does the VR movement have on human physical and mental health.
Sheng Bin believes that behind these efforts lies his scientific belief, “If we finally follow The communication between machines is through AR, VR or MR, then through the comprehensive use of multi-disciplinary knowledge, based on big data and high-performance general computing platforms, virtual simulation and artificial intelligence can completely simulate the real world and the laws of human state changes. with forecast. ”
Picture: Qian Xuesen expresses his expectations for the future of VR in his letter
p>