This article is from WeChat official account:Ai Faner (ID: fanr), author: get a new word, title figure from: vision China
The arrival of the epidemic has degraded the noble head of the face unlock function.
When masks become a must-have item for us to go out on the street, we always have to go through the cumbersome process of “facial recognition failure”-“input password” when unlocking the phone. This makes people miss the fingerprint recognition.
In order to optimize the face unlocking experience, earlier this year, foreign entrepreneur Danielle Baskin launched a mask with facial information. The product extracts the user’s facial information and prints it on the outside of the mask. After the user wears the mask, they can piece together a complete face.
It’s actually a bit scary. Picture from: djbaskin
However, the unlocking success rate of this product is not yet clear, and there is no large number of sample verifications. So how can people’s face recognition system stop being bothered by masks?
Some netizens have gradually discovered that as the time of wearing a mask becomes longer and longer, the mobile phone seems to have found “experience” from repeated face unlock failures, and is gradually able to recognize the self wearing a mask successfully.
Relying on this idea, some technology bloggers have also shared more efficient tutorials. For example, if you wear a mask to unlock repeatedly, enter the password immediately if the face recognition fails. Repeat this action for about 30 minutes, and the phone can recognize the wearer. I’m wearing a mask.
Successfully unlock the face while wearing a mask Picture from: Farhad Usmanoff
However, in the process of practice, netizens said that the “learning” speed of different models is different. Some people have repeated the above actions for 20 minutes and have succeeded, but some people have repeated them thousands of times, and their mobile phones still cannot recognize themselves wearing a mask.
Why does this happen? In fact, the answer relates to the AI learning ability of mobile phones.
Picture from: thenextweb
Deep learning tool——NPU
If you have followed the mobile phone conferences in the past two years, you must have found that when mobile phone manufacturers introduce SoC chips, they will focus on NPU upgrades.
The so-called NPU refers to the neural network processor. In a mobile phone chip, it is generally divided into several functional areas. There are three frequently mentioned at the press conference: one is the CPU that is good at handling complicated tasks and issuing commands, the other is the GPU that is good at graphics processing, and the other is NPU who is good at handling artificial intelligence tasks.
Although the NPU “occupies” less space than the CPU and GPU, its capabilities cannot be ignored. The intelligence of a mobile phone mainly depends on it.
Chips with NPUs are often called “AI chips” and “bionic chips”. Image from: Tears Snow Net
The training of mobile phones mentioned above to recognize themselves wearing masks is mainly due to the ability of NPU. After the camera captures the face image, the CPU and GPU will preprocess the image in a very short time, then the NPU and GPU will detect and extract features, and finally the CPU, GPU and NPU will work together to complete the face recognition and classification.
Thanks to the increasingly powerful computing power, the whole process has been able to be “insensitive”. The moment we picked up the phone, the above process was completed.
The addition of NPU allows mobile phones to recognize you in different states. When you wake up in the morning, even if your face is swollen, your phone knows that this is you. Even after being stung by a wasp, his mouth swollen into a “sausage”, the phone can still recognize it.
Picture from: Captain Han Drifting Notes
So after a certain amount of training, the mobile phone can recognize you without fear of masks.
In fact, if you only rely on algorithms, the CPU and GPU can also cooperate to complete the learning. But the disadvantage is low efficiency and high power consumption. According to the introduction of “Automotive Electronics and Software”, CPU and GPU need thousands of instructions to complete neuron processing, and NPU only needs one or a few to complete.
NPU’s learning efficiency is quite high. Picture from: androidauthority
In addition, under the same power consumption, the performance of the NPU is 18 times that of the GPU. It can be seen that NPU has obvious advantages in the processing efficiency of deep learning.
Speaking of this, I have to mention the working principle of NPU. The reason why the NPU is high in learning efficiency is not because it drank the “six walnuts”, but because it simulates human neurons and synapses in the circuit layer. And use the deep learning instruction set to directly process large-scale neurons and synapses. By highlighting the weight to achieve the integration of storage and computing, one instruction of the NPU can be competent for thousands of instructions of the previous CPU and GPU.
To use a less appropriate analogy, this is like the integration of warehousing and logistics realized by JD Logistics, which greatly improves delivery efficiency, and can be purchased on the same day or even delivered on the same day.
NPU is not tasteless
The earliest domestic company to study NPU was the Cambrian. The Kirin 970 chip released in 2017 used the Cambrian NPU architecture. Kirin 970 has also become the world’s first mobile AI chip.
According to Huawei, the Kirin 970 with integrated NPU unit has about 50 times the energy efficiency and 25 times the performance advantages when processing the same AI application tasks compared to the four Cortex-A73 cores. For example, the image recognition speed can reach about 2000 sheets per minute, which is much higher than the industry level in the same period.
Kirin 970 Picture from: Electronic Engineering Album
Eleven days later, iPhone 8/8 Plus and iPhone X came out carrying the A11 bionic chip. Apple stated at the press conference that this is the most powerful and smartest chip in its history.
A11 Bionic is Apple’s first processor named “Bionic”, and it is also Apple’s first processor to support AI acceleration. For example, in the function of face recognition, its neural network engine allows A11 to support a speed of up to 600 billion operations per second.
Picture from: stealthsettings
From this year, more and more manufacturers began to pay attention to the promotion of mobile phone AI capabilities. For example, Huawei’s main AI photography, super night scenes, air gestures and other functions; iPhone’s proud Face ID, portrait blur, Deep Fusion(Deep Fusion) and other functions closely depend on the capabilities of the NPU.
Huawei AI gesture control
Since June 2019, with the release of the Kirin 810, Huawei began to use the self-developed Da Vinci-based mobile phone AI chip. The cleverness of the Da Vinci architecture is that the division of labor between the units is clear, which enables more efficient AI calculations.
According to the introduction of “Electronic Products World”, the core 3D Cube, Vector vector computing unit, Scalar scalar computing unit, etc. of the Da Vinci architecture are each responsible for different computing tasks to achieve parallel computing models and jointly ensure the efficiency of AI computing deal with. Realize the characteristics of high computing power, high energy efficiency, flexibility and tailorability.
At the recent Mate 40 series press conference, Huawei emphasized that the NPU of the Kirin 9000 chip has been upgraded to version 2.0 of the Vinci architecture, doubling the computing power. While AI computing power is stronger, energy efficiency has increased by 15%, and network performance has also increased by 20%.
In the AI Benchmark list launched by ETH Zurich, Kirin 9000 won the top prize in the Android camp, with a score more than twice that of Qualcomm Snapdragon 865+.
AI Benchmark List
Remember the aforementioned Kirin 970’s ability to recognize 2000 images per minute? Kirin 9000 has evolved to a speed of 2000 frames per second. In addition, the AI air gestures, AI smart screen off, and AI subtitles that were highlighted at the press conference are all manifestations of its NPU capabilities.
I was particularly impressed by the “Smart Payment” function. When the mobile phone senses that it is close to the scan box, it will automatically pop up the payment code page and complete the payment in one go. This represents the direction of the ideal smart terminal: to “know you”, “know you” and “help you”.
Huawei Smart Pay Picture from: VDGER
When the fourth-generation iPad Air was released, Apple also emphasized the improvement of its NPU capabilities. Compared with the A12 bionic processor, the A14 bionic new generation neural network engine makes machine learning performance twice as fast.
The ultra-high speed of machine learning allows the A14 bionic chip to realize the super pixel function. It can be used with pixelmator to enlarge the cropped photos, and the pixels will be automatically added to make the photos clearer.
Reflected in the iPhone 12 series, computational photography capabilities have also been unprecedentedly improved. For a small example, during time-lapse photography, the mobile phone will automatically calculate the subject. If it is shooting traffic, the mobile phone will automatically reduce the shutter speed, so that the car lights have a feeling of smearing and the picture is fluid. Stronger.
Compared with iPhone 11, the new generation of iPhone has visible changes in Deep Fusion and HDR video. This is all thanks to the powerful AI computing power of A14.
What can we expect from NPU?
Although the mobile NPU has only been advertised by manufacturers in the past two or three years, in fact, the concept related to it has appeared in 2013.
At that time, Qualcomm hoped to narrow the gap between ordinary machine operations and the human brain through a computing structure that mimics the human brain. This type of computing processor that simulates neurons is called “Zeroth” by Qualcomm.
Qualcomm’s introduction to Zeroth
Qualcomm’s Zeroth chip has a computational structure that mimics the operating mode of human biological nerve cells, which is mimicked from the structural level of the brain. NPU is imitated at the level of brain function, and the directions of the two are not consistent. And Qualcomm has always insisted on its own direction, not joining the army of independent NPUs, but insisting on the direction of the artificial intelligence engine AI Engine.
According to “Xinzhixun” reports, when Qualcomm Snapdragon 845 was released, some outside voices criticized Qualcomm for not following the NPU trend, so that it lags behind in AI capabilities. Alex Katouzian, senior vice president and general manager of mobile business of Qualcomm, responded that although Qualcomm does not have an independent neural network engine unit, it uses a more flexible machine learning architecture (AI Engine), the kernel is optimized in the general platform and distributed on each unit such as CPU, GPU, DSP, etc., so as to provide flexible calling of various processing units for different mobile terminals.
You can understand this: The direction of NPU is clear division of labor, and the degree of intensification of each unit is relatively high; while the direction of Qualcomm AI Engine is “work together with everyone.”
Until the release of the Snapdragon 865 series chip using the fifth-generation multi-core artificial intelligence engine AI Engine, Qualcomm still had no way to enter the NPU.
Qualcomm emphasized AI capabilities at the bottom left of the picture
However, in actual use, the learning ability of Qualcomm Snapdragon 865 is still worthy of recognition. For example, when I used the vivo X50 Pro+ equipped with Qualcomm Snapdragon 865+ for nearly half a month, it unlocked about ten times a day, and it can now successfully identify me wearing a mask.
However, from the data point of view, its AI learning ability has fallen behind the Kirin 9000 and A14 bionics by a lot. NPU has used data to prove its AI strength time and time again. Whether Qualcomm’s next-generation AI Engine can turn the tide, we still need to wait for the 875 series of chips to be available to know.
In the era of artificial intelligence, the scenario I hope to see is that mobile phones are no longer terminals that passively respond to user needs, but smart terminals that can actively analyze and perceive users’ current needs and provide related services in advance.
Mate40 series AI capability demonstration
In this regard, each manufacturer is still in its infancy. For example, in terms of application suggestions, I personally think that the best one is Xiaomi. Through analysis of factors such as time and scene, it can “guess” the software I want to open every time, and intelligently sort it in the most conspicuous position. The “Smart Payment” supported by the Mate40 series is undoubtedly at the forefront of the AI road, and it also gives us more room for imagination.
It is worth noting that in addition to mobile phones, NPU is also gradually being applied to mobile terminals such as tablets and laptops. Apple’s recently released M1The chip has a 16-core NPU, which can perform 11 trillion operations per second, increasing the speed of machine learning to 11 times, which is difficult for traditional PCs to match and compare.
And what changes in the user experience will be brought about by the MacBook series and Mac mini equipped with the M1 chip, I believe it is expected.
Under the current software ecosystem, the improvement of mobile CPU and GPU is not strong enough for users’ daily use. For example, compared to an iPhone XS and an iPhone 12, the application fluency is almost the same. What affects the user experience more is the change in machine learning capabilities. This is why we should pay attention to the development of NPU.
Perhaps in another ten years, when the development of AI technology becomes more mature, it is time for “smart” phones to be renamed “smart” phones.
This article is from WeChat official account:Ai Faner (ID: fanr), author: renewal statement