Although originally only a gimmick in mobile phone camera technology, pixel synthesis has now become a killer.
Editor’s note: This article is from WeChat public account “Three Easy Life” (ID: IT-3eLife), author of Trichomonas. Not long ago, we at Sanyi Life wrote an article reviewing the “ultra-high pixel” trend of smartphones in taking pictures in 2019. At that time we made it clear that the “ultra-high pixel design was not easy to use from the beginning, but it did gradually become truly valuable in 2019”, and it has also received a lot of reader and friend responses. . 
But we found out afterwards When we talked about the technological advancement in the design of ultra-high-pixel mobile phone photography, we missed a crucial part. This is the CMOS “pixel synthesis” technology that is commonplace nowadays-it is the foundation of ultra-high pixel mobile phones. But it is precisely that, it has added a lot of mixed elements to the ultra-high pixel camera models so far.
Undeniable, pixel synthesis was a gimmick at first
What is pixel synthesis? In simple terms, it describes an imaging method commonly used by ultra-high-pixel camera phones when they actually take pictures. For example, a 40 million pixel CMOS, when the shutter is pressed, it is indeed 40 million pixels that are all sensitive, but when the final output is made, the light information captured by every four pixels is “combined into one” and output. 10 megapixel photos. What good is this? Manufacturers will tell everyone that by “combining” four, nine, or even sixteen pixels, an equivalently larger single-pixel area can be obtained, thereby effectively improving the sensitivity in low light (at night).
Pixel synthesis is good for night scenes
But then the problem arises. First of all, as our Sanyi Life has already “revealed” a long time ago, at least so far, the “pixel synthesis” mechanism of ultra-high pixel mobile phones may not be as advertised by manufacturers. That way, it will only start in low light environment, but it is enabled by default every day. This means that, on the one hand, the hardware (processor, flash memory) of the mobile phone is not really ready for ultra-high-resolution imaging, and on the other hand, the ultra-high-resolution camera products that consumers spend on money ultimately get The actual photo resolution is only a fraction of its “nominal value”, which is a bit “incorrect.”
But even in daylight, you do n’t use full pixel mode by default
Today’s ultra-high-pixel camera smartphones generally enable the pixel synthesis mode by default, instead of working with all independent pixels by default. This shows that mobile phone manufacturers actually understand that in order to get better picture quality, “large pixel area” is still more important than “high pixel count”. That being the case, why not design a sensor with a large bottom and a large pixel directly at the beginning, and first make a nominal ultra-high pixel CMOS, and then “synthesize” it and use it after reducing the pixel. Is n’t it a man-made gimmick? ?
Manufacturers are not stupid, of course they know how to make up for themselves. Yes, as we said in the criticism and questioning of ultra-high pixels by Sanyi Life half a year ago, we It is believed that as the “core design” of ultra-high-pixel camera phones, the pixel synthesis technology was actually quite tasteless when it was first born, and it also had a strong gimmicky meaning. After all, consumers can’t really enjoy the nominal pixel value of the mobile phones they buy by default, which is enough to criticize. ![The pixel synthesis technology in mobile phone photography, from gimmicks to killers]()
The earliest feature design other than improving night shots is the so-called “single frame HDR”. As we all know, the initial HDR (high dynamic range) photos are to take at least three photos of underexposure, normal exposure and overexposure respectively, and then through post-compositing to achieve a high dynamic effect of “can be seen clearly in bright and dark places” . This not only makes the exposure time of the photo longer, but it is also easy to cause the composition to fail or the image quality to be degraded due to hand shake or subject movement.
![The pixel synthesis technology in mobile phone photography, from gimmicks to killers]()
As technology advances, gimmicks can become killers. Not only that. If you used to pay attention to the content of our mobile phone photos in Sanyi Life, you may remember that we praised it more than once. Once “12 million full-pixel dual-core focus” CMOS. However, it is estimated that few people know that the 12-megapixel dual-core CMOS is called “dual-core” because it actually has 24 million photosensitive elements. It is just that each two photosensitive elements share one pixel well and pixel. The lens is therefore considered a pixel. In this case, the two pixels in a single pixel well can be used for focus calculation because of the phase difference. All 12 million pairs of pixels participate in the phase focus calculation. This is the secret of the “all-pixel dual-core CMOS” focusing extremely fast Where it is. 
The difference between pure ultra-high pixels, four-pixel composite ultra-high pixels, full-pixel dual-core, and full-pixel quad-core structures
Obviously, this design can of course be applied to today’s ultra-high pixel CMOS. For example, Sony has developed a “2×2 On-Chip Lens” CMOS that allows four photosensitive elements to share a micro lens. It combines all the advantages of “multi-core focusing” and “multi-pixel synthetic imaging”. It has a native “four-in-one pixel imaging” structure, which is convenient for mobile phone manufacturers to make products with high number of pixels and large “equivalent pixel area”. At the same time, it also has the fast focusing performance not available in the ultra-high pixel CMOS that each pixel corresponds to a lens. More importantly, compared to the “full-pixel dual-core focus” CMOS structure that used to be well-known, this “full-pixel quad-core focus” has horizontal, vertical, and oblique phase difference pixels at the same time, but instead It is also expected to achieve better focusing and tracking performance than the most advanced focusing technology in the past.
What does this mean? From a consumer’s perspective, this may represent the once-complex pixel synthesis structure and ultra-high pixel CMOS, which will soon become the mobile phone CMOS with the best focusing performance when taking pictures and shooting videos in the near future. From a technical point of view, this also explains several issues. For example, there is no absolutely useless technology, but only whether the manufacturer will use it. For example, for new technologies and new products, sometimes they are not in a hurry to try new things. , Wait for a while to make it mature and improve, maybe surprises may happen.
The earliest feature design other than improving night shots is the so-called “single frame HDR”. As we all know, the initial HDR (high dynamic range) photos are to take at least three photos of underexposure, normal exposure and overexposure respectively, and then through post-compositing to achieve a high dynamic effect of “can be seen clearly in bright and dark places” . This not only makes the exposure time of the photo longer, but it is also easy to cause the composition to fail or the image quality to be degraded due to hand shake or subject movement.
As technology advances, gimmicks can become killers. Not only that. If you used to pay attention to the content of our mobile phone photos in Sanyi Life, you may remember that we praised it more than once. Once “12 million full-pixel dual-core focus” CMOS. However, it is estimated that few people know that the 12-megapixel dual-core CMOS is called “dual-core” because it actually has 24 million photosensitive elements. It is just that each two photosensitive elements share one pixel well and pixel. The lens is therefore considered a pixel. In this case, the two pixels in a single pixel well can be used for focus calculation because of the phase difference. All 12 million pairs of pixels participate in the phase focus calculation. This is the secret of the “all-pixel dual-core CMOS” focusing extremely fast Where it is.
The difference between pure ultra-high pixels, four-pixel composite ultra-high pixels, full-pixel dual-core, and full-pixel quad-core structures
Obviously, this design can of course be applied to today’s ultra-high pixel CMOS. For example, Sony has developed a “2×2 On-Chip Lens” CMOS that allows four photosensitive elements to share a micro lens. It combines all the advantages of “multi-core focusing” and “multi-pixel synthetic imaging”. It has a native “four-in-one pixel imaging” structure, which is convenient for mobile phone manufacturers to make products with high number of pixels and large “equivalent pixel area”. At the same time, it also has the fast focusing performance not available in the ultra-high pixel CMOS that each pixel corresponds to a lens. More importantly, compared to the “full-pixel dual-core focus” CMOS structure that used to be well-known, this “full-pixel quad-core focus” has horizontal, vertical, and oblique phase difference pixels at the same time, but instead It is also expected to achieve better focusing and tracking performance than the most advanced focusing technology in the past.
What does this mean? From a consumer’s perspective, this may represent the once-complex pixel synthesis structure and ultra-high pixel CMOS, which will soon become the mobile phone CMOS with the best focusing performance when taking pictures and shooting videos in the near future. From a technical point of view, this also explains several issues. For example, there is no absolutely useless technology, but only whether the manufacturer will use it. For example, for new technologies and new products, sometimes they are not in a hurry to try new things. , Wait for a while to make it mature and improve, maybe surprises may happen.
What does this mean? From a consumer’s perspective, this may represent the once-complex pixel synthesis structure and ultra-high pixel CMOS, which will soon become the mobile phone CMOS with the best focusing performance when taking pictures and shooting videos in the near future. From a technical point of view, this also explains several issues. For example, there is no absolutely useless technology, but only whether the manufacturer will use it. For example, for new technologies and new products, sometimes they are not in a hurry to try new things. , Wait for a while to make it mature and improve, maybe surprises may happen.