Qualcomm's new depth-sensing camera is surprisingly effective
The IR-based system could be the next dual camera.
Dual cameras are so passé. Qualcomm is getting ready to define the next generation of cameras for the Android ecosystem. It's adding three new camera modules to its Spectra Module Program, which lets device manufacturers select readymade parts for their products. The additions are an iris-authentication front-facing option, an Entry-Level Computer Vision setup and a Premium Computer Vision kit. The latter two carry out passive and active depth sensing, respectively, using Qualcomm's newly revamped image-signal-processing (ISP) architecture.
Of the three new modules, the most intriguing is the premium computer vision kit. That option is capable of active depth sensing, using an infrared illuminator, IR camera and a 16-megapixel RGB camera (or 20-MP, depending on the configuration). The illuminator fires a light that creates a dot pattern (using a filter), and the IR camera searches for and reads the pattern. By calculating how the dots warp over a subject and the distance between points, the system can tell how far away something is. And since this technology uses infrared light, it can also work in the dark.
During a recent briefing in New York, Qualcomm representatives also showed me a video (embedded above) of the setup in action. It was truly remarkable how the system was able to accurately display the profile of a person's hands playing the piano, despite only shooting from above. We'll have to wait and see it in action for ourselves before knowing if it'll be effective in the real world, but so far the technology is impressive.
The module can get very detailed, since it uses more than 10,000 points of depth and can discern up to 0.125mm between the dots. This precision is important. "Depth sensing is going to be mission critical going forward," Qualcomm's product marketing lead for camera and computer vision, Philip-James Jacobowitz, told Engadget.
There are plenty of useful applications for depth sensing, one of the most widespread being creating artificial depth of field in images. It can also help in facial detection, recognition and authentication; 3D object reconstruction; and localization and mapping, according to Qualcomm.
If this is all starting to sound familiar, you might be thinking of Intel's RealSense. It's used more commonly in laptops for facial or iris authentication via Windows Hello, although it has recently made its way to a few phones and even drones as well. But Qualcomm claims that according to its understanding, the thermal overhead on Intel RealSense is a barrier to entry for smartphones. That means it most likely runs too hot to work well in a handset. Regardless of the differences in equipment, it's clear that depth sensing has a wide variety of useful applications, and Qualcomm working on a competing option could help improve performance across the board.
That's not to say the two other modules Qualcomm announced today aren't useful. The entry-level computer-vision option, for instance, allows for similar (albeit less-precise) features at a lower price. But it consumes more power than the premium kit. And according to Qualcomm, its iris authentication module can read your eyes even when you have sunglasses on -- something the company's representatives demonstrated effectively during my demo.
There's no word yet on exactly when we can expect the new cameras to ship inside devices, but Qualcomm says the new modules and ISPs "are expected to be part of the next flagship Snapdragon Mobile Platform." Smartphone makers have to incorporate the components into their products first too, meaning we'll probably have to wait for next year's phones.
