Facebook reveals its camera-centric AR future

Facebook is convinced that folks are going to ply the world with cameras that can enhance your reality, as it told us last year. Now, at F8 in San Francisco, Mark Zuckerberg has unveiled the "Camera Effects Platform," the first developer system for augmented reality. The idea is to let companies build apps that can show you multiple things laid on top of reality: Information, digital objects and enhanced effects, all interacting with real-world objects like your home or face.

1 / 7

There are several elements to the Camera Effects Platform. One is Frame Studio, an app that lets anyone with a profile or page design frames for profile pictures and share them with friends. The more interesting tool is AR Studio, which is now in a closed beta on the Mac OS X platform. That app lets developers and artists build animated frames, masks and effects that can be used during Live broadcasts to add animated frame, masks and effects to video (below).

Zuckerberg showed a few examples of what such apps will look like. There's a screen-based Facebook Stories app that overlays camera effects on your selfies, for one. Another is called 3D effects, a sort of portable CGI that lets you capture and interact with 3D scenes at a high level of precision, something akin to what Microsoft has done with Hololens. (By the way, Snapchat "coincidentally" launched a very similar app called 3D objects today, which must have annoyed the hell out of the Facebook team.)

As an example of 3D effects, Zuck showed off an app that adds CGI to images, filling a room with skittles and liquids. Another involves image recognition -- if you touch a coffee mug, the camera can figure out what it is and add virtual steam, for instance. Or, if you tap a plant, you can add a shower of rain, or overlay a label, complete with a rating, onto a wine bottle.

There are also typical filters (like those first pioneered by Snapchat, of course) that can slap a Nike headband on your selfie as part of Nike's Run Club app. Other examples include an AR art project that lets you digitally "paint" an entire room, and one that used a real table as an AR game platform. You can also leave digital "notes" on your real fridge or read a simulated menu at a restaurant.

Of interest to gamers, Facebook showed off a Mass Effect demo (above) that builds on the social features it first revealed at CES 2017 earlier this year. It lets you overlay the look of the game onto real life, display 3D leaderboards and more. It also supports face tracking using both the front and back cameras, real-time 3D rendering and more. Another demo showed how you can overlay Manchester United-themed confetti onto a team win, or overlay Giphy GIFs onto live video.

All of this sounds pretty nice, but it's also merely an augmentation of current AR apps like Pokémon Go or selfie effects from Facebook, Snapchat and others. Zuckerberg didn't make much mention of new-fangled AR platforms like Hololens-like glasses or contacts that can give you simulated information or effects on the go. To be fair, however, there aren't really any such products on the market yet.

It's also worth noting that a lot of these apps are oriented towards businesses like Nike and Mass Effect developer BioWare. As such, they could turn the Facebook app into even more of a corporate ad space, much as it's doing with Messenger. So while the new AR stuff sounds cool, it's also a way to for Facebook to insinuate itself even more into our real-world lives, with the not-so-benevolent purpose of gaining even more ad revenue.

Click here to catch up on the latest news from F8 2017!