Microsoft's Layout and Remote Assist apps are just what HoloLens needs

It's one thing to talk about how much HoloLens could help businesses, and it's another to show it. That's just what Microsoft did yesterday, when it led me into two elaborate demos for its latest Mixed Reality apps: Layout and Remote Assist. The former lets you design physical spaces virtually, rather than just relying on blueprints. Remote Assist, on the other hand, lets you collaborate with people while letting them see through your eyes using HoloLens.

While they're not the sort of apps that would make HoloLens a must-have for everyone, based on my demos, they could make life easier for certain professionals. And that's exactly what Microsoft wants.

To put me in the shoes of a firstline worker, or someone who directly engages with customers and a company's product, Microsoft filled a hotel room with large and scary-looking factory equipment. Soon after I stepped in, there was a loud crash and everything ground to a halt. A quick glance at a status panel showed me an error code that I didn't recognize. Typically, I'd have to dig up an instruction manual to figure out what's up. A faster option? Just use Remote Assist.

After putting on a HoloLens, I rung up my colleague Kate using Microsoft Teams, who had much more expertise in the faulty machines. She first appeared in a window right in front of me, but since that made it awkward to see the machinery, I found a blank spot on a wall and said "Remote Assistance Pin" to get her video feed out of the way. I moved over to the status panel, and she quickly figured out what was up: there was an issue with a nearby belt machine, as well as some power wiring in another panel. Time to get to work.

Kate walked me through every step of the repair, since she had a clear view of what I was seeing through the HoloLens. Upon opening the first machine, the issue was obvious: the belt was loose and hanging off of its pulleys. Kate had me raise the tension lever, which gave me access to the belt, and she drew a few arrows in AR to show me how to reattach the belt. I followed her instructions, secured the tension, and hit a button to test the belt. Everything worked smoothly, naturally.

The next machine was a complicated bundle of wires and connections. To make sense of it, Kate pulled up a diagram from the manual and sent it to my HoloLens, which I then pinned on a nearby wall. After a few glances back and forth, it was clear the problem was a single burnt out wire. After replacing that, I headed back to the status screen and everything looked good to go. I hit a large button, and the machines were back up and running.

Clearly, without Remote Assist, fixing the machines would have been much more complicated. Microsoft says it can reduce repair times for issues like this to just a few hours, rather than the days or weeks it might take to fly in an expert. Mostly, I was struck by how seamless the experience was. With HoloLens, I was able to keep both hands free, while simultaneously showing Kate everything she needed to fix the situation. And thanks to augmented reality, I also had access to diagrams and manuals right where I needed them.

To demo Layout, Microsoft set up another hotel room to look like an empty factory. It was a mostly open space, with a large pipe in one corner and power conduits on the floor. My role was to figure out the best way to arrange new equipment on those floors. Some machines had to be near power, but couldn't be sitting directly over the conduits. And another had to fit right into the corner, where the pipe might get in the way. Normally, designers would use 2D blueprints to configure equipment like this. But of course, that's not the best solution when you don't know the exact size of the machinery, or if those documents don't reflect everything in the room. So, once again I slipped on a HoloLens and got to work.

As soon as I put on the headset, which was already running the Layout app, I saw a grid of AR lines covering the factory floor. Clearly, they were meant to help with placing the virtual machinery, which were sitting in an augmented reality box near the ceiling. With a flick of my index finger (the HoloLens Air Tap gesture), I selected one device to make it follow my gaze. Then, I looked towards a spot beside a power conduit and made another Air Tap gesture to secure it. Since the AR model matched the equipment's actual dimensions, I could walk around to make sure it fit exactly where I needed it to.

As for the second piece of equipment, I had to maneuver it right beside the pipe in the corner. Once I set it in place, though, I realized that there was no way to access it. The only option was to Air Tap the machine again, and rotate it by 90 degrees. Luckily, it still fit in the corner. That issue was something a a designer could easily miss, especially since the pipe (which was a remnant of older equipment) didn't show up on the blueprints.

In many ways, Layout and Remote Assistance both fulfill the original promise of HoloLens. They tap into the power of augmented reality, and they each let you do things that wouldn't be possible with any other combination of devices. While HoloLens might not seem as exciting as it used to, thanks to consumer VR and the impending arrival of powerful AR glasses from Magic Leap, it's matured into something that could be genuinely useful for a huge portion of workers. Ultimately, that means a lot more than just buzz.

Click here to catch up on the latest news from Microsoft Build 2018!