Playing with Tech: ZED Mini

A couple months ago a few team members headed out to Augmented World Expo (AWE). They had a blast reviewing emerging technologies in the AR/VR space, like exploring an immersive environment in VR on a 360 treadmill.

-- CrossComm's lead immersive app developer Mike Harris showing his skill in Skyrim VR --

One device in particular, the ZED Mini, was so impressive in its capabilities they bought a couple to bring home with them. The ZED Mini is a camera you can mount to VR headsets (like the Oculus Rift and HTC Vive) and then map your surrounding environment through depth sensing technology.

In a few short days after the conference Mike built a working demo of a passthrough AR experience. We're excited about these types of technological innovations because of they open the door to increasing collaboration at distance. Imagine a group of engineers hundreds of miles apart reviewing a physical prototype and adding annotations in a shared space for future revisions. Sounds far fetched, but that reality can exist today.

The makers of the ZED Mini, Stereolabs, were keen on the demo and wrote up a neat article about it on their blog. Check out the story below or on their site and let us know if you want to try it out!

Developer Builds Amazing AR Meeting App in One Weekend with ZED Mini

Days after this year’s Augmented World Expo, an attendee who bought a ZED Mini sent me the most interesting meeting request I’ve ever received.

Mike Harris is the lead AR/VR developer at CrossComm, a leading mobile, web and AR/VR development studio in North Carolina. Right after AWE, Mike built a multi-user, AR drawing app with the ZED Mini. So when Mike asked for an online meeting, he sent CrossComm’s new app. Not a boring phone number or Skype handle. How could I turn him down?

The app uses an HMD-mounted ZED Mini, using its pass-through AR features to put virtual objects in your real surroundings. In this case, the “virtual object” represented a real person: Mike was suddenly standing on our San Francisco office behind my colleague, Victor.

A Visitor In the Office

His body wasn’t visible – I saw his avatar, made up of a floating HMD model and two controllers. But it felt like he was really there. The app networked his every movement in real-time, so every nod, shrug and gesture made it clear his avatar was a real human being. I knew he was slightly taller than me, and that he was in an energetic mood. As we shook hands, it was easy to forget he was in the other side of the country.

Through the app’s built-in VOIP, Mike explained to me that head and hand movements are enough for social presence within XR. He once gave on the topic where his girlfriend helped demonstrate this. She wore boxes over her head and hands, so the audience couldn’t see her expressions. Then he’d ask her to act “angry” or “excited.” Even with the boxes, dozens of signals make the emotions clear. Her hands would wave, her shoulders would shift, her posture would stiffen or relax. The audience laughed in astonishment as they realized how much emotion she could express through the boxes.

CrossComm’s app also lets you draw in 3D, similar to Tilt Brush or Oculus Medium. This has entirely new uses in a social situation. The first thing we did with this was to “draw” swords and have a little duel. Then, to tell a story, he drew a horizontal square over the top of his real-life desk, so I could see where it was in his room. Later, I drew an arrow to point to where my colleague sat so Mike could give him a virtual hug.

At the end, he explained a feature he wanted to add. He would take advantage of the ZED’s 3D scanning features to quickly share textured objects in your room with other users. He showed me what this would be like by drawing a chair, a bike and some other objects he hypothetically had in his room. It was clear the feature could be transformative; I could point at, pick up, and even modify real-life objects that were thousands of miles away.

Innovation Made Simple

It turned out that He built the app on an existing VR Unity project from before getting the ZED Mini. But that made it more impressive. He converted an existing VR project to AR without having to recreate it from scratch. When asked about it, he said that the ZED’s Unity plugin made this surprisingly simple.

“Adding the ZED was the easiest part of the project,” Mike says. “It was drag-and-drop, plus some trivial changes to the SteamVR plugin to accommodate. I spent way more time getting the drawing lines to collide properly than I did on making it work in AR.”

This bodes well for social AR applications. CrossComm’s app proves that the presence of a person who’s not really there is at least as compelling in AR as it is in VR, where playing with friends has been a key factor to the success of some of VR’s greatest titles. With such an experience being this easy to create, meetings like the one Mike and I shared could become commonplace.

But we’re most eager to see what developers do when they can push AR to its limits. Mike says CrossComm is just getting started with this app. He’s looking for ways to stream 3D assets across the network so users can share their live environments with one another. He’s also experimenting with adding Leap Motion to the mix. Mike’s a seasoned XR developer at the forefront of what’s possible, and his work at CrossComm may yet provide even more chances to move the industry forward. With tools like the ZED Mini, we’re thrilled to see what he and CrossComm does next.