The Biggest Announcements from Oculus Connect 6
Last month, we were in-house for both days of Oculus Connect 6, the sixth annual VR conference hosted by Facebook. Some significant updates were announced for the new (and quite popular) Oculus Quest. Facebook also announced plans to expand the social and business use cases of virtual reality. Here are the key takeaways:
Facebook is Firmly Aimed at AR
For veterans of Oculus Connect, the industry’s leading VR-only conference, one of the most notable features of this year's gathering was just how often AR was mentioned in the same breath as VR. This was evident in the somewhat forced language used both during the keynote and in the titles of conference sessions. It is no mystery that Facebook is heavily invested in getting a toehold on the heavily predicted market for consumer AR glasses, and sees current-gen VR as an important step towards that future.
To this point, they announced the first step of that process, LiveMaps, which is a “multi-layer representation of the world.” This will not only include information regarding the AR glasses wearer’s position in space, but also semantic understanding of the objects within that space.
Much like the often-referenced idea of an “AR Cloud” or “mirror world,” LiveMaps would allow for the sharing of spatial understanding and augmented content between AR users. And while Andrew Bosworth, Facebook’s VP of AR/VR, stated that Facebook’s VR glasses are still “a few years out,” it is clear that they are leveraging the Oculus ecosystem and developer base to lay that foundation.
Social VR is Crucial to Facebook’s Strategy
Another big announcement at this year’s Oculus Connect is Facebook Horizon, Facebook’s newest (and fifth) attempt at developing a foundational social app for VR. Horizon will be launching early next year. For being the premier social media juggernaut, Facebook has repeatedly struggled with developing a truly compelling social VR experience.
Facebook is banking on two key features to help Horizon thrive in a way that their former forays into social VR did not:
Horizon will allow for shareable, user-generated content that can be created in VR and shared with others. This will allow users to create new worlds by moving, rotating, and stretching shapes, and will include the ability to do “light” visual scripting to create simple interactions at a level “somewhere between Scratch and Blueprints.”
Horizon will focus on being a “welcoming and inclusive community” with an emphasis on the promotion of “safe and positive” interactions through blocking/moderation tools, as well as Facebook-associated “guides” within VR to help users navigate the new environment.
This focus on social VR was emphasized by Michael Abrash, Chief Scientist of Facebook Reality Labs, who, per-custom, ended the first day’s keynote with a look to the future of VR. Comparing VR to the personal computer, Abrash mentioned that the “Quantum Leap” that led to the explosive proliferation of the PC has yet to come for VR.
In his view, this leap will likely come in the form of a reliable and intuitive virtual collaborative workspace. Abrash indicated that they are actively working, prototyping, and testing this application internally as a proof-of-concept. He showed several advancements that will facilitate social interactions in VR, including improvements in real-time facial tracking for photo-realistic avatars, as well as impressive virtualization of actual environments through photogrammetry.
The Quest Will Soon Double as a PC-Based Headset
With its release this year, Oculus Quest has set a new standard for ease of access to VR by providing a quality, self-contained VR experience in a single headset. However, given the limited processing power of the headset (it incorporates the same Qualcomm Snapdragon 835 Mobile Platform used in many of last year’s high-end smartphones), the most realistic and cinematic experiences were still reserved for headsets that were attached to a high-powered PC.
Enter Oculus Link, a software upgrade to the Oculus Quest headset that will allow it to function as a PC-based tethered headset. John Carmack, Oculus’ CTO, said this feature will essentially turn the headset into a “Rift Q.” Facebook accomplishes this by streaming video from a high-powered PC to the headset via a USB-C connection. Oculus Link will reduce streaming latency by more aggressively compressing portions of the image that will appear in the viewer’s periphery, and slicing up the encoding/transmission/decoding of the frames.
These technological accomplishments are important because they pave the way for a low-latency, wireless streaming solution sometime in the future. This possibility was indicated by Carmack in his day two keynote, where he stated, “Clearly we’d like this to work on Wi-Fi eventually.”
Facebook will be releasing a software update allowing for this functionality in November, although early adopters will need to provide their own USB-C cable until Facebook releases its official fiber optic, 5-meter, Oculus Link cable early next year.
Hand Tracking is Back, for Better and Worse
Anticipated by developers for some time, Facebook finally announced a software update for the Oculus Quest that will allow the headset to track users' hands. Releasing early next year, the update will use the headset's existing cameras (currently used for spatial understanding, controller tracking, and pass-through video) to also estimate the user's hand positions and poses in real-time.
While this is a clear win in the move towards reducing friction for general VR usage, there are still some reasons for concern:
Concern 1: Tracking
Quest tracking will have some of the same problems as previous hand tracking solutions, such as the Leap Motion, Hololens, and Meta 2. First, it will only work in the area directly in front of the user. Second, it will introduce a slight latency between the movement of the user's hands and their representations in virtual reality. And finally, it will still be susceptible to tracking errors when the hands are viewed from odd angles or when they overlap (occlude) each other with respect to the user's point of view.
At CrossComm, we’ve worked extensively with hand-tracking-based user input in both AR and VR and can attest to the time-consuming challenges these limitations introduce when trying to create reliable and responsive XR interactions. As a result, these challenges might pose a hurdle for many of the experiences needed to drive user adoption.
Concern 2: Lack of Precision and Feedback
One of the main problems with hand-tracking-based XR interfaces is the absence of physical keys, buttons, or surfaces to precisely register user input while also providing tactile signals (commonly referred to as haptic feedback) back to the user.
A common trick used for such interfaces is employing a pinch gesture to register selections. The advantage of this approach (coined cleverly by Facebook as “self-haptic feedback” because the user can feel their finger and thumb touch when a pinch is executed) is that it provides a specific gesture that can be used for input which the user can feel. The disadvantage is that this reduces the types of input available to developers to a single button, while also adding a layer of abstraction to the user’s ability to use their hand in a virtual space. This undermines the value and intuitiveness of allowing the users to interact with their hands as opposed to tracked controllers.
Concern 3: User Interaction Expectations
Developing for hand-based interactions in VR requires a wholly different approach than developing for tracked controllers, akin to the difference between designing for an Xbox controller and a mobile touch screen. Many developers will likely not have the resources to design for both paradigms and/or may be creating experiences that only work within one or the other.
For example, the dynamic, precise tracking required for the blockbuster VR title Beat Saber would be impossible using the Quest's hand tracking as it currently exists. Conversely, an application that responds to subtle changes in hand gestures would not translate to the limited hand poses that can be replicated using Oculus Touch controllers.
To avoid the pitfalls of confusing consumers and segmenting the user base, it will be crucial for Facebook to manage user expectations and effectively communicate the differences between the interaction paradigms to users on a per-app basis.
VR Is Not Just About Games
Originally announced at F8, Facebook further clarified details regarding the rollout of Oculus Business—an arm of the company focused solely on meeting the needs of developers and businesses wishing to use VR for non-gaming purposes. This is a direct attempt to address the limitations of the existing Oculus platform in non-gaming contexts. For example, currently, there is no way to block users from having unrestricted access to the Oculus store and web browser.
Oculus Business will allow managers to create custom home environments that only allow for launching into manager-approved applications, or directly into a single predefined application, and will provide “enterprise-grade” security structures and customer support.
Further, on the heels of the catalyzing UCLA study showing a 230% improvement in performance for a set of knee surgeons prepared with VR training, Johnson and Johnson also shared key findings from an Imperial College London study regarding hip surgery. In its keynote presentation, Johnson and Johnson stated that 83% of residents that trained in VR “completed surgery in the lab environment with minimal guidance” versus 0% of those trained via traditional methods. These real-world examples of the added value provided by VR-based training will undoubtedly drive further activity in developing non-gaming VR applications.
We witnessed this growth first-hand while attending the first-ever Oculus Education Summit, just prior to Oculus Connect, in which over a dozen presenters described results from numerous non-gaming applications of VR. These were some of the use cases presented:
a vaping prevention application for high school students
remote architectural collaboration
the capturing of oral stories and traditions amongst indigenous populations in Northern California
dental surgical preparation using patient scans
advancements in collaborative virtual learning spaces
It’s not surprising that some of Facebook’s biggest announcements at Oculus Connect 6 (such as hand tracking and Oculus Link) pertain specifically to the Oculus Quest, especially when considering that Quest software accounts for more than 20% of the Oculus store’s lifetime revenue, and that the headset itself has the highest retention (repeat use) of all Oculus devices.
Simultaneously, Facebook is focused on expanding the possibility of VR, with sights on making the technology more social and more attractive to businesses. But despite the fact that Facebook is seemingly doubling down on its investment in VR, it is clear that AR will also be a large part of the company’s roadmap, which has us super excited for Oculus Connect 7 next year.
Stay tuned for Part 2 of our Oculus Connect 6 recap, as we discuss how Facebook is making it easier to create and share XR experiences.
