If you’re familiar with the Marvel Cinematic Universe, you are likely knowledgeable about the Multiverse. It’s the theory that we currently live in one version of multiple universes. In each alternate reality, a whole different set of circumstances are at play in our lives, and we are possibly very different people in each. If you can get comfortable with the idea that there are multiple copies of you in the Multiverse, you can easily perceive the intricacies of many versions of yourself existing here and now in a Reality-Virtuality Continuum (RVC).
The State of Reality: Part II, The XR Glossary
We talked briefly about the various RVC realities in a prior blog post, The State of Reality: Part I, A Brief History of Augmented and Virtual Reality. But in this post, we’ll dig a little deeper and provide you with a glossary of terms that help make sense of the worlds you can explore along the RVC.
What is the Reality-Virtuality Continuum?
The RVC is a decades-old academic term describing the spectrum of immersive technologies extending our physical reality. It includes things you might have already heard of, like augmented and virtual reality.
Placed within the extreme portions of the continuum is the real environment (our actual physical space), and the virtual environment (a purely artificial world which often requires a headset to experience). Everything else in between is considered “mixed reality.”
Source: Milgram and Kishino, 1994
Now that more than 20 years have elapsed since the RVC was conceived, the traditional use of the term Mixed Reality no longer holds broad appeal. Today, the terminologies found in the RVC are increasingly refined. While there is still a continuum, new applications of immersive tech blur the distinctions between realities. Just to give you an idea, you can see some great modern-day examples of reality-breaking AR/VR platforms in CrossComm’s Reality Recap video series.
With the RVC terminologies continuing to evolve in real-time, you’re probably wondering– What are the types of realities that are in existence now? To keep it simple, below is a Reality-Virtuality list of terms.
Reality-Virtuality Terms
Virtual Reality (VR)
VR is an experience that puts a user entirely in another world or reality, replacing their physical visual reality with a virtual one. Typically, VR requires the creation of an artificial 3D space set around a user. A user can experience games, movies, events, and even exotic locations as if they were really there. These immersive experiences often require a VR headset or head-mounted display (HMD).
Augmented Reality (AR)
AR is an experience that modifies or “augments” a user’s perspective of their reality. AR is similar to VR except with some key differences. Unlike with VR, a user can still see their physical reality when experiencing AR, but it is overlaid with computer-generated visual content.
Web-based Virtual Reality (WebVR)
WebVR are virtual reality experiences accessible directly through a web browser. It allows users to immerse themselves in VR without downloading additional software or buying expensive devices.
Web-based Augmented Reality (WebAR)
The latest in AR technology, web-based AR uses web browsers, rather than apps, to deliver an AR experience.
Mixed Reality (MR)
Much like AR, an MR experience overlays artificial content onto the real world. The difference is that the user can interact, in real time, with the overlaid objects. Essentially, today’s use of MR allows for interactive digital objects to be anchored in the real world. In contrast, AR objects present information but lack the interactivity that MR provides.
Extended Reality (XR)
In the modern day, XR has been used as an umbrella term for VR, AR, and MR. Additionally, the term has been a go-to to encompass technology found in the entire spectrum of the RVC. For example, it has also been used in reference to human-machine interactions generated by any computer technology, such as wearables.
Metaverse
A glossary about reality can’t exist without mentioning the Metaverse. It was first coined by a science fiction author to describe a space where virtual, augmented, and physical reality meet. Recently, Meta CEO, Mark Zuckerberg, described the Metaverse as a virtual environment where all forms of media converge.
Reality Reality (RR)
You are probably confident that you know what this is, but it’s worth it to recap just in case. Cambridge dictionary defines reality as, “the state of things as they are, rather than as they are imagined to be.”
What Makes the RVC Experience Feel Real?
Example of an augmented reality experience developed by CrossComm
As you survey your reality you might ask yourself, should the artificial 3D objects created in these RVC worlds be considered imaginary, despite the fact we can actually see them? There is really no right answer. Even now, this question leaves so many confused about how to define reality itself.
One interesting way to assess your reality in the RVC is to consider the degree you are immersed in an AR or VR experience. Immersion is the degree to which a person feels present in an environment, virtual or otherwise. It has been debated as either a psychological state or objective property of a technology. As a psychological state, you can think of it as the mental absorption where a person can be deeply engaged in an experience. One example would be reading a book where the reader is immersed in the narrative.
When it comes to defining immersion as a property of technology, the sensory experience becomes an essential part of the definition. Often, technology-focused definitions describe immersion as multi-sensory stimulation, where sound, visualization, movement, interactions, and design are used to bring together the experience of being in a real environment. To that effect, immersive tech developers and designers strive to build place illusion, where the feeling of reality is produced by creating vivid sensorimotor experiences in an artificially-produced 3D world.
In the next sections, we provide key terms regarding the technology and features related to sight, sound, feel, movement and interaction.
Terms for Creating a Sensory Experience
Head-Mounted Display (HMD)
An HMD refers to a device that can be worn on the head. This could be used to describe any headset, from simple plastic and cardboard arrangements, to more advanced HMDs which contain lenses and sensors that track movement. The term generally refers to VR headsets, but has also been used to refer to AR glasses and visors.
Eye Tracking
Eye tracking is a process used in headsets that tracks a user’s gaze and reproduces the eye’s natural tendency to bring objects into and out of focus. In a device, it can render sharpness to particular areas of a display that the user is viewing.
Field of View (FOV)
The field of view is the number of degrees that can be seen from a given point of view. Most people have an FOV of 200 degrees. Most VR headsets provide approximately 100 degrees in their FOV. The more an experience deviates from the normal FOV, the more it can break the immersive feel.
Parallax
In VR or 3D videos, parallax is the movement of virtual objects relative to a user’s change in point of view. Essentially, computer-generated objects are designed to be perceived as moving when the user moves, and objects in the distance should appear to move slower in relation to objects closer to the viewer.
Stereoscopy
Also known as stereo imaging, it is a technology that creates the illusion of depth by offsetting images separately for the viewer’s left and right eye. For example, in VR headsets, the stereoscopic displays provide the three-dimensional depth by showing a scene that is angled differently for each eye, creating the viewer’s perception of depth.
Ambisonics
Ambisonics is a method to reproduce sound in 360° using a special array of at least 4 microphones capturing sounds from every direction. The sounds that are captured are used with surround sound and 360° videos.
Positional Audio
Positional audio is a technique that anchors a sound to a specific source or environment. This allows for sound to come from the expected position relative to where a listener might be. In XR, this would require that the angle and volume of a sound should move as the user moves through the environment.
Spatial Audio
Spatial audio is another sound technique where the sounds heard by a listener will change based on the position of the user’s head. For example, if a VR headset user turns their head to the right, the volume should increase on the right-hand side while decreasing on the left-hand side.
Haptics
Haptics is a feedback mechanism that simulates what a user might expect from their own movement. For example, when trying to grab a digital object in VR, a controller can signal back a vibration similar to the pressure of holding the object in real life. It produces a sense of realism that mimics touch.
Terms for Movement and Interaction
Degrees of Freedom (DoF)
DoF refers to the degrees a user can move within a 3D space. As an example, VR headsets can offer 3 DoF (three degrees of freedom) and 6 DoF (six degrees of freedom).
In 3DoF, there is only rotational tracking. A user can look around an environment by rotating their head, left, right, up or down, but if they were to attempt to change positions in the environment (i.e., walking or leaning forwards, backwards, left or right) their position within the virtual world wouldn’t change. With 6DoF there is both rotational and positional tracking. A user can look around an environment and can change their position within the 3D space.
Degrees of Freedom refers to the way a user interacts with a virtual environment.
Positional Tracking
Positional tracking is done with a combination of hardware and software that tracks the position of a user. A user’s position is estimated relative to the virtual environment surrounding them. Positional tracking is essential for VR that allows for 6DoF.
Inside-out Tracking
A type of positional tracking used to track the position of head-mounted displays. In inside-out tracking, the cameras and sensors are on the HMD device, which can determine an object’s position relative to a space.
Outside-in Tracking
A type of positional tracking where cameras and sensors are placed in a stationary position and situated to track an object which can move freely around a designated area.
Source: Acer, 2019
Room Scale
Room scale is a term to describe how the positional tracking technology positions the VR environment within a defined real-world room boundary. This allows a user to walk in 6DoF and interact with objects in the VR world inside a room.
Locomotion
Locomotion refers to ways a user can move around their virtual environment. There are typically three types: perambulation, teleportation, and transportation. Perambulation is the use of controllers to track movements and reflects these movements into the environment. Teleportation allows a user to point and click at a location and travel quickly to the pointed spot. Transportation makes the user a passenger in a vehicle which moves in a predefined path and does not allow the user to alter the predefined movement.
Hand Tracking
Some XR headsets are able to track the hand gestures of a user. This allows users to interact with the virtual world using their actual hands as the controllers, which facilitates more familiar hand gestures and movements.
Testing hand tracking in Oculus Quest
Terms That Impact Immersion
Latency
Latency refers to a delay between a user’s movements (or input) and what is being visualized or experienced (the output). It can be caused by technical problems and impact the feeling of immersion a user experiences. As technology improves, latency will be less problematic.
Frame Rates
Frame rates are the rate that an image is replaced by another image in a monitor. It is typically characterized as fps (frames per second). The higher the frame rate, the smoother the visual experience. To maintain the feeling of immersion, VR platforms aim to have at least 90 fps.
Resolution
Represented by the number of pixels, resolution describes the degree of detail an image can hold. The higher the resolution, the sharper the images. Screen size can additionally impact the sharpness. For example, a small enough screen can make a low-resolution image seem much sharper than a large screen would with the same pixels.
Tethered Headset
Tethered headsets are HMDs that are physically connected to a computer using a cable. Because they are physically connected to hardware that has a graphics card, these headsets can provide high fidelity realistic experiences. The downside is that the connected cable limits motion and can become a physical reminder of the physical world outside of the virtual one.
Volumetric Capture
Also known as “volumetric video,” volumetric capture involves the capturing of multiple angles of an event by surrounding the subject with precisely-synchronized cameras and sensors. The video is then processed and reconstructed via software. This method can be used to create 3D models in AR and VR.
360 Degree Video
360 videos are captured with cameras that allow for a 360 degree point of view of a scene. The entire scene in VR can appear to surround a person without breaking immersion because the entire line of sight is filled.
180 Degree Video
180 degree videos show a scene that cuts off at the edges of a viewer’s line of sight. In comparison to 360 video, 180 reduces the amount of movement needed for a viewer to watch a scene.
Marker-Based Augmented Reality
A visual symbol is picked up by an AR device’s camera, and it creates a spatial anchor or reference point by which an app knows where and how to place computer generated content. In essence, the scanning of the marker triggers the augmented experience, which will appear on the device.
Markerless Augmented Reality
AR that uses an understanding of the world around a device in lieu of markers. In this instance, the physical features that can be found in the real world are what is registered to overlay the 3D content. Simply put, it is the real-world environment’s features that are registered rather than an identifying marker, such as the ones required in marker-based AR.
Simultaneous Localization and Mapping Technology (SLAM) Tracking
SLAM is a form of markerless AR where the tech is developed to understand the features of the physical world. AR apps can then recognize the real world and overlay digital augmentations.
The Evolving Reality-Virtuality Continuum
Immersive technologies are born from some very interesting history, from a filmmaker’s dream to create a clunky machine that engages all the senses, to current headsets that can transport you to a realistic virtual world. Today, these technologies continue to grow and evolve, expanding the reality-virtuality continuum, and possibly making some of us question the meaning of reality itself.
We hope the key terms highlighted in this glossary help you make sense of the immersive tech landscape. While we couldn’t capture all the terminology that highlight AR, VR, and everything in between, we’ll make sure to add more to the glossary as the technology evolves.
If you’re interested in taking the next step in your XR project or idea, we would love to hear from you and discuss whether CrossComm would be the right technology development team for you. Tell us more and receive a free assessment from CrossComm.
