Stereoscopic Vision and Virtual Reality

Stereoscopy is intrinsic to VR filmmaking. “Stereoscopy,” refers to a technique for creating a sense of depth for an image that makes each eye focus in harmony (the way binoculars do). The term comes from the Greek word στερεός (stereos), which means “solid” and σκοπέω (skopeō), meaning “to see.”

Essentially, human vision can be described as combining two perspectives–those from the right eye and left eye–into a single image. The bi-zonal input allows the brain to perform computations in milliseconds to determine distance, speed and dimension. In much the same way, stereoscopic video is designed to give each eye information to enrich a viewing experience with depth and dimension.  

Anaglyph glasses create fuzziness and degradation of color. Polarized lenses don't inherit this problem.

For 3D filming, two cameras side-by-side capture an image from a different perspective. The difference is called parallax. By showing a different image from the left camera to the left eye and another for the right, the brain perceives lifelike depth.“Cyber-sickness,” similar to motion-sickness, can happen when there is a visual mismatching of the images.

For 3D viewing, glasses are used. In the past, this was done by creating a red filter and cyan filters; one color for each eye. Now polarisation filters are used. The camera projects one image that polarizes vertically, and another polarizes horizontally. For traditional media or more specific cinema, stereoscopy is often seen as an add-on to the experience.

In VR video on the other hand, stereoscopy is essential. As we are creating a virtual world, we need to make it as immersive as possible and depth is an important part of that. To create depth in virtual reality, we use the same technique as in stereoscopic filming. Cameras used for 360 filming are separated into left and right cameras or sometimes algorithmically combined to create two panoramic images; one for the left eye and one for the right eye, with the same perspective difference to create a perfect parallax effect.

Multiple cameras and the resulting stitching workflows ( gluing the camera outputs together ), are at the heart of the challenge for creating virtual reality video production.  Whether the production is monoscopic or stereoscopic, multiple cameras are necessary to create a 360 image. The overlapping image sections that are captured produce parallax when put on top of each other. When the parallax effect becomes extreme, it can cause stitching errors and/or cybersickness. ( as shown in the example below where a person is standing to close to the camera ).


An example of extreme parallax.
A technical overview of the situation. This can be caused by an object to close, or the camera's too far apart from each other.

Images without parallax errors can be made with monoscopic panoramic photography, but it in 360 video, simultaneous shots with multiple camera makes avoiding parallax difficult. ( for example when a person walks over from one camera into the other ).

For stereoscopic VR videos, however, parallax is necessary for the depth effect. Parallax is not possible with cameras directed towards one eye or image. But when the two eyes are engaged, we need some parallax between the cameras. For camera rigs with multiple cameras, the goal is no longer parallax removal, but parallax management. The design of a VR camera array aims to keep all the cameras as close together as is physically possible.

The bigger the eye separation offset becomes the smaller the world will look ( and a more significant chance on parallax errors).

The all-in-one 360 video systems are better at this. With these small cameras and systems, we are able to place the lenses roughly the same distance apart as our eyes. ( Small cameras also means small sensors, which creates different drawbacks ). This creates the correct depth feeling instead of creating a miniature world effect which happens if the lenses are too far apart.  For decent stereoscopic rigs, you need at least six cameras, three cameras for each eye. With only two cameras back to back it is not possible to create the desired parallax difference.

Below you find a collection of how variated or different VR/360 video systems can look.

Image © by Radiant Images.