We’re fresh back from the epic Digital Hollywood conference / rolling party in LA, and chock full of ideas of how to integrate classic cinema techniques with our native videogame tropes.

See, 80+% of the participants at the conference were from traditional media backgrounds — music, TV, film. And while VR was absolutely the hot topic of the show — as it was for CES, GDC, and NAB — there was as much confusion as there was excitement about the commercial and artistic promise of this brand spanking new medium where cinema meets videogames.

One of the key findings on our part was a genuine need to integrate cinema techniques, aka linearity, composition, color and storytelling —  into our hyper-interactive realm of videogame design. Thus began our investigations. What exactly does it take to make full, HD, 3D, 360 captures of real-world environments?

We’ll get into more details later, but for now I want to spell it out, if only for technical humor: It takes a:

  • 12-camera
  • 4k
  • 360°
  • 3D stereoscopic
  • live capture stream
  • …stitched to form an:
  • equirectangular projection
  • over / under
  • 1440p
  • panoramic video

Say that 12 times fast. Oh, and be ready for handling the approximately 200GB / minute data stream that these rigs generate. Thank god for GPUs.

What does that look like in practice?
Like this:

A still frame from a 360 stereoscopic over/under video. Playback software feeds a warped portion of each image to each of the viewers eyes.

This format has a name, which wasn’t written by marketers : its called

equirectilinear 360 over-under stereoscopic

It looks like two mercator projections stacked atop one another in a 2:1 aspect ratio. Common resolutions are 4096×2048 and 8192×4096. While these resolutions are massive compared to standard HD 1920×1080, remember that the audience at any one time is only able to perceive a significant sub-view of the entire 360 field of view. So even a massive 8192 x 4096 gives us something slightly better than 720p resolution once all the conformations are completed.

Here’s that fast math :

  • 8192 x 4096 / 2 eyes (top bottom)= 8192 x 2048.
  • if 8096 = 360 degrees panorama, and 2048 = 180° vertical FoV,
  • the HMD generally has, at best an FoV of 90° wide and 60° high.
    So that’s 1/4 the horizontal rez, and 1/3rd the vertical.
  • Leaving us final “to the eye” resolution of:
  • (8192 / 4) x (2048 / 3), or
  • ~2048w x 682h = ~1.4 megapixels
  • which is somewhere between 720p and 1080p HD quality in terms of pixels.

Next question :
how do you capture it?

With something like this:

360Hero GoPro stereo 360 rig

Or, if you’re really high-budget, this:

Red Dragon 6k 360 3D stereoscopic capture rig by NextVR_Rig1

Though personally, we really prefer the sci-fi aesthetic:

jaunt-sci-fi-rig-header

What do you think? Is the realism of live capture worth the trouble? Would you prefer “passive” VR experiences that transport you to hard-to-get-to real world places and events, “interactive” experiences more akin to xBox and PlayStation games, or some combination of the two?

Join the conversation below

 

Leave a Reply

Your email address will not be published. Required fields are marked *