How to enjoy stereo photos using a smartphone and a cheap "VR headset"

  • Posted on: 6 January 2018
  • By: hapebe

As a long-term fan of stereo photography (i.e. creating a true 3D perception by using separate photos of the same scene for the left eye and right eye) and proud owner of a Fuji FinePix REAL 3D W3 (which is able to capture such images; and also video clips), I've bought a very cheap set of "VR glasses" last year. It is basically just a plastic box where you can insert your smartphone and it has two eyepieces, so your right eye sees the right half of the display, and the left eye the other half. It is somehow compatible with the famous (?) "Google cardboard" VR set.

Because once your smartphone has gone into the box and the whole thing sits on your face you can no longer use touchscreen controls (and also rather none of the hardware buttons), I thought it might be nice to create slideshow movies of sets of holiday pictures, so I can start them once and enjoy the 3D photos without need for further interaction.

Turns out this is quite a lengthy process - at least the way I decided to do it:

  1. Because you need to start video playback first and then insert the phone into to VR set, I wanted to prefix it with an intro (12 seconds) before showing the first important photo - and as an exercise I chose to use POVRay to create a rendered 3D scene with animation for it; and again (of course?) using two camera perspectives to create a 3D movie.
    • Arrange the scene and two cameras; among other object it comprises of a "picture frame" that plays back normal 2D photos during the intro.
    • Create a slideshow movie of 12 seconds length. (I used Corel Video Studio, but you can certainly use a plethora of other software just as well.)
    • Convert the slideshow into single frame JPEGs (so they can be used as map source files in the POVRay scene) - VirtualDub did the job.
    • Render the 3D scene along a timeline of 288 frames (12 seconds at 24 frames per second)
    • Convert the stream of rendered frames (PNGs) into a video clip - I used ffmpeg to create an MP4 video stream. See code snippet 1 below.
  2. Use the marvellous Stereo Photo Maker to align the stereo images (coming as MPO files from the camera) and save them as "Side-by-side" JPEG files.
  3. Extract the left and right movies from original camera AVIs of the Fuji camera.
    1. Split the files, and because Corel Video Studio could not handle the plain copies of the MJPG streams, save them as uncompressed / raw video. Only the left video gets the audio track. See code snippet 2 below.
    2. Combine left and right movies into an extra wide side-by-side movie clip. I had to shift the right movie by one frame and thus delete the first frame of the left movie to make them match...
  4. Create the actual movie - again, I used Corel Video Studio:
    • Insert and position left and right eye intro video clips
    • Insert sequence of side-by-side stereo still images and side-by-side video clips. Each still image got a duration of 7 seconds - you might want to allow even some longer if you use very "challenging" 3D images, i.e. scenes with a very close foreground and a very distant background. Those scenes might need some eye and brain work! I decided not to use transition effects between the pictures, because that might hurt the brain if it involves motion / rotation / scaling; and it might hurt the brain differently if it involves (cross-)fading. Simple fade-to-/fade-from-black might have worked well, thought.
      • Actually I tried shorter display duration and transition effects for some white water images close to the end of the movie - it worked quite well, I think.
    • The last clip actually gets a fade-to-black effect, so it gives the viewer a hint when the end is near and (depending on the behaviour of your smartphone's video player) you might want to shut your eyes and remove the VR set before the display returns to some kind of none-SBS mode.
    • Export the movie without scaling, using quite HQ settings:
      • AVC (H.264) 4K (3840x2160px) 24p, max. 50 MBit/s
      • Dolby Digital Audio 48000 Hz, 256 kBit/s
    • Scale it to the native display resolution of the target device using XMedia Recode, in my case:
      • Lanczos rescaling to 1920x1080px, MP4/H.264, q=25
      • AAC audio, 48000 Hz, 128 kBit/s

After going through the whole process once, I realized that it would be quite easy to repeat the steps for another subject (holiday memories, in my case). In the meantime I finished two such movies (one for Sassnitz and the awesome chalk cliffs of Jasmund on Rügen island, one for the scenic beauty of the Yorkshire Dales), and at the time of writing I'm working on the third one (river Mosel). Please find the  movie below:

 

Code snippet 1: Create a video from a PNG image sequence - make-mp4.sh

#!/bin/bash
ffmpeg -framerate 24 -i hapebe-intro%03d.png -c:v libx264 -profile:v high -crf 10 -pix_fmt yuv420p intro.mp4

 

Code snippet 2: Extract left and right movies from cameras AVIs - split-fuji-3d-RAWVIDEO.sh

#!/bin/bash
# this script generates usable input for Corel Video Studio 9 (lossless!)
ffmpeg -i $1 \
    -c:v rawvideo -pix_fmt bgr24 -vtag "DIB " -c:a copy $1.left-audio.avi -map 0:2 -map 0:1 \
    -c:v rawvideo -pix_fmt bgr24 -vtag "DIB " -an $1.right.avi -map 0:0

 

As a reminder for myself (please excuse me, it does not bring added use to you as a reader, I guess...) - the archived locations of working files:

  • $HOME/POV-Ray/v3.7/scenes/3d-intro-mosel-2017
  • $PHOTOS/2017-09-03to08 Mosel
  • $MEDIAXL/video/2017-09-03to08 Mosel
  • $MEDIAXL/video/2017-05-06to14 London und Ilkley, Yorkshire Dales/documentation.txt (further tech docs)
Tags: 
graphics
log
how-to
3D
projects