In order to get panoramic video working with the RIFT, it might be possible to import the video footage into the Unity engine and play the video on a spherical plane with the POV in the center. Something like this is already being implemented with FOV2GO scripts and the iPhone / Android, but it can only be done with a stereoscopic panoramic still image, not video:
http://projects.ict.usc.edu/mxr/diy/fov2go-developer/But I'm not sure if its physically possible to do both stereoscopic 3D and 360 video capture, given the fact that you need SBS cameras to record the footage and the cameras would inherently block each other partially.
What about positional movement? If real-life 360 video were to be a supplement for VR in a photorealistic graphics engine, it could always be explored with orientational movement. However, due to the nature of being pre-recorded video, it would be physically impossible to move directionally within the "environment" (ie, be positionally tracked) like you can in a real-time engine. Unless the video wasn't pre-recorded and you were experiencing the POV of a moving robot!
