Gopro HD and the rift.

[Namielus]

The gopro hd cameras can be mounted in stereo and record high fov 3d video.
The picture is distorted/fisheye, and the distortion is toward the viewer.

As I understand, the rift pre-distorted image is also toward the viewer, so my question is how close is
the gopro cameras distortion to the required pre-distortion?


Any input?

[Fredz]

With a 30mm separation I'm not sure the Gopro would be the best cam to use for the Rift.

[Namielus]

care to elaborate?
You mean the distance between the lenses should be further apart or what?
Sounds easy to mod

[DFP]

These recently went on sale.
http://www.roxy.com/product/index.jsp?p ... p=dshopbot

GOPRO HD HELMET HERO CAMERA#CHDHH001
Regular $299.99 Sale $137.99

GOPRO HD SURF HERO CAMERA#CHDSH001
Regular $269.99 Sale $156.99

Free shipping via Purolator Ground Service
13% sales tax

Cheapest we've ever seen for a GoPro package! Note this is not the new HERO2, but the improvements in Hero2 don't justify double the price for a lot of people.

[Fredz]

You mean the distance between the lenses should be further apart or what?

Yes, since the Rift uses a separation close to the average human separation (around 65mm) you wouldn't be able to experience orthostereoscopy when using a Gopro 3D system.

Sounds easy to mod

I'm not sure building a waterproof housing with a precise alignment for the cameras is going to be that easy. It already cost $99 for the Gopro and it's mass produced, so I think it'll be significantly more expensive to do it yourself.

Also you wouldn't have any advantage over using StereoData Maker with supported cameras, you could even find cheaper cameras since they would only require a 800x640 capture resolution (at 60 fps preferably).

[Namielus]

Hey!

I dont really need it to be waterproof, just (maybe) water resistant. And ive built water resistant houses for dslrs before with button functionality, so I cant see this being impossible for me.

I basicall only need the sync cable, do you by any chance know how long it is?
Its longer than it needs to be in the 3d-housing I think.
http://cdn2.gopro.com/wp-content/upload ... a_0612.jpg
Tho Im unsure if its 65mm+
And how many wires is there, for a potential modification? Ive extended non-propietary usb-cables before,
cant imagine more wires than usb just for time-syncing.

Please humour me here; Lets say for a fact I can make the separation/angles work and maintain
water resistance Also, make a head mount etc.
Will any of the field of view modes and the fisheye barrel distortion be close to what the rift needs?

170º Wide FOV (Including 1080p)
127º Medium FOV (In 1080p)
90º Narrow FOV



I could make an adjustable bracket to change seperation/angles.
As I understand it, the perticular angle relative to eachother decide what distance the 3d-effect is most optimal?
I could hook it up to them live on both eyes to twist and adjust to accordingly looking through the rift.


PS: Sound recording will be done with external audio.

[Fredz]

I basicall only need the sync cable, do you by any chance know how long it is?

From the image you posted and by measuring the dimensions directly on the screen I'd say it's around 4.5/5cm long.

The separation is at 30mm when the cams are in the housing, so you'd only need 35mm more to get 65mm. But the cams need to be at a fixed relative height to have their sensors at the same level, I'd say at around 8mm from the images on their site. So the height difference between the two attaching points of the wire should be 50mm (42mm height + 8mm space) or less, let's say 45mm because they're not exactly at the top or bottom of the cams.

With some maths this gives sqrt(4.5^2 + 3.5^2) = 5.7cm, so I think the cable will be too short and you'd need to modify it. You can still try to contact them to have a confirmation about the dimensions though.

[EdZ]

I'm not sure you'd be able to get the GoPro optics to be identical the the RIFTs distortion without making your own lenses. It might be less elegant, but a lot easier, to use a lens with sufficient field-of-view (110deg if you use the GoPro sideways), correct the GoPro's distortion in software to get a rectilinear image, then distort this image to suit the RIFT's optics. It's slower and might have some quality loss (though you're downscaling between the GoPro and the RIFT anyway, so this should be minimal), but it also generalises the output so it can be used with HMDs using different distortions (e.g. future consumer RIFT).

[Namielus]

So these are the configurations I consider all with 65mm seperation.
The bottom one with one camera upside down is where the connectors for the link cable are the closest.






About the distortion.

Im just going to use it for pov-video, why would I need to correct the image to rectilinear first and then distort it again?
Im not going to use the video footage to track anything.
Wouldnt it be enough to distort the image more or less according to what the rift needs?
When passively viewing a video, you can get away with _SOME_ distortion in the final result cant you? For watching a movie and interacting?
I cant imagine it to be that crucial to have absolutely perfect rectilinear end result.

[nrp]

Yep, you could just directly transform from the GoPro's fisheye to the Rift's. You could do it live without much difficulty using libvlc to decode and OpenGL shaders to warp.

[Fredz]

I think the arrangement that would minimize the length of the cable would be this one (seen from the back, lens shown in transparency) :



As for distortion, maybe you could simply set the FOV to something just higher than what the Rift supports (ie. 127° in 1080p) then use the same type of algorithm Emerson and Cybereality used for their interceptor driver.

[EdZ]

Im just going to use it for pov-video, why would I need to correct the image to rectilinear first and then distort it again?

Not really, you could just combine the two transforms into one. Performing one un-distort and one distort has the benefit of a) making the resulting video general for any type of wide FoV HMD (not useful now, but it saves you a bunch of cre-conversion later when more are available), and b) allows you to use the drives designed for the RIFT to do the correct distortion, rather than writing the same code again.

When passively viewing a video, you can get away with _SOME_ distortion in the final result cant you? For watching a movie and interacting?
I cant imagine it to be that crucial to have absolutely perfect rectilinear end result.

One of the big advantages of wide FoV HMDs is 'Orthostereo', where objects in the 'virtual' world appear in the correct scale and proportion. For real footage, this means that what you view in the headset exactly matches what you see in real life. This is a very powerful effect, and having things 'a bit off' might be rather disorientating, like looking at the world through someone else's prescription glasses.

[BOLL]

I am in the process of moving to start studies elsewhere so I cannot spend much time to experiment with this, but I actually recorded a test clip today using two GoPro HDs mounted in a way I came up with when I first heard of the Rift. I wanted to use the full FOV of the cameras to try and record immersive video for use with the unit.

[youtube-hd]http://www.youtube.com/watch?v=KHLi-_PSjj4[/youtube-hd]

Sony Vegas crapped on me when I tried to preview 3D with rotated clips, and I really have no time to pre-render subclips for editing, so I'm not entirely sure stereo matching is good enough. I also happened to get way too high convergence as I don't have a real rig but held the cameras towards a flat surface, held together at the center by a bolt, but I tried to adjust the parallax to counter that, but it was almost done blindly as again could not preview.

I exported the clip cropped to 720p to get it to work on YouTube, for easy access, it is tagged as 3D but shown side by side it should be fairly much the Rift view. Oh, and for the sake of discussion, the lens separation with this rig is 74 mm. (now when finally looking at the clip, processing was slow, the compression really is horrible... ugh..)

(Oh, also seeing that while I've been exporting/processing lots of new posts appeared, I used the arrangement Fredz posted)

[Namielus]

Looking forward to seeing more of your work BOLL, can you give a quick sketch on how youve mounted them?
Have you thought about how they are angled with eachother?


I found a few clues that Gopro themself actually makes a waterproof housing for 3D with 65mm separation.



With a different type of connector




I threw together a 3dmodel that has exactly 65mm separation to compare, but I was unable to get them in the exact orientation/perspective so they are not 100% aligned.



Still it shows that its pretty darn close to 65mm.
Question is if it works at all in horizontal configuration. I wont know until December.

[BOLL]

Haha, it's almost shameful to show this solution, but here you go.

I'm still wondering if the distortion is at all similar to the Rift distortion, I'd like to see a grid representation of the Rift distortion so it could be used for comparisons. If I remember correctly the Rift uses aspherical lenses, and I would guess that the GoPros uses spherical ones. Not sure about either one though, haha. Optimally aftermarket lenses could be molded just for this purpose, but I'm not sure which business would do such a limited run, in addition to development.

[Namielus]

Lol! Sometimes the simplest solution is the best, and it seems maybe with a little improvement that solution isnt bad at all.


Once you get your hands on an oculus rift, you can mount that in front of you and twist them to get the right angles and adjust then tighten it down.
If you walk around with a laptop in your backpack, you will have the exact recording perspective that ends up in the video.

The distortion is definitely closer to the rift than a normal video, and at glance it seems very similar. So in theory it should require less distortion. Lets hope for a grid pattern that illustrates it well soon, ive seen others ask for it as well.

You should also check out the thread on binaural audio. It will greatly enhance this concept.

[BOLL]

Yeah I am subscribed that thread haha, binarual is awesome, I've listened to the barber sketch before I came here. I commented on one of the clips on YouTube, and Skaven252 (I assume) said he used Soundman OKM II ear microphones, which aren't super expensive. Right now I have zero income as I quit my job back in April (long story) and am about to start studying... but... I'm tempted :P It all boils down to if the GoPros actually works with the Rift without too much hassle. I use Sony Vegas for editing, and it can do a few simple distortion effects, so I guess even if the GoPros are not perfect some adjustments can be made. I guess moving to Premiere or AE would open up more options.

Optimally I would like to have the cameras mounted to a helmet, and to wear ear microphones like the Soundmans. It would add a new dimension to vacation footage for sure ;)

[Namielus]

Do you have some old earbuds? If you have a recording device with an amplifier, and plug earbuds into the mic-input, they work like little microphones. Then you can just find a way to mitigate windnoise, and mount one bud to each ear.

You dont have to spend alot of money to get that effect, especially not for experiments.

[alvarop]

By the time the Oculus Rift is available, gopro is supposed to support live streaming with the wifi backpack. That should get rid of the cable problem, although you might have to reverse-engineer whatever protocol they end up using to get the video without using their phone apps.

[cybereality]

Just don't end up like this guy:

[Dycus]

Tested out the video on my prototype - it looks great! The warping actually seems almost perfect, funnily enough. My only gripe is that the left video is slightly brighter than the right one. It causes a little haziness and discomfort, but it's not too bad.

[Namielus]

thank you dycus!
SUCH good news!

The problems you mention seems easy to overcome in post-production.
I'll be delivering some epic snowboard footage for everyone come wintertime,
if this goes as planned

[BOLL]

Ah, very interesting, thanks for checking Dycus :D Like you say the video is not perfectly matched, the color and brightness differences come from me not using a sync cable but instead just two separate cameras, and I did not do any manual correction as it was a quick job. But that the distortion is a good match is interesting, I only did minor cropping as I had to rotate the clips slightly due to my rig not being fully rigid/straight.

The absolute best thing would be if GoPro could make a special housing including a sync cable for Rift footage, haha, but I guess that is a far fetched idea :) ... or is it?

[Namielus]

I contacted Gopro. They dont seem interested at all, they wouldnt even disclose any information on the distortion.

Can you view your project file and give me as much info as you can? On crop, what setting/FOV did you record at, what model camera etc.

[Fredz]

@BOLL : I just watched your video through my 7X magnifiers from the FOV2GO on a 5.6" 1024x600 display and it's looking very good, the warp corresponds very well to my rig and I'm not experiencing discomfort with the 74mm separation.

Do you have a non youtube-compressed version of your video that you could make available for download ? Because the compression artifacts are a bit annoying with so few pixels/degree.

Thank you.

@Namielus : too bad about GoPro, I heard someone else trying to contact them to consider higher separation for they 3D kit (around 6.5cm) but they weren't interested either. There is only the DIY way then...

[Namielus]

BOLL; would you also be so kind to make us another video, for the people who has a rift (or equivalent) at the moment.
Try looking at objects that are closer to you and in front.
I just want to hear how people experience it.

[BOLL]

Hi Guys! Sorry for not responding earlier. Since last time I have moved from a 10k town to the second largest city in Sweden (still not that large, but still) and I've just started studying app development... interesting but so far little time for procrastination! I am right now in my old apartment, will try to fix it up to rent it out so I don't have to store my furniture somewhere as I only rent a room in Gothenburg.

So yeah. Right now I have no access to my project file, the cameras or anything like that :D as all that stuff has been moved already :p

Still. Let me try to elaborate on things. To make the video I did only do so much cropping as was needed to get rid of black bars that appeared due to the need to make rotational adjustments. I cannot remember how much I had to rotate the clips to match them right now, but between 1-5 degrees, hahaha. As for exposure settings, I'd love to get my hands on a 3D-sync cable without buying the actual 3D-housing... it's expensive.

Yes! I could try to distribute the rendered file directly, just not sure how to do it. Like a torrent? I only have mobile Internets where I am staying so preferably some external hosting... was thinking dropbox, just that mine is running out of space hahaha. Might clean it up and make it available for a shorter time... we'll see.

I'd also love to make a new video, as soon as I am settled in at the new place and have some time on my hands. I'm planning to try and have the cameras point perfectly straight forward with no convergence, it is probably easier to make stereo adjustments then... yeah. I feel like building a better rig for this, my brother is soon investing in a 3D-printer which could be helpful, he lives plenty of hours away though haha.

And now I really have lots to do, haha, laters!

[Fredz]

Good luck with your new life, seems it's not the best moment to ask you things.

When everything is more calm and you can get access to your video files again, you could share them over rapidshare or other file sharing sites.

Thanks for that video and the photo of your rig still, great inspiration for everyone.

[troffmo5]

Do you think the new HackHD camera, sold from sparkfun, can also be used?
https://www.sparkfun.com/products/11418
(Lens: 2.5mm (EFL), F2.8, 160 degree (diagonal) wide angle lens)
The sync between the two cameras is to implement but the separation is customizable.

[raijinspecial]

Wow that video looked great. I just held my phone close to my face and crossed my eyes and it looked pretty impressive.

[BOLL]

Just don't end up like this guy:

Haha, I accidentally found him in video form while browsing another thread and checking related videos... how random.

As for progress, otherwise. Studying is happening, but now I'm utterly sick due to exhaustion, so yeah...

I've been sketching on a 3D-printed housing for the purpose of making Rift-video with my old GoPros, it's a way off though as my dear brother has not ordered the printer yet and I haven't checked how to make a working 3D-file for that printer. Though, I have some other thoughts too.

The Rift seems to use aspherical lenses, I am sure that is not a perfect match for the GoPro, so a video will probably need some distortion adjustment. The question is if this is best to do in post, or as a filter in the video player, like VLC. I am speculating that future versions of the Rift could get a slightly different distortion which would mean it might be stupid to have permanently adjusted media that then would no longer be optimally adjusted. It might just be better to have a filter for each version of the Rift is that becomes the case. Of course compatible video files would have to follow some sort of standard in formatting, the same amount of crop or something, if there is viewing angle left. Perhaps there will also be interocular distance adjustments in software for the Rift, that would make videos hard to standardize I guess... erhm.

While reading online I found out that the field of view of the GoPro is higher without the case, I always thought it was the other way around, but I tested it out and it's true :P Whodathunkit.

I also contacted GoPro about purchasing a 3D sync cable without the case, even as a spare part, but they said there was currently no way to get your hands on a cable without buying the case too. Bleh. I found a reverse engineering post on a GoPro forum (requires registration) but it does seem a bit complicated for a nub like me :p

Oh, and my last thoughs, before I fall asleep. As people have reported motion/simulator sickness, I've been pondering if the best kind of video would be very stable, non-tilting, slow-rotating, smoothly accelerating/decelerating footage would be easiest to consume. Will walk-head-bobbing make people throw up? I guess it'll be interesting to experiment :)