New to VR - any ideas for a worthwhile hardware project?

This is for discussion and development of non-commercial open source VR/AR projects (e.g. Kickstarter applicable, etc). Contact MTBS admins at customerservice@mtbs3d.com if you are unsure if your efforts qualify.
Post Reply
divedeep
One Eyed Hopeful
Posts: 1
Joined: Sat Sep 20, 2014 4:24 pm

New to VR - any ideas for a worthwhile hardware project?

Post by divedeep »

Hey everyone,

I'm new to VR and the Oculus Rift craze and I could use your help.
I'm a hardware engineer with a lot of experience in FPGA design, PCB design, and video.

I want to get involved. Do you have any suggestions for a compelling hardware project for the Oculus Rift or some other piece of VR gear? I'm trying to figure out where to focus my efforts.

Thanks!
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New to VR - any ideas for a worthwhile hardware project?

Post by MSat »

Consumer VR is so new with so many missing pieces, you really have your pick of the litter. There's still much to do in input devices (controllers, motion sening), and output devices (haptic/force/motion feedback) so you should be able to find something.
NeoTokyo_Nori
Cross Eyed!
Posts: 144
Joined: Wed Jul 16, 2014 10:29 am
Location: Tokyo, Japan

Re: New to VR - any ideas for a worthwhile hardware project?

Post by NeoTokyo_Nori »

divedeep wrote:Hey everyone,
I'm new to VR and the Oculus Rift craze and I could use your help.
I'm a hardware engineer with a lot of experience in FPGA design, PCB design, and video.

I want to get involved. Do you have any suggestions for a compelling hardware project for the Oculus Rift or some other piece of VR gear? I'm trying to figure out where to focus my efforts. Thanks!
Hello, indeed there are many tasks to be solved, but as far as FPGA related issues are concerned,
I am interested to see how it can be applied for processing external facing video camera data
(i.e. from camera mounted on front face).

And use the processed data (such as with Canny Edge Detection) to provide "drift error correction" in tracking of tools you hold, such as gun controllers and swords, as drift is still a big problem with these.
It could potentially be used for HMD tracking also, besides many other fun things you could do with the camera feed. I think outward facing video camera based tracking solutions could provide a lot of future benefit.

note: while real-time processing speeds with latency of under 20ms is desirable,
apparently it is alright to have much longer latency, since it is possible to calculate where the tracking target was X ms ago - if I understood John Carmacks talk correctly ; )

Hope that helps.
MSat
Golden Eyed Wiseman! (or woman!)
Posts: 1329
Joined: Fri Jun 08, 2012 8:18 pm

Re: New to VR - any ideas for a worthwhile hardware project?

Post by MSat »

For image processing, I would think a DSP would be more suitable than trying to accomplish the task with an FPGA which would be extremely complex, probably have lower performance, and more expensive for an end product.

I've mentioned motion sensor arrays (multiple accelerometers & gyros)on a single board processed to form a single virtual sensor which research has shown to greatly reduce the noise floor (thus lowering drift) but I haven't heard of anyone in the VR space undertaking such a project. Not having to have external equipment (such as cameras, or base station hardware) or markers is the "holy grail" of motion tracking. Outwards looking cameras require sophisticated software algorithms and significant processing power while still having only mediocre performance in terms of latency (hard to get a high-res and high frame rate image sensor, at least at reasonable prices). A strictly IMU setup is relatively inexpensive (sensors being a couple bucks a pop) and even having to process an array of them would likely take far less computation resources than any reasonable image-based motion sensing while also eliminating occlusion issues.
NeoTokyo_Nori
Cross Eyed!
Posts: 144
Joined: Wed Jul 16, 2014 10:29 am
Location: Tokyo, Japan

Re: New to VR - any ideas for a worthwhile hardware project?

Post by NeoTokyo_Nori »

MSat wrote:For image processing, I would think a DSP would be more suitable than trying to accomplish the task with an FPGA which would be extremely complex, probably have lower performance, and more expensive for an end product.

Outwards looking cameras require sophisticated software algorithms and significant processing power while still having only mediocre performance in terms of latency (hard to get a high-res and high frame rate image sensor, at least at reasonable prices).
Hello, while I am no expert on FPGA design and capabilities, and I am also not certain that we are understanding each others comments completely. I should point out that FPGA has already been proved to be viable for this type of use in an existing HMD (in development)
https://www.kickstarter.com/projects/20 ... reality-he
quote: "When we designed the Totem, we wanted to create a headset that would be able to evolve with the current fast changing VR space. Our two main processing components, the USB microcontroller and the Audio/Video processor (FPGA), are both field-upgradable with firmware updates."

Also allow me to re-iterate my point, which is that I am proposing the use of the FPGA for the SOLE purpose of error-correction of the drift in IMU based tracking data, NOT in order to replace the entire orientation/position tracking capability. The IMU is obviously going to be the main tracking data provider.

In addition, if latency of the fpga error correction data can be compensated for by applying it to the tracking targets position at say 100 ms ago (which is a known position), then I would expect there is less need for faster expensive hardware.. which as we know gets cheaper every year too.

Then again, with a pc based system, it may just be better to do all that on the CPU, so this is just
throwing ideas at the wall, and see what sticks. ;)
Last edited by NeoTokyo_Nori on Tue Sep 23, 2014 10:26 am, edited 1 time in total.
FMPrime
One Eyed Hopeful
Posts: 23
Joined: Wed Sep 25, 2013 12:36 am

Re: New to VR - any ideas for a worthwhile hardware project?

Post by FMPrime »

A small "intelligent camera" package for eye gaze tracking with near-zero latency and a hmd-friendly size.

Most existing head mounted solutions use modified usb webcams with open source software, but the latency, cpu load, and mounting is far from optimal.
It looks like the ps3eye with ITU Haythan software is still the state of the art for cheap and semi-fast eye tracking.
http://www.amazon.com/Sony-PlayStation- ... 0072I2240/
http://www.itu.dk/research/eye/?page_id=19
http://jdarpinian.blogspot.ca/2014/06/h ... acker.html

There are fpga + camera kits that look interesting, but they too wont fit well on an hmd.
http://www.latticesemi.com/en/Products/ ... ntKit.aspx

Fast gaze tracking would unlock foveated displays and new user interfaces.
Post Reply

Return to “VR/AR Research & Development”