Binaural Audio and the Oculus Rift

[Shyguy]

I'm new to the forums and not much of a 3d gamer, however i've been following the Oculus Rift rabidly lately so I thought i'd sign up in the hopes that maybe Palmer see's this thread and finds it cool to pass the idea on to some important peeps. :p

Also to get a discussion going on with you guys and see what you all think.

https://www.youtube.com/watch?v=u163wC6mP2A

Keep listening, especially when they whisper in your ear. Warning though, some people may find the language graphic. Personally I think this would be perfect in combination with the Rift.

EDIT: Here's a friendly version. https://www.youtube.com/watch?v=IUDTlvagjJA

Also a minor question. What's the news on positional tracking and the OR? Like peeking around corners and leaning over stuff like ledges.

[PasticheDonkey]

it's a good idea that been discussed here before. the question is whether it can be replicated by a computer using the sound data it has. there's also some concerns with using earphones with the rift at all.

[mahler]

.... there's also some concerns with using earphones with the rift at all.

What concern is that? That it's too immersive and reporters are said to have been lost that way?

[snorelab]

Also a minor question. What's the news on positional tracking and the OR? Like peeking around corners and leaning over stuff like ledges.

It has been confirmed by Palmer that the consumer version will have positional tracking (correct me if I'm wrong). The dev kit will not have positional tracking.

.... there's also some concerns with using earphones with the rift at all.

What concern is that? That it's too immersive and reporters are said to have been lost that way?

1) I can see a lot of Star Wars Kid type videos showing up on You Tube in which some unsuspecting kid is being recorded using their Rift.
2) You know that moment in which you think you're alone and you are singing to yourself and then you notice that someone else is in the room? There could be a lot of that with the Rift. You could be having the most amazing sword fight in VR and then take off your Rift to discover that your entire family is sitting there watching you for the pure entertainment value.
3) Missed phone calls, doorbell rings, dog choking on a bone, children screaming for food, house on fire, bee invasion, etc.

[Mystify]

So don't get headphones that block out all sound and don't turn them up way too high. You don't need headphones that deafen you.

[cybereality]

@snorelab: My best friend calls it "doing a Stevie Wonder."

[KBK]

Then the other side of it - ending up on Yootoob. The film 'Brainstorm' and the pron loop.

That guy looked like he needed to smoke a cigarette for about a month straight.

[PatimPatam]

related old thread:

3D Binaural OpenAL driver for gaming

and related sad story:

So, what's Creative doing about OpenAL and the rise of software audio?

[wileythecoyote]

There's a slight issue with binaural 3d audio. It is very dependent of the persons unique ear geometry.
I've tried numerous 3d audio demos, and I always hear the audio like it's coming from behind. The sound tracks rather well on all but z-axis.
To really address this issue, we'd need a means calibrate the sound output to our unique ear geometry.

[ColdCutz]

Rats, didn't realize that. After listening to the video above it's considerably apparent. The only time you can kind of sense that the subject is in front of you is when they circle you.

[KBK]

It is possible to do a model test, like that of setting up a HMD for one's given situation. Calibration, as it were. That the software can test you on different conversion algorithms. You choose the ones that are most realistic sounding for your exact situation.

But this technique works best with in-ear cans, not over ear cans. The load on the PC increases, and the need for instantaneous conversion is just as important as updating the video on the HMD. that downloading as much of the number crunching as is possible, to the sound card would be the way to go. And that, would require, possibly, a redesign of current soundcards and their associated hardware.

That the normal sound, needs to be run through a conversion algorithm, then the conversion for one's particular situation, regarding ear shape. That's a fair amount of number crunching, if a given system is already on the edge of being capable of getting the HMD to work at 60hz in the given game or whatnot. That one is taking 5.1 surround data, and then crunching it (binaural conversion), then crunching it again (specific individual ear shape), then sending it out the dacs. I see no reason that the two first steps cannot be combined, but what do I know.

[oculusfan]

1) I can see a lot of Star Wars Kid type videos showing up on You Tube in which some unsuspecting kid is being recorded using their Rift.
2) You know that moment in which you think you're alone and you are singing to yourself and then you notice that someone else is in the room? There could be a lot of that with the Rift. You could be having the most amazing sword fight in VR and then take off your Rift to discover that your entire family is sitting there watching you for the pure entertainment value.
3) Missed phone calls, doorbell rings, dog choking on a bone, children screaming for food, house on fire, bee invasion, etc.

Bee invasion

What if they implement motion controls and give you a full sized sword to swing (like Link's sword for the Wii). Family members couldn't get within 5 feet of you without being in danger of taking a slice to the face.

[Dakor]

Tell me if you are talking about something completely different but in my experience using a Soundcard like the Creative Sound Blaster Recon3D (or better) which support something like the THX Studio pro, you can get a pretty good Surround Sound using a Stereo Headset. (old Xfi cards are nice as well but finding drivers is horrible)
Playing Source or Unreal Engine Games (or any game that supports 3d or surround Sound) you have a pretty good positional finding. Not as impressive as the clip above, specially the close stuff, but already pretty good.
Those Virtual-Sorround-Headsets with build-in Soundcards (like Logitechs G35) can't compete with that. And those "real" Sorround-Headsets like the one by Razer even less.

PS here is another one with several sounds.. it's one of my favorites:
[youtube]http://www.youtube.com/watch?v=wT1XuB95qMk[/youtube]

[Chriky]

There is an open source library that allows you to generate binaural audio. It's called Slab3D, I made a thread about it a while ago - http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15307

[knuf]

you can also use supercollider for binaural experiments.

it is very easy to construct basic processing chain as i did last week and it took just few hours of studying and coding:

1st order ambisonic recording (WXYZ channels)
|
v
soundfield transformation (rotation/tumble based on x/y mouse position - can be replaced with headtracker data)
|
v
binaural rendering