Cobbling together a 5 sense simulator with the Rift

[snorelab]

A post by Dycus and some follow-up in the Rift kickstarter thread has given me a thought. It would be nice to get some full 5 sense experiences right out of the gate with the Rift. They would have to be limited in terms of what you can do within the simulation. A good example would be a beach simulator. The idea is that it would be used for relaxation. You just sit or lay there and watch the waves, the boats, the people walking by, the birds flying. All while listening to the sounds, feeling the warmth of the sun, the feel of the sand or of the water. All in your basement, bedroom, or living room (or back yard if it's warm out).

I don't have the time/resources to work on this myself, but it would be interesting to see other's ideas about what is possible with 5 sense simulations in the near term.

Beach simulator:
Items required for full 5 senses (Rift + positional tracking is assumed):

1) lounge chair
2) Shallow sand box (for feet)
3) Lamp (for heat)
4) Positional tracking of a beverage.
5) Some way to get the smell of sea water in the air. Perhaps a fan over a tank of smelly salt water.

Sitting on the dock of the bay simulator:
The above, but swap out the lounge chair for a wooden bench and the sand box for a container of water. Something that has enough volume to kick your feet around a bit. I suppose you could use your bath.

Sitting under a tree simulator (for winter):
You get the idea.

Ha! I just had an image in my mind. I'm walking down the street and a neighbor is reclining in his front yard. His feet in a sand box, a rift strapped to his head, a cold beer in his hand, waving to some unseen woman in a string bikini. That'll be the day!

[virror]

I hope we will see a lot of these "simulators" once the rift is released : D

[Delryn]

Eh, I don't know if I would go for this. I'd always feel like i'm sitting in an awkward contraption, not on the beach.

What I would go for though, is using the rift in a sensory deprivation chamber, or something along those lines. At that point I wouldn't be surprised if the brain starts imagining touch/smell stimulus based off of the visual and audible stimuli.

[TheHolyChicken]

Sounds like the ideal programming environment, to me. Combine this with the desktop virtualisation stuff and we're ready to rock and roll! (or gently dabble my feet in the 'ocean' while doing some C#)

[virror]

Eh, I don't know if I would go for this. I'd always feel like i'm sitting in an awkward contraption, not on the beach.

What I would go for though, is using the rift in a sensory deprivation chamber, or something along those lines. At that point I wouldn't be surprised if the brain starts imagining touch/smell stimulus based off of the visual and audible stimuli.

You cant know until you have tried : p

[snorelab]

Would sensory deprivation work with an HMD and headphones? I've never done it so I don't know, but to me the key to sensory deprivation is depriving the senses.

To reduce what I'm getting at down to a simple question...what kind of compelling virtual reality environments that utilize all 5 senses could be created with current tech?

I wonder myself how compelling the experience could be and if added extra sensory stimuli would make the experience more compelling. Perhaps a rough attempt at adding additional sensory stimuli would break the immersion as opposed to aid in it. Once I have my Rift I can play around a little.

[snorelab]

Sounds like the ideal programming environment, to me. Combine this with the desktop virtualisation stuff and we're ready to rock and roll! (or gently dabble my feet in the 'ocean' while doing some C#)

Absolutely.

How about when you want to read a book? Instead of just picking up a book (or kindle) you could strap on your Rift and read a virtual book in any environment you'd like. That one would REALLY tick off the Luddites.

[Delryn]

You cant know until you have tried : p

I grew up in the Caribbean. I expect I'd be a snob about such things.

Would sensory deprivation work with an HMD and headphones? I've never done it so I don't know, but to me the key to sensory deprivation is depriving the senses.

To reduce what I'm getting at down to a simple question...what kind of compelling virtual reality environments that utilize all 5 senses could be created with current tech?

I wonder myself how compelling the experience could be and if added extra sensory stimuli would make the experience more compelling. Perhaps a rough attempt at adding additional sensory stimuli would break the immersion as opposed to aid in it. Once I have my Rift I can play around a little.

The HMD and headphones would have to be waterproof. Sensory deprivation is meant to deprive all sense, but if you throw in an HMD and headphones you're providing audio and visual content in a closed environment.

I think the next step for many people is an omnidirectional treadmill. In general I think you'll get more immersion by making your physical actions affect the virtual accurately. That's why I hope to see Kinect work with Rift, at least on the PC.

[snorelab]

I grew up in the Caribbean. I expect I'd be a snob about such things.

I think you're missing the point by focusing on my ocean-y examples. That is the virtual reality experience that I want because I grew up in Chicago. Thus the longing for the ocean setting. What would be an environment you'd love to just relax in? I could whip up a Chicago winter for you?

1) Sub Zero walk in freezer
2) Industrial fan

Having said that, it is supposed to be 65 degrees tomorrow in Chicago!

[Delryn]

Ironically I always wanted to live in a city.

I would probably try to use the Rift to relax in an unearthly, perhaps extraterrestrial scene.

[Mystify]

Ironically I always wanted to live in a city.

I would probably try to use the Rift to relax in an unearthly, perhaps extraterrestrial scene.

Alien landscapes could be pretty sweet

[WiredEarp]

@ snorelab: I think your beach simulation idea is pretty good really. I can see it might be good for people who suffer from stress, it might be a good way for people to virtually relax etc.
I dont think it would be truly practical however until we have greater resolution for the headset, so that it more closely approximates reality that it will with the first generation Rifts.

One of the things I'm quite interested in, is a bird flight simulator, where you lie on your stomach on a weight bench type platform with your arms on 'wings' that let you feel the currents and updrafts using both force feedback by using strategically placed fans. Adding the sense of smell with salt water etc could be a nice touch as well...

[Moriarty]

How about a dry swimming machine in combination with the Rift, that seems feasible. You could even add virtual sharks for extra motivation.

[2EyeGuy]

You have more than 5 senses. The heat-lamp for example is for a sense that's not in the 5.

[PasticheDonkey]

just consider them the five sensory organs skin, tongue, nose, ears and eyes.

[snorelab]

How about a dry swimming machine in combination with the Rift, that seems feasible. You could even add virtual sharks for extra motivation.

You have more than 5 senses. The heat-lamp for example is for a sense that's not in the 5.

Some humans even have the power to sense trolls.

jk

The 5 senses I am talking about are seeing, hearing, feeling, tasting, and smelling. If you think about the senses that way, you really do capture most everything in a broad net (especially replacing touch with feel). I realize that the human body is much more complicated than this. Tasting and smelling are intertwined. We feel pain, sense trouble. But it's a useful heuristic to discuss "full immersion" virtual reality IMO. While you do detect heat from a heat lamp, you can easily lump that one into feeling.

[MSat]

This is something that would likely be most suitable for AR so that you could see certain things such as your body - feet in the sand, etc. while transforming the rest of the world around you.

I like the various ideas mentioned. Such as just relaxing, or having a serene place to read, write, or do other suitable productive tasks. Why do desk work in a cubicle when you could instead be doing it on top of a mountain? What about live streaming of actual locations using a plenoptic camera with a large spatial resolution so that you can look around naturally - all in 3D? You could go on a sort of mini vacation whenever you want!

[Mystify]

The "sense" that I am most interested in, beyond the easy ones like sight and sound, is acceleration. Its not listed among the standard 5, but I think it is more important to immersion than most of them, and it is undeniably one of our senses- we even have organs specifically for dealing with it. It is also a very tricky one to simulate though, and its effects are not constrained to the inner ear.

[cerulianbaloo]

You're speakin my language mystify. One of my favorite things to do is just drive around listening to music, that acceleration makes half the experience. If we can figure out a way to realistically simulate that, coupled with a nice seat/wheel for driving, the rift would be even more of a hit. Hell I know I'd save tons on gas money, and who knows, maybe it would cut down on instances of drunk driving. You can endanger all the virtual lives you want now!

[snorelab]

A scuba diving simulator would be nice. It would be lacking because you would not be able to feel the neutral buoyancy. But you may be able to trick your brain if you could control your perceived buoyancy with a sensor. If you perceived yourself as floating up and down a bit based on the amount of air in your lungs or the flow of air in and out, it may trick your body into feeling neutrally buoyant. You can hook an air flow sensor up to your mouth and it could represent the regulator.

AND you could add sharks.

[geekmaster]

A scuba diving simulator would be nice. It would be lacking because you would not be able to feel the neutral buoyancy. But you may be able to trick your brain if you could control your perceived buoyancy with a sensor. If you perceived yourself as floating up and down a bit based on the amount of air in your lungs or the flow of air in and out, it may trick your body into feeling neutrally buoyant. You can hook an air flow sensor up to your mouth and it could represent the regulator.

AND you could add sharks.

Instead of measuring air flow, you could measure pressure on a strap around your chest to measure lung volume. It has been awhile since I have been scuba diving, but I recall that your depth is determined by average lung capacity, and after awhile you lose awareness of that and it just feels like flying, where you go up when you THINK up and down when you THINK down. With experience, average lung capacity becomes somewhat autonomic.

This is much like driving a car while daydreaming and suddenly becoming aware of situations that require conscious awareness (where the vehicle controls are operated subconsciously). Daydreaming does increase your reaction time though, which is why talking on a cellphone while driving is discouraged (where many people are in a virtual meeting in their mind, with their remote conversation partner).

What I trying to convey is that even artificial control mechanisms become an autonomic extension of your body with practice and experience, which means the immersive effects of VR will increase with usage and experience despite the awkwardness of the control devices.

And no sharks, unless they have lasers! Sharks with lasers, for sure!

[youtube]http://www.youtube.com/watch?v=Bh7bYNAHXxw[/youtube]

[Delryn]

The "sense" that I am most interested in, beyond the easy ones like sight and sound, is acceleration. Its not listed among the standard 5, but I think it is more important to immersion than most of them, and it is undeniably one of our senses- we even have organs specifically for dealing with it. It is also a very tricky one to simulate though, and its effects are not constrained to the inner ear.

That would be awesome. Acceleration would also be apart of the feeling sense, since you feel like your chest is being crushed and your organs are being recombobulated. Well I guess it depends on your rate of acceleration.

The cheapest way to simulate this would be a seat-belt-vest that pulls you towards the back of your seat as you accelerate. To get a realistic feel you will need the pull to be distributed evenly so it feels like a compression.

A less practical way would be to set your rig up inside a NASA-style centrifuge:

[geekmaster]

Your perception of gravity and acceleration is determined by the perceived weight of your body and limbs, and the direction of force. Perhaps the perceived weight could be adjusted by controlling the amount of fluid pumped in and out of bladders in your helmet, shoulder pads, vest and pant legs? The increased fluid weight would increase pressure on your feet (or seated posterior), and also pull down on your head and limbs.

I think that would be a good addition to immersion, when changing the amount of acceleration (and rotating the cockpit to change the direction of acceleration). And perhaps GVS (Galvanic Vestibular Stimulation) could add to that effect while using less physical rotation.

[PasticheDonkey]

galvanic response coupled with d-box (motion controlled seats) can give you acceleration. it's actually the next best imitated sense after sound and vision. adapting the principals of how those work to various VR set ups can be done.

i'm thinking eventually well all have some kind of iron maiden (without the spikes of course,) that would provide force feedback and d-box like movement. this apparatus could then include tech that makes you feel what a surface feels like using controlled pressure waves.

[geekmaster]

... i'm thinking eventually well all have some kind of iron maiden (without the spikes of course,) that would provide force feedback and d-box like movement. this apparatus could then include tech that makes you feel what a surface feels like using controlled pressure waves.

Actually, you would need spikes, just lots of small ones in the form of a Tactile Pin Array:
http://www.dcs.gla.ac.uk/~eve/TactonsWebsite/pin.html

One thing mentioned there is feeding a back-mounted pin array with low resolution video. To show that this works, have somebody draw letters on your back, and you can visualize them easily. The roof of the mouth can perceive even higher resolution video from tactile stimulation, and this has been used in military applications where additional (or covert) visual input was required using a mouth insert.

It would be nice to take advantage of that level of tactile stimulation in VR haptic feedback applications.

You can also see with sound (great for the blind, and could be interesting in a VR game):
http://www.artificialvision.com/winvoice.htm

[youtube]http://www.youtube.com/watch?v=TAq26-HE0yI[/youtube]

I think that synesthetic feedback would be great in VR:
http://en.wikipedia.org/wiki/Synesthesia

[PasticheDonkey]

naw i read about something at sometime where they made people feel like they were touching cat's fur just using controlled air. so no need for pins. calibrating to different body types would difficult enough without tiny pins having to touch all over.