Project Looking at the waves (Helping the deafs with oculus)

[Rest]

Hello Guys, Im Marcelo Guibout R from Chile and i would like to show the project that im gonna build in the oculus rift.
The project is a software named Looking the waves with a Redesigned Speech recognition system focusing on the sounds that deaf people make and phoneme's. ( Deeply sorry for my english (any grammar or any verbal mistake that i could make) i will be tryng to talk to a friend who is better at writing in english that me to check this and fix the mistake as soon as posible).
What is Looking the waves?
Looking the waves is a software and a interactive game that will be focusing on helping Prelingual deafness childrens and people to learn to speak and is specifically designed for teachers of deaf children with some experience in providing habilitation or rehabilitation services to children with implants.

Basically this project is relying on the immersion of virtual reallity as his main support to create a experience in which the sounds will have a visible effect that we all can see and i want to say that the sound becoming a visible impact on the world around him is the most important and wonderful thing of this project.

Why the sound becoming a visual effect so key to this project?
Well i will try to explain what im tryng to accomplish with this analogy (its not a perfect analogy but its explain a little better what im doing):
What Happens if you give a play-doh to a blind kid who dont know what is a ball and ask them to build one? He couldnt be able to make a ball even after you explain how a ball is but if he uses his somesthetic senses (touch) and feel a ball he could be able to build one with just a little of help in the technique and some time.
im tryngto use the sight to teach them to talk in a similar way as we teach the play doh kid to make a ball in the analogy of course this is a lot more difficult as is harder to teach them how to make differents sound and other factors are in the table too.

I hear that most of the childrens dont really enjoy the therapy and practice involved in the hard way that is learning to talk but i think that using this software as support could just not make the proceess a lot more fun but it could be a improvement and a great step in making this processes faster and easy for them and at the same time we would be doing test to see if its possible to teach deaf people without the implants using the oculus rift , the software and vibration methods.

How its work: Using Speech perception knowledge and Speech recognition system like DragonDictate and a custom device to have enough keys as values. I will be tryng developing a interactive game where you can see the sounds you make as a visual thing.

Basic concept : this is just a base idea of how this could be done and as i work with experts teachers and prelingual deafness childrens/ peoples i hope to find the best way to do this.
In the interactive game every sound you make will create a color spot in front of you making the illusion as is going out for your mouth . The color spot will have difference's (form , intensity,etc )depending on the sound that you make and the correct pronunciation.
After the student and teacher decide what will be the initial sound that they will be working on the teacher will make the closest phoneme to that sound with the correct pronunciation and will make certain color spot and the student will approach to copy it and make the initial sound change to the the teacher one.
Using the form and intensity as criteria the student will be able to see his progress as he try to make the phoneme.
This process will be working on words, phrases and sentences at far as we can. As we improve and work with better Speech recognition system's this will be improved.

My dream is a social project after making Looking the waves successful and having something real but is a social project but is way more.
The social project : Is the implementation of this system as main source of teaching just using the vibration and others way.

With this we could teach people from South America and other places of the world who dont have a cochlear implants implementation or have a older cochlear implants implementation.

As we know the cost of the cochlear implants the implementation is very high wikipedia says that according to U.S NIDCD the estimated total cost is $60,000 per person implanted. if we could reduce the price to teach deaf people to talk it would be great.
Its my dream that deaf people could learn to talk without a cochlear implants and without a really expensive teacher with just the next things: a oculus rift, my software and a deaf teacher with a really good educational system for them.

what we could be able to do if we could have with just a decent monetary help from the capital of a south american country or the South American country if we have a system so cheap? A whole center helping and teaching prelingual deafness childrens and people of South America to talk.

Background of the project :
This is one of my projects that was pretty much impossible a couple of years ago but since then i was really wanted to work on this , i maked some test and see what was possible and how i could accomplished what i wanted in this two years thanks to one of my friends who has been studying Audio engineering since a few a years and others friends who are mainly studyng computer science and programming making basically doing some tests when something new come up without the vr thing because even when im a enthusiastic of vr always following vr news related to gaming my knowledges of vr were very limited and my experience with vr is practically to little to taked seriusly the possibility of doing vr to finally start this project.
i meet a friend who knows about vr a few months ago and after a few talks i started to think about this he is from u.s and tested the oculus rift so i could ask him about if i could have some chance of doing the sort of things i wanted to do and after seeing the oculus gdc videos, reading the whole Michael Abrash presentation and the sdk developer forum released i finally decided to work on this project.

A little about me and how is going on this.
Hello before all i want to say thanks you for reading the whole text and apologize for my bad english and any grammar mistake.
Im 18 years old normal guy and even if they are some personal experience making me to do a project like this i prefer to not to talk about this in this thread and just go with other stuff that i think more important, i started with web developement at the age of 12 learning lamp/wamp i really enjoyed the whole web developement world for a few years after that i spent some time doing c/c++ stuff but just for helping some friends projects.
At this moment i dont have a oculus rift or any income of money to buy them i was hoping that some friends who are studyng at a good comptuer science college from chile who have one of the the equipament on computer science field would buy at least one and i could grab them to see what i could do but since i dont think that will happenand and i decided to do this beside of that i will sell some stuff and try to make some money as soon as possible to buy the oculus rift meanwhile i will be working in the nest things :learning to develop on oculus rift and keep working on a better speech recognition system with some friends .

i would really like to know what do you think about my project.

[Namielus]

Since you consistently make the mistake, I hope you don't mind me pointing out that its spelled project with a J.

Interesting Project. This concept might be further enhanced using vibrational motors to give physical feedback of the sound.
Sort of like little transducers. This will let them feel the sound they are making.

Speech recognition software is becoming good enough today that they can read themselves on the fly what they saying too.
If you plan on using this device on children with a small interpupilliary distance you would need to make a custom model so that their eyes converge correctly.
Even then im not sure if its a good idea to have children using this over longer periods of time because their eyes are still developing.
I dont know this for a fact, its just something you need to look into.

[Namielus]

Btw, take a look at this. It is a game where the gameplay is based around making different sounds and playing with your voice.
[youtube]http://www.youtube.com/watch?v=-k-qdfLvimo[/youtube]

[Rest]

Since you consistently make the mistake, I hope you don't mind me pointing out that its spelled project with a J.

Interesting Project. This concept might be further enhanced using vibrational motors to give physical feedback of the sound.
Sort of like little transducers. This will let them feel the sound they are making.

Speech recognition software is becoming good enough today that they can read themselves on the fly what they saying too.
If you plan on using this device on children with a small interpupilliary distance you would need to make a custom model so that their eyes converge correctly.
Even then im not sure if its a good idea to have children using this over longer periods of time because their eyes are still developing.
I dont know this for a fact, its just something you need to look into.

Thanks you . i already fixed the projected mistake.

It would be pretty cool if we enhanced the experience with vibrational motors and the whole thing of making a physicial feedback of the sound seems seems interesting.


As the fact that the eyes are still developing that why really was waiting to have a oculus rift to take decisions but if we think that the game probably wont be a fast paced game and the lesson could be shorts enough to prevent eye damage.

The other thing is right now is a fact that the Cochlear implants give better results on childrens and with this support other people beside childrens with prelingual deafness could have a much better chance of getting better results.

I think that the chance of this being helpful to give better results to people beside childrens with the Cochlear implants worth the shoot.

btw: i think i make the post at the wrong subforum what i should do if i want to move this?

[Namielus]

Just PM Cybereality or Neil if you want the post moved.

[amermahdy70]

there are headphones that having Bass voice vibration powered by USB, so it has USB for power to vibration motors in addition to audio jack for voice, so when ever there is bass the head ear side will be vibrated, they are very cheap like 15 $ (Chinese of course) you can mod them to have them vibrate on all frequencies if you want.

[Rest]

there are headphones that having Bass voice vibration powered by USB, so it has USB for power to vibration motors in addition to audio jack for voice, so when ever there is bass the head ear side will be vibrated, they are very cheap like 15 $ (Chinese of course) you can mod them to have them vibrate on all frequencies if you want.

Thx . that will help me a lot because im tryng to run a alpha test on a college friend who doesnt know how to talk (she just know braille) (she has high Cognitive skills) and im looking forward to see what we can get using this concepts at an early stage.