Project Looking at the waves (Helping the deafs with oculus)
Posted: Mon Apr 01, 2013 6:20 pm
Hello Guys, Im Marcelo Guibout R from Chile and i would like to show the project that im gonna build in the oculus rift.
The project is a software named Looking the waves with a Redesigned Speech recognition system focusing on the sounds that deaf people make and phoneme's. ( Deeply sorry for my english (any grammar or any verbal mistake that i could make) i will be tryng to talk to a friend who is better at writing in english that me to check this and fix the mistake as soon as posible).
What is Looking the waves?
Looking the waves is a software and a interactive game that will be focusing on helping Prelingual deafness childrens and people to learn to speak and is specifically designed for teachers of deaf children with some experience in providing habilitation or rehabilitation services to children with implants.
Basically this project is relying on the immersion of virtual reallity as his main support to create a experience in which the sounds will have a visible effect that we all can see and i want to say that the sound becoming a visible impact on the world around him is the most important and wonderful thing of this project.
Why the sound becoming a visual effect so key to this project?
Well i will try to explain what im tryng to accomplish with this analogy (its not a perfect analogy but its explain a little better what im doing):
What Happens if you give a play-doh to a blind kid who dont know what is a ball and ask them to build one? He couldnt be able to make a ball even after you explain how a ball is but if he uses his somesthetic senses (touch) and feel a ball he could be able to build one with just a little of help in the technique and some time.
im tryngto use the sight to teach them to talk in a similar way as we teach the play doh kid to make a ball in the analogy of course this is a lot more difficult as is harder to teach them how to make differents sound and other factors are in the table too.
I hear that most of the childrens dont really enjoy the therapy and practice involved in the hard way that is learning to talk but i think that using this software as support could just not make the proceess a lot more fun but it could be a improvement and a great step in making this processes faster and easy for them and at the same time we would be doing test to see if its possible to teach deaf people without the implants using the oculus rift , the software and vibration methods.
How its work: Using Speech perception knowledge and Speech recognition system like DragonDictate and a custom device to have enough keys as values. I will be tryng developing a interactive game where you can see the sounds you make as a visual thing.
Basic concept : this is just a base idea of how this could be done and as i work with experts teachers and prelingual deafness childrens/ peoples i hope to find the best way to do this.
In the interactive game every sound you make will create a color spot in front of you making the illusion as is going out for your mouth . The color spot will have difference's (form , intensity,etc )depending on the sound that you make and the correct pronunciation.
After the student and teacher decide what will be the initial sound that they will be working on the teacher will make the closest phoneme to that sound with the correct pronunciation and will make certain color spot and the student will approach to copy it and make the initial sound change to the the teacher one.
Using the form and intensity as criteria the student will be able to see his progress as he try to make the phoneme.
This process will be working on words, phrases and sentences at far as we can. As we improve and work with better Speech recognition system's this will be improved.
My dream is a social project after making Looking the waves successful and having something real but is a social project but is way more.
The social project : Is the implementation of this system as main source of teaching just using the vibration and others way.
With this we could teach people from South America and other places of the world who dont have a cochlear implants implementation or have a older cochlear implants implementation.
As we know the cost of the cochlear implants the implementation is very high wikipedia says that according to U.S NIDCD the estimated total cost is $60,000 per person implanted. if we could reduce the price to teach deaf people to talk it would be great.
Its my dream that deaf people could learn to talk without a cochlear implants and without a really expensive teacher with just the next things: a oculus rift, my software and a deaf teacher with a really good educational system for them.
what we could be able to do if we could have with just a decent monetary help from the capital of a south american country or the South American country if we have a system so cheap? A whole center helping and teaching prelingual deafness childrens and people of South America to talk.
Background of the project :
This is one of my projects that was pretty much impossible a couple of years ago but since then i was really wanted to work on this , i maked some test and see what was possible and how i could accomplished what i wanted in this two years thanks to one of my friends who has been studying Audio engineering since a few a years and others friends who are mainly studyng computer science and programming making basically doing some tests when something new come up without the vr thing because even when im a enthusiastic of vr always following vr news related to gaming my knowledges of vr were very limited and my experience with vr is practically to little to taked seriusly the possibility of doing vr to finally start this project.
i meet a friend who knows about vr a few months ago and after a few talks i started to think about this he is from u.s and tested the oculus rift so i could ask him about if i could have some chance of doing the sort of things i wanted to do and after seeing the oculus gdc videos, reading the whole Michael Abrash presentation and the sdk developer forum released i finally decided to work on this project.
A little about me and how is going on this.
Hello before all i want to say thanks you for reading the whole text and apologize for my bad english and any grammar mistake.
Im 18 years old normal guy and even if they are some personal experience making me to do a project like this i prefer to not to talk about this in this thread and just go with other stuff that i think more important, i started with web developement at the age of 12 learning lamp/wamp i really enjoyed the whole web developement world for a few years after that i spent some time doing c/c++ stuff but just for helping some friends projects.
At this moment i dont have a oculus rift or any income of money to buy them i was hoping that some friends who are studyng at a good comptuer science college from chile who have one of the the equipament on computer science field would buy at least one and i could grab them to see what i could do but since i dont think that will happenand and i decided to do this beside of that i will sell some stuff and try to make some money as soon as possible to buy the oculus rift meanwhile i will be working in the nest things :learning to develop on oculus rift and keep working on a better speech recognition system with some friends .
i would really like to know what do you think about my project.
The project is a software named Looking the waves with a Redesigned Speech recognition system focusing on the sounds that deaf people make and phoneme's. ( Deeply sorry for my english (any grammar or any verbal mistake that i could make) i will be tryng to talk to a friend who is better at writing in english that me to check this and fix the mistake as soon as posible).
What is Looking the waves?
Looking the waves is a software and a interactive game that will be focusing on helping Prelingual deafness childrens and people to learn to speak and is specifically designed for teachers of deaf children with some experience in providing habilitation or rehabilitation services to children with implants.
Basically this project is relying on the immersion of virtual reallity as his main support to create a experience in which the sounds will have a visible effect that we all can see and i want to say that the sound becoming a visible impact on the world around him is the most important and wonderful thing of this project.
Why the sound becoming a visual effect so key to this project?
Well i will try to explain what im tryng to accomplish with this analogy (its not a perfect analogy but its explain a little better what im doing):
What Happens if you give a play-doh to a blind kid who dont know what is a ball and ask them to build one? He couldnt be able to make a ball even after you explain how a ball is but if he uses his somesthetic senses (touch) and feel a ball he could be able to build one with just a little of help in the technique and some time.
im tryngto use the sight to teach them to talk in a similar way as we teach the play doh kid to make a ball in the analogy of course this is a lot more difficult as is harder to teach them how to make differents sound and other factors are in the table too.
I hear that most of the childrens dont really enjoy the therapy and practice involved in the hard way that is learning to talk but i think that using this software as support could just not make the proceess a lot more fun but it could be a improvement and a great step in making this processes faster and easy for them and at the same time we would be doing test to see if its possible to teach deaf people without the implants using the oculus rift , the software and vibration methods.
How its work: Using Speech perception knowledge and Speech recognition system like DragonDictate and a custom device to have enough keys as values. I will be tryng developing a interactive game where you can see the sounds you make as a visual thing.
Basic concept : this is just a base idea of how this could be done and as i work with experts teachers and prelingual deafness childrens/ peoples i hope to find the best way to do this.
In the interactive game every sound you make will create a color spot in front of you making the illusion as is going out for your mouth . The color spot will have difference's (form , intensity,etc )depending on the sound that you make and the correct pronunciation.
After the student and teacher decide what will be the initial sound that they will be working on the teacher will make the closest phoneme to that sound with the correct pronunciation and will make certain color spot and the student will approach to copy it and make the initial sound change to the the teacher one.
Using the form and intensity as criteria the student will be able to see his progress as he try to make the phoneme.
This process will be working on words, phrases and sentences at far as we can. As we improve and work with better Speech recognition system's this will be improved.
My dream is a social project after making Looking the waves successful and having something real but is a social project but is way more.
The social project : Is the implementation of this system as main source of teaching just using the vibration and others way.
With this we could teach people from South America and other places of the world who dont have a cochlear implants implementation or have a older cochlear implants implementation.
As we know the cost of the cochlear implants the implementation is very high wikipedia says that according to U.S NIDCD the estimated total cost is $60,000 per person implanted. if we could reduce the price to teach deaf people to talk it would be great.
Its my dream that deaf people could learn to talk without a cochlear implants and without a really expensive teacher with just the next things: a oculus rift, my software and a deaf teacher with a really good educational system for them.
what we could be able to do if we could have with just a decent monetary help from the capital of a south american country or the South American country if we have a system so cheap? A whole center helping and teaching prelingual deafness childrens and people of South America to talk.
Background of the project :
This is one of my projects that was pretty much impossible a couple of years ago but since then i was really wanted to work on this , i maked some test and see what was possible and how i could accomplished what i wanted in this two years thanks to one of my friends who has been studying Audio engineering since a few a years and others friends who are mainly studyng computer science and programming making basically doing some tests when something new come up without the vr thing because even when im a enthusiastic of vr always following vr news related to gaming my knowledges of vr were very limited and my experience with vr is practically to little to taked seriusly the possibility of doing vr to finally start this project.
i meet a friend who knows about vr a few months ago and after a few talks i started to think about this he is from u.s and tested the oculus rift so i could ask him about if i could have some chance of doing the sort of things i wanted to do and after seeing the oculus gdc videos, reading the whole Michael Abrash presentation and the sdk developer forum released i finally decided to work on this project.
A little about me and how is going on this.
Hello before all i want to say thanks you for reading the whole text and apologize for my bad english and any grammar mistake.
Im 18 years old normal guy and even if they are some personal experience making me to do a project like this i prefer to not to talk about this in this thread and just go with other stuff that i think more important, i started with web developement at the age of 12 learning lamp/wamp i really enjoyed the whole web developement world for a few years after that i spent some time doing c/c++ stuff but just for helping some friends projects.
At this moment i dont have a oculus rift or any income of money to buy them i was hoping that some friends who are studyng at a good comptuer science college from chile who have one of the the equipament on computer science field would buy at least one and i could grab them to see what i could do but since i dont think that will happenand and i decided to do this beside of that i will sell some stuff and try to make some money as soon as possible to buy the oculus rift meanwhile i will be working in the nest things :learning to develop on oculus rift and keep working on a better speech recognition system with some friends .
i would really like to know what do you think about my project.