FreePIE speech recognition

[timmer]

Hello all.

Is it possible to do some speech recognition with FreePIE just like with GlovePIE?

Maybe something like this?

Code: Select all

if said("Free PIE can now use speech"):
   speech.say("Free PIE can now use speech");

Thanks for any info.

[CyberVillain]

I can see what MS Speech has to offer. We need to release soon so might not make it to this release

[CyberVillain]

Please try this version

[timmer]

Sweet! Thanks for that.

Just tried your plugin attachment with this code:

Code: Select all

if said("Hello"):
   speech.say("Yes you said hello");

But it came with this on the error tab

1: name 'said' is not defined

I honestly don't know what code to use. Will you be kind enough to post a working sample code? Thank you very much.

[timmer]

Alright, I gave it a spin and here is what I've got



This is the sample code that I used:

Code: Select all

if speech.said("Hello"):
	speech.say("Yes, you said hello!");

As you all can see, it opens up Microsoft Speech. It goes into listening mode first. Once you say 'Hello', it will reply with 'Yes you said hello'. Cool start.

One thing though, would it be possible to hide or maybe not show up the MS Speech interface and still perform speech recognition just like Glovepie? Thank you very much, CyberVillain.

[CyberVillain]

I've googled on it, and I think its a security thing or something, looks like I cant control that window. Will fix it if something comes up on google / Stackoverflow

[CyberVillain]

There you go

[CyberVillain]

Strange, if I use two phrases like

Code: Select all

if speech.said("Option one"):
	speech.say("Selected one")

if speech.said("Option two"):
	speech.say("Selected two")

It works very well, but If I say something complely different it will still fire one of the two above. Dont know how I can fix that

[timmer]

CyberVillain,

Absolutely amazing stuff there.

I just tried your attachment and the MS Speech UI is gone! Now for the two-phrase (or more) scenario, I think it would be good to have some sort of adjustment with word or phrase matching, again like the Confidence Level of Glovepie. Some more fine-tuning I guess and FreePie would nail it.

Thank you for your contribution and assistance.

[CyberVillain]

I added confidence level, it defaults to 0.9 you can override it like

Code: Select all

if speech.said("Option One", 0.95):

[timmer]

I added confidence level, it defaults to 0.9 you can override it like

Code: Select all

if speech.said("Option One", 0.95):

Nice! Thank you for your continuous updates, CyberVillain.


What are the confidence values that we can use? What does a 0.95 mean as opposed to a value of 0.5?

I tried the following:

Code: Select all

if speech.said("One", 0.95):
	speech.say("You said One!");

if speech.said("Two", 0.95):
	speech.say("You said two!");

But then if I say 'Twenty one', it replies with 'You said one' or if I say 'Twenty two', it returns 'You said two'.

[CyberVillain]

1.0 means that it will only trigger if MS Speech is 100% sure its correct, it will never hit this high. With me not being a native english speaker it hits betwen 0.9 and 0.94 if i say the correct sentence

[xhonzi]

This is really cool.

Is there anyway to integrate into Unity, for example? Say I have a Mass Effect or something like conversation tree. I would like to provide the option of saying the dialogue options in addition to selecting them with the controller. Can I dynamically pass the "if speech.said" variables in that situation, or am I stuck with more regular commands like Fus Do Rah?

[CyberVillain]

are you talking about integrating MS Speech in you own Unity project or a compiled third party Project?

[timmer]

1.0 means that it will only trigger if MS Speech is 100% sure its correct, it will never hit this high. With me not being a native english speaker it hits betwen 0.9 and 0.94 if i say the correct sentence

Thanks for that info.

I tried it with 0.96 to 1.0 but it was not making any match at all even if I yell at the mic. A value of 0.95 does make a match at the cost of accuracy.

Will training the computer improve accuracy in speech recognition?

[CyberVillain]

I'm not expert on MS Speech so I do not know. Sorry, havent found any info on it either when googling

[xhonzi]

are you talking about integrating MS Speech in you own Unity project or a compiled third party Project?

Both really. I meant something that I'd developed, but adding it to something pre-existing would be awesome as well- though I suppose much harder.

[CyberVillain]

Unity is C# right? You can take a look at the source code here

https://github.com/AndersMalmgren/FreeP ... hPlugin.cs

[Balmung]

A other question, is it possible to add a Feature to use different Voices on speech.say()?

For example something like: speech.say("Hello User","Zira")

Zira is the Standard Voice on Win8

Will training the computer improve accuracy in speech recognition?

I have installed Voice Attack today after I test FreePIE with speech recognition, it use the same API for Voice Commands and this Program say that training improve the accuracy on Voice Attack too. And after I train my voice with the Windows Tool I imagine that the accuracy in FreePIE is much better, I can use higher Numbers on speech.said() , before the training I get no response.

[CyberVillain]

I can check the voice config.

Nice about the training, will try to verify

[CyberVillain]

Fixed so that in next version you will be able to select voice

[Balmung]

Nice, thx.

[NoxWings]

Wow! Awesome feature! I personally love it for space sim games, It's so cool to be able to "talk to the ship".

Btw This is working perfect with english, but I just can't make it work with other languages (I've tried it in my laptop wich is set to spanish by default). Is it supposed to work or is it just an english feature? AFAIK Since it is using ms speech I thought it should work.

[CyberVillain]

Maybe the space ship only understands english?

I can look into it

[CyberVillain]

I will fix so that you can choose Langugae and it will default to the system OS one

[xhonzi]

Unity is C# right? You can take a look at the source code here

https://github.com/AndersMalmgren/FreeP ... hPlugin.cs

I looked into this and came up with a bunch of nothing. Unity is C#, but more importantly it is based on Mono, not .NET. Since there is no System.Speech in Mono, the best thing you can do is write a small interpreter in .NET (or FreePie, for that matter) and pass via UDP from the interpreter back and forth to your Unity engine. But then you have to worry about that interpreter running, and it's Windows only.

Too bad. Here's hoping it works in the next version of Unity/Mono.

[CyberVillain]

Wow! Awesome feature! I personally love it for space sim games, It's so cool to be able to "talk to the ship".

Btw This is working perfect with english, but I just can't make it work with other languages (I've tried it in my laptop wich is set to spanish by default). Is it supposed to work or is it just an english feature? AFAIK Since it is using ms speech I thought it should work.

Please try with attached file, it shuld now use users culture if it can

[NoxWings]

Wow! Awesome feature! I personally love it for space sim games, It's so cool to be able to "talk to the ship".

Btw This is working perfect with english, but I just can't make it work with other languages (I've tried it in my laptop wich is set to spanish by default). Is it supposed to work or is it just an english feature? AFAIK Since it is using ms speech I thought it should work.

Please try with attached file, it shuld now use users culture if it can

Wow, thank you very much, I'll check it in a few hours and let you know if it is working (I don't have the pc with non-english language just right now and I also need some sleep xD).

[NoxWings]

Hi again!

I've tried it with your plugin and the default one. First of all I'm really sorry, it seems like speech recognition with other languages is also working with the default plugin. The thing after a lil bit more testing looks like somehow I can't even hit 0.75 confidence level with my native language (spanish) but I can pass 0.9 with english.

So again thank you for your attention and sorry for the inconveniences .

[CyberVillain]

No Worries, how do you change between languages btw?

[NoxWings]

No Worries, how do you change between languages btw?

FreePIE automatically uses the default language, but I never noticed it because The confidence level I was hitting was too low

[CyberVillain]

Complete the speech wizard and it gets better

[kmcwilliams]

Say guys,

I do not know that this has anything to do with issues or problems people are having with MS Speech Recognition and FreePie, but I did run across some discussions on the internets, not related to this specific topic, but which MAY have some bearing on these issues. So when I read some of the comments here .......

In later versions of Windows, MS Speech engine, requires ALL macros/ scripts that run through MS Speech to be "digitally signed". Much like the "driver signing" issues that have come up, and more specifically the x64 version OSes. Some people reported that disabling UAC helped, but ....

From what I read, the first time that the macros were created/ ran, they worked, but any attempts after the first run, MS Speech would refuse to send "macro events" from being sent. Also, I do not know that the events from FreePie would have the same sort of limitations at all, but I also do not know at what point MS limits the functionality of the macros/ scripts.

Like I said earlier, this MAY have nothing to do with these issues posted about here, but IMHO, it is something that should be considered and accounted for, if issues persist.

Sorry if this seems off-topic or it proves to be totally un-related, but as I read through this thread, it seemed a real possiblity.

[CyberVillain]

We only "read" from MS Speech and then all other "writes" are done seperate from MS Speech, there is no way they can stop us from doing that

[Pluckerpluck]

I am having a lot of problems with this. I literally can't get it to work at all.

Code: Select all

import time
if starting:
    speech.say("One!")
    time.sleep(1);
    speech.say("Two!")

That works. I hear "One, two" being said.

However,

Code: Select all

import time

speech.say("Main")
if starting:
    speech.say("One!")
    time.sleep(1);
    speech.say("Two!")

That will say "One" and then freeze. Nothing else will happen.

No idea why. But it doesn't help that I'm not entirely sure how it actually works. (It would actually be really useful if it was possible to display the MS Speech window for debugging).

As for listening. Nothing I do will get a response. But I have absolutely no way to debug it. So I don't know if it just can't hear me. Or if it's freezing when I try to make it listen. What I can say is that the following definitely loops, but at no point can I trigger the "test".

Code: Select all

import time

if starting:
    while(True):
        if speech.said("test", 0.5):
            keyboard.setKeyDown(Key.A)
            time.sleep(0.2)
            keyboard.setKeyUp(Key.A)
        speech.say("Looped")
        time.sleep(2)

FYI: I'm on Windows 7 64bit. I have run this both as a regular user and an administrator. I have done Speech Training.

I hope you can help.

Pluckerpluck

[CyberVillain]

Sleeping or alting the script thread is not a good idea.

What are you trying to achive?

[Pluckerpluck]

Sorry for taking for ages to reply. The point of sleep was just so I could play one sound after the other. If I didn't the second overwrites the first, so you can't run any tests.

Code: Select all

if starting:
    if speech.said("test", 0.5):
    	speech.say("test")

Is that meant to work? Because for me that does literally nothing. No matter what I say, I cannot work out how to get speech.said() to work.

Edit: I've tried without "if starting:" btw, that also doesn't work.

[CyberVillain]

That wont work because if starting will only fire once

But you said that you tried with out that

For me this works perfect and it says test very time I say test

Code: Select all

if speech.said("test"):
       speech.say("test")

[Pluckerpluck]

Nope, that doesn't work for me.

Just wondering, should:

Code: Select all

speech.say("test")

on it's own work? Because it doesn't.

The following will say test:

Code: Select all

if starting:
    speech.say("test")

This will break and say nothing at all:

Code: Select all

speech.say("test")
if starting:
    speech.say("test2")

-----------------------------------

Part of my problem is I have no way to know if this this is even picking up my microphone correctly. It's definitely the default mic, I've even tried it with every other recording device on my PC disabled.

I just have no way to know if it's doing anything at all! I also tried your simple example with an added confidence of 0.2. Still nothing.

I even just tried reinstalling the program.

Any idea at all what could be the problem? Or anything I can do to further troubleshoot this?

----------------------------------

P.S. I'm running these scripts by clicking FreePie.exe (or the shortcut), New, Paste in Script, Run Script (F5). If that's not correct please tell me.

[CyberVillain]

doing speech.say without some kind of if-closure is not wise. It will probably overflow the soundcard buffer

will only fire once when you start the script

Code: Select all

if starting: 
   speech.say("Hello")

Will fire each time you press A

Code: Select all

if keyboard.getPressed(Key.A):
	speech.say("Hello")

Really stramnge that speech regoc does not work, it uses the default soundcard, start "Windows Speech Recognition" and see that all is working there