Voice recognition for increased immersion

[snorelab]

I grew up with adventure games. This was back before point and click adventure games and well before dialogue selection. Back then you would type stuff like "Look at rock" instead of clicking on a rock and selecting look or some such thing. You would also type your own dialogue. I miss the freedom that you had in those days, even if that freedom was something of an illusion. Nowadays we select the text we want our character to say. It will probably be this way for a while. Since I'm stuck here for a few more years waiting for the AI to get better I was thinking of a way to add immersion to dialogue selection when used in a VR setting with something like the Rift.

Natural language processing has actually gotten fairly good, but not good enough to recognize and parse any given dialogue you could dream up. I think it is good enough to detect a dialogue option, however. I think it would be nice for a player to have the option to speak their dialogue selection rather than select it with a mouse click. The NLP can detect which dialogue option was selected and the NPC can respond accordingly. Non roleplaying types might enjoy this as well as it will add to the immersion factor, even if you don't want to "act" out the line.

Anything that takes away mouse clicks is going to add immersion to the VR experience. Once a game is designed with NLP you open up gameplay possibilities as well.

Some thoughts:
Initiating dialogue by saying a characters name or keywords such as "excuse me" while looking at the character within a certain proximity.
Triggers for spells and specialized actions.
Taking control of units/characters in RTS by calling out names.
Pulling up menus.
Getting a zombie's attention by shouting "Hey, stink face!" at it.
Whistling to attract the attention of a guard.


Reality check:
Playing a game in this way could be moderately embarassing if you have your wife/husband/girlfriend/parents/whoever are within earshot. Especially if you're trying to sex up an elf.

I have nothing to offer in terms of programming, just wanted to share my thoughts.

[zalo]

Skyrim's Shouts can be voice activated (by mods, or if you are using a 360 and a kinect). That will be fun with a rift.

Maybe some sort of Speech to Text software can be adapted for existing games like Façade?

[FingerFlinger]

I was thinking about this exact thing in April, but I thought I would be the only person who'd find it interesting. My thoughts were more about how to hack in support for existing titles, however. I figured that you would need to create a map of all of the dialogue in the game and choose a key phrase or word for each dialogue choic, which the player would need to include when acting out their part. To recognize the context, you'd need to run a screenscraper and inject keypresses according to whatever dialogue the player acted out.

I think that your ideas are much better, players would actually use them. And in principle, they would not be hard implement.

[cybereality]

Yes, this would be great. I think about stuff like this all the time.

One of my favorite games of all time (actually the favorite) was Shadowrun for SNES. It had one of the best dialogue systems for a game from that time. Not you didn't have a keyboard, so you couldn't just type in whatever like some older text-adventures. But what they had was keywords. You would start with no keywords but as you talked to people they would mention certain words which would be in bold. Now that word was added to your dictionary. And as you talked to more people, you could then ask them about that thing. It was really deep too, since you could go back and talk with people you already saw, but when you had new words new things could happen. By the end of the game you probably had over 100 different words to choose from. Really nice. The rest of the game was also cool too.