[Tool] Speech Recognition Tool

[Leftos]

Inspired by the Kinect functionality of NBA 2K13, I decided to try and make a tool that will allow PC users to get as much of the same functionality as possible, with any PC microphone. Here's a preview of what I've got so far.

We can only create shortcuts based on keypresses, so nothing too fancy. I'm starting with what's on the D-pad (Right and Left), and then maybe some position specific shortcuts based on Left Bumper + Player Icon, like "5 set a screen" to get your center to set a screen for you. Ideas? Suggestions?

phpBB [video]

Quick Usage Notes

Now, to run this properly, you just need to extract anywhere, run the EXE, and read the Readme (duh). The Readme has the first-time setup that you should do in-game (you need to remap the keyboard controls; make sure to save the user profile assigned to the keyboard after you change the controls so that they don't reset every time you quit the game) and all the commands you can say.

To test the program out, say anything and see if it reports "Speech Detected". Try a command from the list in the Readme, and see if it recognizes it. If you say "Computer?", it should respond. In the game, after you set-up the keyboard controls, you can play with whatever controller you want, just make sure the keyboard is either not assigned to any team, or that it is assigned to your team if you're playing with the keyboard.

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.

Download
NBA 2K Speech Recognition Tool

Requires .NET Framework 4.

[JaoSming]

is it possible to do "hold" button commands? For the pick and roll?

[Leftos]

I was just trying that before going back to working on NST. It is possible, but with a caveat. While the simulated button is being "held", the game will ignore any other command from, say, the gamepad. I'll need to find the sweet spot on the hold duration so that it's enough to call a screen, but not too long so that you're left standing there. Otherwise, I'll need to make it so you can press something and cancel the action if it's taking too long. I'll absolutely be making the hold duration adjustable to the millisecond, so that each user can choose a duration to his liking.

[pHiL Kizer]

Very Nice Work so far!! I can see this being further expanded upon to where we may end up with more options than Kinect!

[Leftos]

pHiL Kizer, we could have more features than Kinect has. Hell, we could make our voice be the gamepad if we wanted.

"Left, left, left. A little more. No, the other way. Pump fake. Pump fake. Shoot! Tease it. Teeeeease it. YEAH, IT'S IN."

But still, we will be missing some of Kinect's features, such as calling a play for a specific player by the player's name, or calling plays by name in general, unless they're tied to a specific button combo that doesn't change.

What I mean is, I can't call "Monroe, set a screen", unless I tie that to "LB, hold LT", which means that I have to ALWAYS play Monroe at center, because the buttons are tied to positions, not players. Only 2K can implement stuff like that.

Still, I'm going to make the tool highly customizable. If you do your own subs, you'll be able to enter the player's names per position somewhere in the tool, so for example, if you always play Monroe at PF, calling out "Monroe, set screen" will be the equivalent of "Power Forward, set screen". I'll make sure to add other kinds of customization as well, such as adding your own commands to specific combos, or even creating your own voice commands and combos.

[Magic Tmac1]

Great great great work!!!

[jaelove]

wow take that 2k

[bfharrin]

What about the following:

1) dive for loose ball
2) double team
3) alley-oop
4) bounce pass

[Leftos]

Anything that is a button combo can be a voice command, but it's preferable that you're not doing anything on the gamepad the moment the command plays out, because you could mess up the combo by mixing what the tool is trying to do with what you're doing currently, or the input will just be ignored as it's coming from two separate sources.

Double team I understand, the others though, why would you need them on voice commands?

[bfharrin]

Just tried to give some things that might work on like player lock or coaching mode but I understand your point. I know a lot of the forum members are younger but I am almost 37 years old and just can't put a lot of time learning advance moves with a wife & new baby ...also I just don't have the stick skills that I had 5-10 years ago so every little bit helps:)

[dare]

Great job on this man. Is it possible for you to create a tool that let us pick which speech to use. Maybe you can have a list then use the tool while running the game. I know it will be a lot more work for you but that would be awesome if it can be done.

[Andrew]

Awesome stuff!

[ramuajith]

This is awesometacular!!!

[Angelo]

This is awesome good work leftos

[Like A Boss]

This is great news Leftos! As always our tools guy. I hope we can use this in NBA 2K13.

[Leftos]

dare, I don't really get what you're trying to say, but I -am- going to offer the users the choice to add their own phrases for each action, and maybe later even add their own button combos to the list of commands.

[dare]

I mean a list that we can select from. We can only use 4 actions right? So a tool like what I said would be good to have.

[Leftos]

Only 4 actions? Who said you can only have 4 actions? You'll have as many actions as you want!

[dare]

Wow, ok now disregard what I said

[Leftos]

v0.1 released!
It's a rough preview, but it works. Read the Readme file for the available commands and what you need to say to activate them. Future versions will be much more adjustable, without requiring you to remap the keyboard in-game, and allowing you to enter your own phrases for each command.

Looking forward to your feedback!

[ThaLiveKing]

That's crazy

[Leftos]

Here's the keyboard mapping you'll need to do for this version. Make sure that a User Profile is assigned to the Keyboard as a controller and that you save that user profile after changing the controls.

[RapboY]

good job! hope this translates to 2k13! or well actually hopefully the kinect commands work on the pc version, i just don't understand why they couldn't implement that into pc as well.

[Sylvian]

Wowo it`s sound great.

It possible to add own commentary, or new Last name`s on database ??

[Leftos]

RapboY, you should be asking why is 2K requiring a Kinect, when nearly every X360 comes with a headset which could do the exact same thing. Over $100 to just use its microphone? There's $5 mics out there, that, as I have proved with this, could work just fine.

The answer is the business deal with Microsoft, in order to sell more Kinects, even though games like this one only utilize a $5 piece of equipment out of the $100.

But to give you hope, this should work with 2K13 day one, with few, if any, changes. If the same plays are called with the same button combos, then you'll be able to use it as soon as you get the game.

Sylvian, that's irrelevant to what this tool is doing. I'm sorry.