[Tool] Kinect-like Speech Recognition Tool

Inspired by the Kinect functionality of the X360 version of NBA 2K13, I decided to try and make a tool that will allow PC users to get as much of the same functionality as possible, with any PC microphone. Here's a preview of what I've got so far.

We can only create shortcuts based on keypresses, so nothing too fancy. I'm starting with what's on the D-pad (Right and Left), and then maybe some position specific shortcuts based on Left Bumper + Player Icon, like "5 set a screen" to get your center to set a screen for you. Ideas? Suggestions?

phpBB [video]

Quick Usage Notes

Now, to run this properly, you just need to extract anywhere, run the EXE, and read the Readme (duh). The Readme has the first-time setup that you should do in-game (you need to remap the keyboard controls; make sure to save the user profile assigned to the keyboard after you change the controls so that they don't reset every time you quit the game) and all the commands you can say.

To test the program out, say anything and see if it reports "Speech Detected". Try a command from the list in the Readme, and see if it recognizes it. If you say "Computer?", it should respond. In the game, after you set-up the keyboard controls, you can play with whatever controller you want, just make sure the keyboard is either not assigned to any team, or that it is assigned to your team if you're playing with the keyboard.

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.

Download
NBA 2K Speech Recognition Tool

Requires .NET Framework 4.

Donate and support my work
If you enjoy using my tools and want to show your appreciation, all you have to do is say a hello and a thank you in a reply here. If, however, you want to show a little more appreciation, you can always consider making a small donation towards my everyday and studies expenses. Any donation, no matter the amount, will give me even greater incentive to develop these tools and helping the community.

Click here to donate!

This tool should already be compatible with NBA 2K13. Don't have a copy of the game yet, so those of you who do, please give me feedback on what works and what needs to be changed.

Also, keep in mind that if you manage to create a nice button combo, you can share it with the other users here! Go to "Documents\NBA 2K Speech Recognition Tool\Commands" and in there you should find the file that corresponds to the command you created. Upload it somewhere, reply with a link to it, and I'll add it to the first post.

Any video that showcases the tool's functionality will also get a first post mention. First one gets to be embedded and I promise to abuse my Team Member status to get it to the NLSC front page as well! Make it a good one, not just one to say "I did it first!" It should showcase the tool's functions in proper context.

Have fun with this!

Hey Leftos, managed to pick up a copy of 2k13 few hours ago, will be trying this out when i wake up :/

Note: I tried v0.1 of this on 2k12, didn't really seem responsive. Lets hope this is better, btw what are the major changes of this version?

It's as responsive as it can be, really. Can't make it go any faster, since I use a prebuilt Speech Recognition engine. Tool is only going to be as responsive as that engine is, there's practically no delay from the point your command is recognized to the point it's called for execution.

Only way to make it more responsive is to maybe give it a higher priority than the NBA 2K executable. If NBA 2K is using all the CPU, your phrase is bound to be detected slower, thus the additional delay.

Are you saying that you felt it was less responsive than the preview video I've put up? Or does the preview video not seem responsive enough to you? If it's the second one, there's nothing I can do to make it faster, sorry. If you compare my preview video to the ones being posted with Kinect on X360, I think the delay is about the same, and as good as it gets.

v0.2 lets you add and edit commands, which means you can add any button combo as a voice command, and activate it with whatever phrase you like! Then, you can share your commands with other users here, by either uploading a screenshot, or the command file from My Documents.

It's as responsive as it can be, really. Can't make it go any faster, since I use a prebuilt Speech Recognition engine. Tool is only going to be as responsive as that engine is, there's practically no delay from the point your command is recognized to the point it's called for execution.

Only way to make it more responsive is to maybe give it a higher priority than the NBA 2K executable. If NBA 2K is using all the CPU, your phrase is bound to be detected slower, thus the additional delay.

Are you saying that you felt it was less responsive than the preview video I've put up? Or does the preview video not seem responsive enough to you? If it's the second one, there's nothing I can do to make it faster, sorry. If you compare my preview video to the ones being posted with Kinect on X360, I think the delay is about the same, and as good as it gets.

v0.2 lets you add and edit commands, which means you can add any button combo as a voice command, and activate it with whatever phrase you like! Then, you can share your commands with other users here, by either uploading a screenshot, or the command file from My Documents.

Oh v0.2 sounds sweet. I don't know what was happening, could've been my mic as the majority of the time was using my default laptop mic but it would say recognized "isolation" and do nothing..I even set the controls to what you recommended. I also tried using a custom mic and the only command that would be recognized was "Isolation", "Iso", I recorded my self saying the phrases on sound recorder and it was fine and my voice was clear and understandable.

Don't know what to tell you then. Note that just because you can understand yourself, doesn't mean the speech recognition engine does. It depends on accent, what culture it is set to detect (I've set it to en-US, which is practically no accent ). Hope it all works out. I know for a fact all the commands in there work, and I hope you find a way to get them to work for you as well.

Haha for all I know, I'm from Florida which means I've got an American accent :O. Yeah gonna tell you how v0.2 plays out, trying it out after I get some beauty sleep man.

I'll keep you updated.

v0.3 released!
- Added Back/Select as a button for combos
- Added "Call PF for Screen", "Call PF for Double Team", "Call Timeout", "Intentional Foul" commands

Mr Leftos how do i remap keyboard controls? there is no readme file.


BTW, I use xbox controller

Last edited by kendz4494 on Wed Oct 03, 2012 1:38 am, edited 1 time in total.

There is in the program's installation folder. I'll make it viewable from within the program, forgot about that.

Find Control Setup, press RB/R1 until you're at Keyboard/Mouse, and remap the keys accordingly.



Only key not shown here is the Timeout key which should be Backspace.

Thanks for your fast reply. I appreciate it. More power!

bro.. can i use my controller even if i change the keyboard setup??

Sure. You don't assign the keyboard to any team. You just need to change the keyboard setup so that the tool can send commands through it, but you can use any gamepad you want. Just leave the keyboard at center when selecting sides.

so this works on NBA 2K13?

yep, works on all games technically.

yep, works on all games technically.

Cant wait to use it on 2K13 when i get it! Will def. make it easier for me to get clips for my edits

v0.3.1 released!
- Minor bug fixes

Let me star by saying this is great idea for pc users.

I tryed this on 2k13 and seems like everything is working, but commands are wrong. F.e. When i say iso player passes the ball. I changed my joystick conntrollers too maybe thats the reason. Mybe the reason that controllers are a bit different from 2k12 to 2k13 .Only thing that seems to work is 3-2 zone and set screen , nothing else. How can i make it work

Iso is D-Pad Right + A, which leads me to think that either:
- You didn't set some of the keyboard bindings correctly.
- You're using your gamepad right after calling out the command. When you say the last word of the command, you must stop using your gamepad until the command starts getting played out on the floor, and then you can start using your gamepad again. If you keep using your gamepad, for example moving the player, when the tool tries to pass on the command, your movement conflicts with the key presses the tool sends, and so the command isn't carried out right.

The tool sends keyboard presses, so your joystic/gamepad bindings don't matter. Only keyboard bindings matter.

Ill try this out once i get a new microphone my old one seems not working anymore

v0.3.1.1
- Attempt to fix Speech Recognition Engine culture issue

this program is either not working , extremely delayed, or doing wrong plays/// what should i do ?

Jarus, don't know what to tell you. Practice without running the game, with just the tool open, so that you can see whether it recognizes your commands or not, and how fast.

How fast it recognizes your command depends on your CPU and how clearly you're speaking, so... I don't know. I can't make it go any faster than what you see in the preview video, which is the same speed as the Kinect recognizes commands on the X360 version of 2K13.

As for "not working"...

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.

If that's not the case, you'll have to explain "not working" a bit more.

gotta test it sounds really cool thx for the hard work

This is an amazing idea! Thanks for trying to do this. This would be an amazing feature.

Inspired by the Kinect functionality of the X360 version of NBA 2K13, I decided to try and make a tool that will allow PC users to get as much of the same functionality as possible, with any PC microphone. Here's a preview of what I've got so far.

We can only create shortcuts based on keypresses, so nothing too fancy. I'm starting with what's on the D-pad (Right and Left), and then maybe some position specific shortcuts based on Left Bumper + Player Icon, like "5 set a screen" to get your center to set a screen for you. Ideas? Suggestions?

phpBB [video]

Quick Usage Notes

Now, to run this properly, you just need to extract anywhere, run the EXE, and read the Readme (duh). The Readme has the first-time setup that you should do in-game (you need to remap the keyboard controls; make sure to save the user profile assigned to the keyboard after you change the controls so that they don't reset every time you quit the game) and all the commands you can say.

To test the program out, say anything and see if it reports "Speech Detected". Try a command from the list in the Readme, and see if it recognizes it. If you say "Computer?", it should respond. In the game, after you set-up the keyboard controls, you can play with whatever controller you want, just make sure the keyboard is either not assigned to any team, or that it is assigned to your team if you're playing with the keyboard.

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.

Download
NBA 2K Speech Recognition Tool

Requires .NET Framework 4.