[Tool] Kinect-like Speech Recognition Tool

A hub for everything related to NBA 2K13 modding. Releases, previews, requests, and other modding discussion belongs here.

[Tool] Kinect-like Speech Recognition Tool

Postby Leftos on Tue Oct 02, 2012 5:59 pm

Inspired by the Kinect functionality of the X360 version of NBA 2K13, I decided to try and make a tool that will allow PC users to get as much of the same functionality as possible, with any PC microphone. Here's a preview of what I've got so far.

We can only create shortcuts based on keypresses, so nothing too fancy. I'm starting with what's on the D-pad (Right and Left), and then maybe some position specific shortcuts based on Left Bumper + Player Icon, like "5 set a screen" to get your center to set a screen for you. Ideas? Suggestions?

phpBB [video]


Quick Usage Notes
Now, to run this properly, you just need to extract anywhere, run the EXE, and read the Readme (duh). The Readme has the first-time setup that you should do in-game (you need to remap the keyboard controls; make sure to save the user profile assigned to the keyboard after you change the controls so that they don't reset every time you quit the game) and all the commands you can say.

To test the program out, say anything and see if it reports "Speech Detected". Try a command from the list in the Readme, and see if it recognizes it. If you say "Computer?", it should respond. :P In the game, after you set-up the keyboard controls, you can play with whatever controller you want, just make sure the keyboard is either not assigned to any team, or that it is assigned to your team if you're playing with the keyboard.

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.


Download
NBA 2K Speech Recognition Tool

Requires .NET Framework 4.

Donate and support my work
If you enjoy using my tools and want to show your appreciation, all you have to do is say a hello and a thank you in a reply here. If, however, you want to show a little more appreciation, you can always consider making a small donation towards my everyday and studies expenses. Any donation, no matter the amount, will give me even greater incentive to develop these tools and helping the community.

Click here to donate!
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Leftos on Tue Oct 02, 2012 6:04 pm

This tool should already be compatible with NBA 2K13. Don't have a copy of the game yet, so those of you who do, please give me feedback on what works and what needs to be changed.

Also, keep in mind that if you manage to create a nice button combo, you can share it with the other users here! Go to "Documents\NBA 2K Speech Recognition Tool\Commands" and in there you should find the file that corresponds to the command you created. Upload it somewhere, reply with a link to it, and I'll add it to the first post.

Any video that showcases the tool's functionality will also get a first post mention. First one gets to be embedded and I promise to abuse my Team Member status to get it to the NLSC front page as well! Make it a good one, not just one to say "I did it first!" It should showcase the tool's functions in proper context.

Have fun with this!
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Kinggdawg on Tue Oct 02, 2012 6:58 pm

Hey Leftos, managed to pick up a copy of 2k13 few hours ago, will be trying this out when i wake up :/

Note: I tried v0.1 of this on 2k12, didn't really seem responsive. Lets hope this is better, btw what are the major changes of this version?
The King
LeBron James #23
CLEVELAND CAVALIERS
User avatar
Kinggdawg
 
Posts: 133
Joined: Sun May 09, 2010 7:54 am

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Leftos on Tue Oct 02, 2012 7:04 pm

It's as responsive as it can be, really. Can't make it go any faster, since I use a prebuilt Speech Recognition engine. Tool is only going to be as responsive as that engine is, there's practically no delay from the point your command is recognized to the point it's called for execution.

Only way to make it more responsive is to maybe give it a higher priority than the NBA 2K executable. If NBA 2K is using all the CPU, your phrase is bound to be detected slower, thus the additional delay.

Are you saying that you felt it was less responsive than the preview video I've put up? Or does the preview video not seem responsive enough to you? If it's the second one, there's nothing I can do to make it faster, sorry. If you compare my preview video to the ones being posted with Kinect on X360, I think the delay is about the same, and as good as it gets.

v0.2 lets you add and edit commands, which means you can add any button combo as a voice command, and activate it with whatever phrase you like! Then, you can share your commands with other users here, by either uploading a screenshot, or the command file from My Documents.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Kinggdawg on Tue Oct 02, 2012 7:07 pm

Leftos wrote:It's as responsive as it can be, really. Can't make it go any faster, since I use a prebuilt Speech Recognition engine. Tool is only going to be as responsive as that engine is, there's practically no delay from the point your command is recognized to the point it's called for execution.

Only way to make it more responsive is to maybe give it a higher priority than the NBA 2K executable. If NBA 2K is using all the CPU, your phrase is bound to be detected slower, thus the additional delay.

Are you saying that you felt it was less responsive than the preview video I've put up? Or does the preview video not seem responsive enough to you? If it's the second one, there's nothing I can do to make it faster, sorry. If you compare my preview video to the ones being posted with Kinect on X360, I think the delay is about the same, and as good as it gets.

v0.2 lets you add and edit commands, which means you can add any button combo as a voice command, and activate it with whatever phrase you like! Then, you can share your commands with other users here, by either uploading a screenshot, or the command file from My Documents.

Oh v0.2 sounds sweet. I don't know what was happening, could've been my mic as the majority of the time was using my default laptop mic but it would say recognized "isolation" and do nothing..I even set the controls to what you recommended. I also tried using a custom mic and the only command that would be recognized was "Isolation", "Iso", I recorded my self saying the phrases on sound recorder and it was fine and my voice was clear and understandable.
The King
LeBron James #23
CLEVELAND CAVALIERS
User avatar
Kinggdawg
 
Posts: 133
Joined: Sun May 09, 2010 7:54 am

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Leftos on Tue Oct 02, 2012 7:10 pm

Don't know what to tell you then. Note that just because you can understand yourself, doesn't mean the speech recognition engine does. It depends on accent, what culture it is set to detect (I've set it to en-US, which is practically no accent :P). Hope it all works out. I know for a fact all the commands in there work, and I hope you find a way to get them to work for you as well.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Speech Recognition Tool v0.2 (Kinect-like functio

Postby Kinggdawg on Tue Oct 02, 2012 7:20 pm

Haha for all I know, I'm from Florida which means I've got an American accent :O. Yeah gonna tell you how v0.2 plays out, trying it out after I get some beauty sleep man.

I'll keep you updated.
The King
LeBron James #23
CLEVELAND CAVALIERS
User avatar
Kinggdawg
 
Posts: 133
Joined: Sun May 09, 2010 7:54 am

Re: [Tool] Speech Recognition Tool v0.3 (Kinect-like functio

Postby Leftos on Wed Oct 03, 2012 1:22 am

v0.3 released!
- Added Back/Select as a button for combos
- Added "Call PF for Screen", "Call PF for Double Team", "Call Timeout", "Intentional Foul" commands
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby kendz4494 on Wed Oct 03, 2012 1:29 am

Mr Leftos how do i remap keyboard controls? there is no readme file.


BTW, I use xbox controller
Last edited by kendz4494 on Wed Oct 03, 2012 1:38 am, edited 1 time in total.
CPU: Intel Core i7-2600k
CPU Cooler: Corsair H60
GPU: msi GeForce GTX 660 Twin Frozr OC 2GB
MOBO: Asus P8Z77-V LX
Boot Drive: Samsung 840 Series 120 GB
HDD: Seagate 1.5 TB SATA 6Gbps Barracuda Green
RAM: Kingston 16GB
PSU: Corsair CX600M
DVD: Liteon 24x Dvd/CD Writer
User avatar
kendz4494
 
Posts: 218
Joined: Mon Dec 20, 2010 5:39 pm
Location: New York

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby Leftos on Wed Oct 03, 2012 1:32 am

There is in the program's installation folder. I'll make it viewable from within the program, forgot about that.

Find Control Setup, press RB/R1 until you're at Keyboard/Mouse, and remap the keys accordingly.

Image

Only key not shown here is the Timeout key which should be Backspace.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby kendz4494 on Wed Oct 03, 2012 1:40 am

Thanks for your fast reply. I appreciate it. More power! :D
CPU: Intel Core i7-2600k
CPU Cooler: Corsair H60
GPU: msi GeForce GTX 660 Twin Frozr OC 2GB
MOBO: Asus P8Z77-V LX
Boot Drive: Samsung 840 Series 120 GB
HDD: Seagate 1.5 TB SATA 6Gbps Barracuda Green
RAM: Kingston 16GB
PSU: Corsair CX600M
DVD: Liteon 24x Dvd/CD Writer
User avatar
kendz4494
 
Posts: 218
Joined: Mon Dec 20, 2010 5:39 pm
Location: New York

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby seanjohn2525 on Wed Oct 03, 2012 2:29 am

bro.. can i use my controller even if i change the keyboard setup??
User avatar
seanjohn2525
 
Posts: 1410
Joined: Tue Oct 20, 2009 7:03 pm
Location: imus, cavite

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby Leftos on Wed Oct 03, 2012 2:38 am

Sure. You don't assign the keyboard to any team. You just need to change the keyboard setup so that the tool can send commands through it, but you can use any gamepad you want. Just leave the keyboard at center when selecting sides.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby Rangez on Wed Oct 03, 2012 6:44 am

so this works on NBA 2K13?
°°°°°Rangez°°°°°
I can accept failure, everyone fails at something. But I can't accept not trying.
Michael Jordan
NBA 2k Montage/Mix Account: http://www.youtube.com/user/Mr2KImpossible
NBA Real Life Mixes: http://www.youtube.com/user/TdotPrid3
Comment, Rate and Subscribe! :)
Rangez
 
Posts: 1245
Joined: Fri Dec 31, 2010 10:20 am

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby JaoSming on Wed Oct 03, 2012 7:08 am

yep, works on all games technically.
Opinions are my own.

JaoSming
2KTV Producer
NBA 2K Developer
 
Posts: 29904
Joined: Tue Sep 13, 2005 12:45 am
Location: 2K

Re: [Tool] Kinect-like Speech Recognition Tool v0.3

Postby Rangez on Wed Oct 03, 2012 7:33 am

JaoSming wrote:yep, works on all games technically.

Cant wait to use it on 2K13 when i get it! Will def. make it easier for me to get clips for my edits :)
°°°°°Rangez°°°°°
I can accept failure, everyone fails at something. But I can't accept not trying.
Michael Jordan
NBA 2k Montage/Mix Account: http://www.youtube.com/user/Mr2KImpossible
NBA Real Life Mixes: http://www.youtube.com/user/TdotPrid3
Comment, Rate and Subscribe! :)
Rangez
 
Posts: 1245
Joined: Fri Dec 31, 2010 10:20 am

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1

Postby Leftos on Wed Oct 03, 2012 2:59 pm

v0.3.1 released!
- Minor bug fixes
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1

Postby Westwindization on Wed Oct 03, 2012 10:40 pm

Let me star by saying this is great idea for pc users.

I tryed this on 2k13 and seems like everything is working, but commands are wrong. F.e. When i say iso player passes the ball. I changed my joystick conntrollers too maybe thats the reason. Mybe the reason that controllers are a bit different from 2k12 to 2k13 .Only thing that seems to work is 3-2 zone and set screen , nothing else. How can i make it work
Westwindization
 
Posts: 71
Joined: Sun Nov 21, 2010 2:13 am

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1

Postby Leftos on Wed Oct 03, 2012 10:45 pm

Iso is D-Pad Right + A, which leads me to think that either:
- You didn't set some of the keyboard bindings correctly.
- You're using your gamepad right after calling out the command. When you say the last word of the command, you must stop using your gamepad until the command starts getting played out on the floor, and then you can start using your gamepad again. If you keep using your gamepad, for example moving the player, when the tool tries to pass on the command, your movement conflicts with the key presses the tool sends, and so the command isn't carried out right.

The tool sends keyboard presses, so your joystic/gamepad bindings don't matter. Only keyboard bindings matter.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1

Postby Angelo on Thu Oct 04, 2012 4:55 am

Ill try this out once i get a new microphone my old one seems not working anymore :)
Image
User avatar
Angelo
Holy Tits
NLSC Team Member
 
Posts: 5895
Joined: Fri Nov 03, 2006 1:40 am
Location: Philippines

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1.1

Postby Leftos on Sat Oct 06, 2012 2:55 am

v0.3.1.1
- Attempt to fix Speech Recognition Engine culture issue
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1.1

Postby Jarus on Sat Oct 06, 2012 3:48 am

this program is either not working , extremely delayed, or doing wrong plays/// what should i do ?
Image
User avatar
Jarus
 
Posts: 401
Joined: Thu Apr 30, 2009 3:50 am
Location: Behind the Arc

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1.1

Postby Leftos on Sat Oct 06, 2012 4:04 am

Jarus, don't know what to tell you. Practice without running the game, with just the tool open, so that you can see whether it recognizes your commands or not, and how fast.

How fast it recognizes your command depends on your CPU and how clearly you're speaking, so... I don't know. I can't make it go any faster than what you see in the preview video, which is the same speed as the Kinect recognizes commands on the X360 version of 2K13.

As for "not working"...
As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.


If that's not the case, you'll have to explain "not working" a bit more.
Eleftherios "Leftos" Aslanoglou
NBA 2K AI Software Engineer
Visual Concepts Entertainment / 2K Sports

Used to be "That Tools Guy" around here during the good ol' days. Although you probably remember me as your favorite Podcast host.
User avatar
Leftos
I'm The Pipeline, The Pipeline Is Me
NBA 2K Developer
 
Posts: 5223
Joined: Sun Jun 07, 2009 7:44 am
Location: Novato, CA, USA

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1.1

Postby tnt23 on Sat Oct 06, 2012 2:56 pm

gotta test it :) sounds really cool (Y) :bowdown: thx for the hard work
ALLTiME ALLSTARS
Image
User avatar
tnt23
 
Posts: 845
Joined: Sat Oct 07, 2006 10:50 pm

Re: [Tool] Kinect-like Speech Recognition Tool v0.3.1.1

Postby wisealma on Sun Oct 07, 2012 4:20 am

This is an amazing idea! Thanks for trying to do this. This would be an amazing feature.

Leftos wrote:Inspired by the Kinect functionality of the X360 version of NBA 2K13, I decided to try and make a tool that will allow PC users to get as much of the same functionality as possible, with any PC microphone. Here's a preview of what I've got so far.

We can only create shortcuts based on keypresses, so nothing too fancy. I'm starting with what's on the D-pad (Right and Left), and then maybe some position specific shortcuts based on Left Bumper + Player Icon, like "5 set a screen" to get your center to set a screen for you. Ideas? Suggestions?

phpBB [video]


Quick Usage Notes
Now, to run this properly, you just need to extract anywhere, run the EXE, and read the Readme (duh). The Readme has the first-time setup that you should do in-game (you need to remap the keyboard controls; make sure to save the user profile assigned to the keyboard after you change the controls so that they don't reset every time you quit the game) and all the commands you can say.

To test the program out, say anything and see if it reports "Speech Detected". Try a command from the list in the Readme, and see if it recognizes it. If you say "Computer?", it should respond. :P In the game, after you set-up the keyboard controls, you can play with whatever controller you want, just make sure the keyboard is either not assigned to any team, or that it is assigned to your team if you're playing with the keyboard.

As for a tip, since the tool will need to take over control for the few moments that your command is being executed, make sure not to be using the controller when you're calling out a voice command, or to stop using it just as you finish speaking the command. When you practice the timing, you'll see that the tool can be really responsive, without taking control from you for too long.

So, if you say "Get Open", just as you finish the word "Open" stop doing anything on your controller, and once your teammate starts his movement and the command has been executed (or the Playcalling Message has been shown; I HIGHLY recommend you enable those for "User Plays Only" in the Coaching Settings), you can start using your controller to do whatever with your player. You can see the timing (and I still don't have it down perfectly) in the preview video. The circle becomes green when the tool takes over, and blue again when I do something with the controller to get back the control. Do NOT expect to be given the control back automatically (i.e. don't expect the circle to turn blue by itself); once your command starts playing out, use your controller to regain control yourself.


Download
NBA 2K Speech Recognition Tool

Requires .NET Framework 4.
Thanks
Wisealma

Image
User avatar
wisealma
 
Posts: 90
Joined: Wed Oct 04, 2006 11:15 am

Next

Return to NBA 2K13 Modding

Who is online

Users browsing this forum: Google [Bot] and 6 guests