Serious effort to reverse engineer and change the "OK Google Now" activation phrase
I just got my Moto X dev edition today, and I plan to try and reverse engineer the activation phrase code in order to change it. I'm going to start by decompiling the touchless application, but I think it will be far more difficult and deeper in the system than that.
Does anyone know where the activation phrase is actually stored? Does anyone know anything about the internal structure of the touchless stuff? And finally, is anyone aware of any other attempts to do this so I can collaborate with them?
Thanks!
glitch003 said:
I just got my Moto X dev edition today, and I plan to try and reverse engineer the activation phrase code in order to change it. I'm going to start by decompiling the touchless application, but I think it will be far more difficult and deeper in the system than that.
Does anyone know where the activation phrase is actually stored? Does anyone know anything about the internal structure of the touchless stuff? And finally, is anyone aware of any other attempts to do this so I can collaborate with them?
Thanks!
Click to expand...
Click to collapse
I got no idea about any of your questions. I wanted to say that doing what your doing would be a very cool feature so thanks for the effort if you get anything going.
I also wanted to mention that I know the OK Google Now phrase is processed on the phone. The voice commands first get processed through Motorola's touchless control app or processes then if it is not recognized as a command it can deal with it then uploads it to google owned google now app. This is why you sometimes get different looking dialers or screens when sending texts depending on if Moto touchless controls handled the action or if google now handled the action.
Just thought of this but I bet it is extremely hard to do what you are doing because no one has created apps to use the low power processor so i presume its locked up somewhere in the phone.
a small start
So, I decompiled the Touchless APK and opened it in Eclipse. I found some interesting stuff in strings.xml, but I think it's just the strings that are displayed to prompt the user This one stood out to me though:
Code:
<string name="hello_moto">OK Google Now</string>
Could it be that the phrase was originally going to be "hello moto" but when Google bought Motorola they changed it for branding purposes?
I think these other string names indicate that could be the case:
Code:
<string name="training_say_hello_moto">Say OK Google Now…</string>
<string name="training_second_hello_moto">Say OK Google Now a second time…</string>
<string name="training_third_hello_moto">Say OK Google Now a third time…</string>
<string name="training_final_hello_moto">Say OK Google Now a final time…</string>
Anyway, if someone wants to change that hello_moto string in the strings.xml file to something else, it might be worth a shot. I can't test any of this yet, because I can't root my phone because it has a spec of dirt under the lens and I'm mailing it back to Motorola to get fixed. But if somebody out there with the knowledge to change this setting in the strings.xml file of the apk wants to try it, I think it's a promising lead.
Hello moto would be cool.. Much better than the OK Moto Magic that was also on the cards... That would just be embarrassing! Good luck man
Sent from my XT1058 using Tapatalk 4
That application is an almighty mess. It has a tremendous amount of classes named things like "a", "aa", "ab", and so forth. Right now, I'm trying to make an Xposed module to send an intent to Tasker rather then start Motorola's interface when the key phase is said. This has been a nightmare of its own. I'm going to start digging for the start phase in earnest once I get this part done.
Sent from my Galaxy Nexus using XDA Premium 4 mobile app
hawkjm73 said:
That application is an almighty mess. It has a tremendous amount of classes named things like "a", "aa", "ab", and so forth. Right now, I'm trying to make an Xposed module to send an intent to Tasker rather then start Motorola's interface when the key phase is said. This has been a nightmare of its own. I'm going to start digging for the start phase in earnest once I get this part done.
Sent from my Galaxy Nexus using XDA Premium 4 mobile app
Click to expand...
Click to collapse
Names like that usually indicate that the code was obfuscated . I would be stunned if they coded it that way!
@glitch003 any luck with this?
Sent from non rooted motoX :'(
I'd totally change "Ok Google Now" to "Jarvis" if I ever had the chance.
hawkjm73 said:
That application is an almighty mess. It has a tremendous amount of classes named things like "a", "aa", "ab", and so forth. Right now, I'm trying to make an Xposed module to send an intent to Tasker rather then start Motorola's interface when the key phase is said. This has been a nightmare of its own. I'm going to start digging for the start phase in earnest once I get this part done.
Sent from my Galaxy Nexus using XDA Premium 4 mobile app
Click to expand...
Click to collapse
It's meant to be an almighty mess, so people like us struggle for months to figure out what the code actually does and to prevent copying of their code
While I haven't been able to change the wakeup phrase, I have changed what happens when you say it. With Tasker, I am able to launch AutoVoice and/or !utter instead of Google Now.
This essentially allows you to control your entire phone with voice, while it's still in your pocket.
Kev1000000 said:
While I haven't been able to change the wakeup phrase, I have changed what happens when you say it. With Tasker, I am able to launch AutoVoice and/or !utter instead of Google Now.
This essentially allows you to control your entire phone with voice, while it's still in your pocket.
Click to expand...
Click to collapse
what exactly are you not launching? From my understanding Moto's runs their own voice recognition stuff, then when it can't recognize the command it then sends it to google now. (even though it looks like it is just running googlw now the entire time it is not. So at what point in the process are you hijacking the code? Also, how easily can you use it? Does it take a long time or does it start just as quick as touchless controls/google now was.
jayboyyyy said:
what exactly are you not launching? From my understanding Moto's runs their own voice recognition stuff, then when it can't recognize the command it then sends it to google now. (even though it looks like it is just running googlw now the entire time it is not. So at what point in the process are you hijacking the code? Also, how easily can you use it? Does it take a long time or does it start just as quick as touchless controls/google now was.
Click to expand...
Click to collapse
It's pretty easy. I simply use Tasker to detect when the Touchless Controls app becomes the foreground app, then immediately kill its process. After that, I launch AutoVoice. With AutoVoice, I can then say "Search" to bring me to Google Now, or "utter" to launch !utter. I can also say any of my home automation commands like, "Turn off the lights" or "Watch TV."
I essentially use the Touchless Controls app as simply a detection of when the phrase was said, and then launch my own processes that I want to launch once it's detected, and kill the app immediately before it sends any data to Google Now.
It's seamless, and extremely quick.
Kev1000000 said:
It's pretty easy. I simply use Tasker to detect when the Touchless Controls app becomes the foreground app, then immediately kill its process. After that, I launch AutoVoice. With AutoVoice, I can then say "Search" to bring me to Google Now, or "utter" to launch !utter. I can also say any of my home automation commands like, "Turn off the lights" or "Watch TV."
I essentially use the Touchless Controls app as simply a detection of when the phrase was said, and then launch my own processes that I want to launch once it's detected, and kill the app immediately before it sends any data to Google Now.
It's seamless, and extremely quick.
Click to expand...
Click to collapse
you want to run a test with a lock on your phone? I'm interested to know if touchless control bypasses the lock or not. I know it can do certain things with the phone in lock. I would assume the few options we have has to do with the fact that once touchless controls pushes the command to google now because it doesn't recognize it they can no longer keep the phone in a locked state. Anyway, if you could run some tests on that and report back I'm interested. Thanks in advance.
I'd like to help
Hey could you upload the apk?
Although I haven't done any reverse engineering before but I have a lot of spare time right now and I'd like to try and help. I just got a developer edition of the Moto X and annoyingly it doesn't seem like there's any easy way to root it right now, since it came with the newest update(at least as far as I can tell).
The only reason that matters to this is that I can't get the apk off of the device, as I don't have the sufficient permissions for data/app. If there's another way to do this then please let me know too!
Thanks
EDIT: nevermind, I realized there's a way to root it if I just unlock the device.
Recompiling with signature
I'm not sure if there is still any interest in this, but I've been trying to work on it nonetheless as a personal project.
This is my first time doing any sort of reverse engineering, and my worry is that we won't be able to sign the app after modifying it.
From what I've read of using apktool, it seems that if you want to keep the signatures you have to move the modified files you have back into the original apk.
Problem is, on OS X there doesn't seem to be a good way to open apks and modify them. Can someone else try and see if it can be recompiled and reinstalled?
Thanks
I messed with this for a short while. I found you can do it just by training. I was able to say "OK f$%& Wad", "OK Moto" and "OK Siri" (joke for my wife). I thought it was due to hex editing some files but that was only a coincidence. Train a phrase using "OK [your word] Now". Sometimes it is tough. Now most of the time you can start it with only "OK [your word]". They use some of the Nuance sdk. Nuance app looks for syllables if I read correctly. I will dig into the files again to see if it can be done more easily. All we need is one guy shouting the phrase and 10 phones want to react. ?
Good luck.
DroidJunk said:
I messed with this for a short while. I found you can do it just by training. I was able to say "OK f$%& Wad", "OK Moto" and "OK Siri" (joke for my wife). I thought it was due to hex editing some files but that was only a coincidence. Train a phrase using "OK [your word] Now". Sometimes it is tough. Now most of the time you can start it with only "OK [your word]". They use some of the Nuance sdk. Nuance app looks for syllables if I read correctly. I will dig into the files again to see if it can be done more easily. All we need is one guy shouting the phrase and 10 phones want to react. ?
Good luck.
Click to expand...
Click to collapse
Well the idea is that we want this to work even if you don't say OK, and you should be able to just say anything you want. Also, it should work without having to try many times.
On a side note, I've run into the issue where basically I can not install on top of the app because it's installed as a system app. I'm not too sure if there's any way to get around this besides removing the permissions check from the device which would be a huge security hole. I also don't even know if that itself will work..
It is probably a signature issue. I have a complete deodex of the original KK dump. If you sign with the same key that might help. You can verify by looking at your logcat when trying to install. Check out data/data/blabla audiomonitor. In the files Dir there is a file svsid_trigger. This is recreated each time you train. I would believe it to be a voice sample if you want to call it that. There are other files in there that when opened show the phrase OK Google now. This is where I hex edited but it did nothing. It will even run without some of these files. My theory is one of these files contains a sample of what it should look like. Decompiling the apk is about as messy as you described. When you check the prefs you see the phrase is set to custom phrase which appears to be a function/class. This also gets rewritten when the phone is booted. Curious to see what happens. I really expect Google or moto to change this. It just does not make sense to have all the phones responding to the same thing. Even with training others can activate it. I was watching a video on the phone and that activated it. LOL
DroidJunk said:
It is probably a signature issue. I have a complete deodex of the original KK dump. If you sign with the same key that might help. You can verify by looking at your logcat when trying to install. Check out data/data/blabla audiomonitor. In the files Dir there is a file svsid_trigger. This is recreated each time you train. I would believe it to be a voice sample if you want to call it that. There are other files in there that when opened show the phrase OK Google now. This is where I hex edited but it did nothing. It will even run without some of these files. My theory is one of these files contains a sample of what it should look like. Decompiling the apk is about as messy as you described. When you check the prefs you see the phrase is set to custom phrase which appears to be a function/class. This also gets rewritten when the phone is booted. Curious to see what happens. I really expect Google or moto to change this. It just does not make sense to have all the phones responding to the same thing. Even with training others can activate it. I was watching a video on the phone and that activated it. LOL
Click to expand...
Click to collapse
I know it's a signature issue when I am trying to install it, but even when I uninstall and install again, I get a permissions issue because it's not installed as a system app.
Also, changing the value of the audio file is interesting, but the problem is that unless we know the algorithms to generate the audio files, I don't know how we can adjust that. It seems cleaner to just go into the code and remove the checks altogether. On the other hand, it takes more investigating, and right now I am completely blocked on this. I don't have much time to mess with this anymore unfortunately, but I do want to try to remove the permissions altogether and see if it will let me install the app as a user app.
Synderesis said:
I know it's a signature issue when I am trying to install it, but even when I uninstall and install again, I get a permissions issue because it's not installed as a system app.
Also, changing the value of the audio file is interesting, but the problem is that unless we know the algorithms to generate the audio files, I don't know how we can adjust that. It seems cleaner to just go into the code and remove the checks altogether. On the other hand, it takes more investigating, and right now I am completely blocked on this. I don't have much time to mess with this anymore unfortunately, but I do want to try to remove the permissions altogether and see if it will let me install the app as a user app.
Click to expand...
Click to collapse
Check here when you get around to decompiling audiomonitor.apk. \AudioMonitor.apk\smali\com\motorola\audiomonitor\uis\training. It might be a good start. I use to be good at smali, but it has been a couple of years.
I've read a couple of threads on the "context" that my module runs in, but I'm not clear on how I send data from my module (running in the hooked application context) back to my settings screen (running in the Xposed context?)
XSharedPreferences is obviously read only, so that doesn't work. I'm guessing I need to write to a file and parse it back from settings, but I'm hoping someone can set me in the right direction. Is there a module that already does this so I can look at the source? Or can someone give me a high level of the file permissions/location/settings to use so that it's readable/writable from both contexts?
Thanks!
Ryan
I'd say your best option is to register a broadcast receiver in your app, and send a broadcast from the hooked app (you just need a Context to do that.
If you can't get one from the app, you could use AndroidAppHelper.currentApplication()).
GermainZ said:
I'd say your best option is to register a broadcast receiver in your app, and send a broadcast from the hooked app (you just need a Context to do that.
If you can't get one from the app, you could use AndroidAppHelper.currentApplication()).
Click to expand...
Click to collapse
Thank you! I got it. I guess that was obvious, but it seemed like there might be a "tighter" way to do the cross-process communication using Xposed as a bridge.
Thanks for all of your help in this forum,
Ryan
So far my attempts to google this answer have come up empty. I can open Google Maps driving mode with AutoShortcut no problem. But making it actually close is turning out to be impossible.
If I try to do a "Kill App", it doesn't kill the notification, leaving it in a bizarre half-open state. Has anyone found an intent or something that can actually kill this off properly?
Have you tried to run a Go Home action before killing the app?
Yes, unfortunately does not help at all. Even gave it ~3 second wait.
Vincent Law said:
So far my attempts to google this answer have come up empty. I can open Google Maps driving mode with AutoShortcut no problem. But making it actually close is turning out to be impossible.
If I try to do a "Kill App", it doesn't kill the notification, leaving it in a bizarre half-open state. Has anyone found an intent or something that can actually kill this off properly?
Click to expand...
Click to collapse
Try code run shell: am force-stop com.google.android.apps.maps & tick "use root". Change the package name if is necessary.
My device is not rooted, is that going to do anything?
Vincent Law said:
My device is not rooted, is that going to do anything?
Click to expand...
Click to collapse
Root is needed to run that command.
But did you try to kill another app with the method you've tried for google maps? Is it working?
So I managed to work out a solution to this, and it involves using the AutoNotifications plugin. I use the plugin to instruct Tasker to press the "end navigation / stop driving" button in the notification tray, which does the correct thing. For those looking to replicate, it's an "internal action", so use the first internal action, not one of the action button options that most guides suggest.
Vincent Law said:
So I managed to work out a solution to this, and it involves using the AutoNotifications plugin. I use the plugin to instruct Tasker to press the "end navigation / stop driving" button in the notification tray, which does the correct thing. For those looking to replicate, it's an "internal action", so use the first internal action, not one of the action button options that most guides suggest.
Click to expand...
Click to collapse
i am trying to "stop driving" using autonotification and cant figure it out. Can you please be more specific than "internal action" which internal action. What variable did you using for the stop driving button. I am trying to use autonotification query and action but i cant figure out which variable in the query to use for the action.
thanks
Vincent Law said:
So I managed to work out a solution to this, and it involves using the AutoNotifications plugin. I use the plugin to instruct Tasker to press the "end navigation / stop driving" button in the notification tray, which does the correct thing. For those looking to replicate, it's an "internal action", so use the first internal action, not one of the action button options that most guides suggest.
Click to expand...
Click to collapse
I'm new to autonotification - I have it set up to query for the "Maps" notification, and it's returning the %anapp variable correctly - I can even click on the notification and bring the focus back to the maps app - but how the heck do you fire this "internal action" to stop driving? I'm sure it must be obvious...
Bulldoguk72 said:
I'm new to autonotification - I have it set up to query for the "Maps" notification, and it's returning the %anapp variable correctly - I can even click on the notification and bring the focus back to the maps app - but how the heck do you fire this "internal action" to stop driving? I'm sure it must be obvious...
Click to expand...
Click to collapse
I just spent a few days trying to figure this out. Here is how I was able to get it working:
Create a new task / add these to a current task:
A1 - Add Action - Plugin, AutoNotification, Query. Tap Edit, Select "Maps" from Apps and under Advanced tap Internal Actions Big, tap the Tag icon. It should populate with %aninternalactions.
A2 - (Only needed for debugging) Add Action, Alert, Flash, tap the Tag and choose %aninternalactions, then edit to choose the first value in the array - %aninternalactions(1)
A3 - Add Action, Plugin, AutoNotification, Actions. Tap Intercept Action, tap the tag and choose %aninternalactions, then edit it again to - %aninternalactions(1)
To test, start driving mode and run the task manually. You should see the button GUID flash (if you enabled A2) and Driving Mode should end :good:
It works now
I figured out a solution.
Tasker gives you the advise to use ADB Wifi.
Here's what I did:
- enable and go to developer settings
- check ADB debugging
- check ADB while charging (only needed, if it stops working for you when you disconnect your USB cable. Try without first!)
- then run from your PC `adb tcpip 5555` to enable adb over wifi
- use Tasker Action "kill app", uncheck "Use Root".
Maps (or Driving Mode) can finally be killed this way.
Tried on Huawei Mate 20 Pro (LYA-L29) running Android 10 (10.0.0.214)
How are you able to open Google maps?
Mine opens for a millisecond then closes.
MSK1 said:
How are you able to open Google maps?
Mine opens for a millisecond then closes.
Click to expand...
Click to collapse
Hi, MSK1,
I searched for a long time for this solution, so maybe this one will help you out.
You can launch Google Maps directly into Navigation mode via using an intent.
You may find this one interesting. https://forum.xda-developers.com/showpost.php?p=46600949&postcount=3
Mine looks like this.
google.navigation:q=Zip+City,+Street+Number
Happy Driving!
Best Regards.
3di said:
Hi, MSK1,
I searched for a long time for this solution, so maybe this one will help you out.
You can launch Google Maps directly into Navigation mode via using an intent.
You may find this one interesting. https://forum.xda-developers.com/showpost.php?p=46600949&postcount=3
Mine looks like this.
google.navigation:q=Zip+City,+Street+Number
Happy Driving!
Best Regards.
Click to expand...
Click to collapse
Thanks.
I found an easier way.
I downloaded the "navigation shortcut" app from the play store.
When I launch this as an app from tasker then the navigation stays on.
Sent from my EVR-N29 using Tapatalk
I have a profile that will turn off wifi and turn data on when I leave work.
I am trying to add a task to this that will open copilotand set the navigation to on of my saved destinations.
Can anyone help me with this please?
Never used copilot but i am pretty sure it is possible with autoinput.
You launch the app with tasker, and use a combination of autoinput commands, to have it open what you want.
The "intent" stuff that i know nothing about might be helpful as well.
Sent from my SM-G900F.
Thank, I hope someone could expand further with the items you mentioned please?
You could take a look at its intents on manifest file of the apk.
CrashOverride93 said:
You could take a look at its intents on manifest file of the apk.
Click to expand...
Click to collapse
Sorry haven't a clue what this means?
Tried using autonotification, it seems like it doesn't recognize what is clickable and what is not within your app and so i couldn't do anything with it. Havn't used it for a long time so i might be missing out on something. I probably am.
About intents check this out. Was just reading it since i don't know about intents either. It was interesting http://forum.xda-developers.com/showthread.php?t=2489449
Its an interesting read. Tried using an intent for google maps, your app recognized it. I could feed an address to the app but it will simply display it on the map.
There is most likely an intent for google maps to navigate to an address from your current position. If thats the case, your app most likely will recognize it as well which is what you are looking for.
Sent from my SM-G900F.