Is this working? about oa-core HOT 10 CLOSED

Thanks Joshua. I'm just a linux end-user, which is to say more sophisticated than a typical windows user but still no coder. Hacking the root mind python file sounds beyond my skills. Populating a config file with things I can say and bash commands I want to execute when I say them is somthing I could manage, if smarter people ever implement the capability. I fgure there'd need to be a couple hundred commands, minimum, especially since I saw another issue that seemed to state commands can't take parameters, so "open firefox" and "open firefox to gmail" would have to be separate commands.

What would be really useful is to convert speech-to-text and then pass the text, frex "open argument" would pass "argument" to krunner (or similar app on desktops other than plasma). Do I understand that passing an argument is impossible right now?

from oa-core.

PS: My distro is openSUSE Tumbleweed, if that makes a difference. -GEF

from oa-core.

Looks pretty normal.. except for the OA> [2020-05-02 00:36:06,223] INFO mind [init.py:load_minds:55]: Loading minds... part.

Try with python3 -m oa -d?

from oa-core.

Wow, you're up late, thanks for taking a look. Same behavior, whih is to say no response. I verfied on dictation.io that my hardware works. Here's the output:

gef@purplebox:~/oa-core> python3 -m oa -d
[2020-05-02 02:28:08,425] INFO MainThread [main.py::102]: Start Open Assistant
[2020-05-02 02:28:08,430] INFO MainThread [hub.py:_load_modules:37]: Loading Modules <- /home/gef/oa-core/oa/modules
[2020-05-02 02:28:08,430] INFO MainThread [util.py:load_module:60]: sound <- /home/gef/oa-core/oa/modules/sound
[2020-05-02 02:28:08,432] INFO MainThread [util.py:load_module:60]: mind <- /home/gef/oa-core/oa/modules/mind
[2020-05-02 02:28:08,669] INFO MainThread [util.py:load_module:60]: voice <- /home/gef/oa-core/oa/modules/voice
[2020-05-02 02:28:08,671] INFO MainThread [util.py:load_module:60]: speech_recognition <- /home/gef/oa-core/oa/modules/speech_recognition
[2020-05-02 02:28:08,678] INFO MainThread [util.py:load_module:60]: ear <- /home/gef/oa-core/oa/modules/ear
[2020-05-02 02:28:08,890] DEBUG sound [hub.py:thread_loop:68]: Starting
[2020-05-02 02:28:08,890] DEBUG mind [hub.py:thread_loop:68]: Starting
[2020-05-02 02:28:08,891] DEBUG voice [hub.py:thread_loop:68]: Starting
[2020-05-02 02:28:08,891] DEBUG speech_recognition [hub.py:thread_loop:68]: Starting
[2020-05-02 02:28:08,892] DEBUG ear [hub.py:thread_loop:68]: Starting
OA> [2020-05-02 02:28:08,894] INFO mind [init.py:load_minds:55]: Loading minds...
[2020-05-02 02:28:08,895] INFO mind [init.py:load_minds:59]: <- root.py
[2020-05-02 02:28:08,986] INFO mind [init.py:load_minds:59]: <- boot.py
[2020-05-02 02:28:08,986] INFO mind [init.py:load_minds:62]: Minds loaded!
[2020-05-02 02:28:08,987] INFO mind [init.py:set_mind:41]: Opening Mind: boot
[2020-05-02 02:28:08,987] DEBUG mind [init.py:_in:70]: "boot" is now listening. Say "Boot Mind!" to see if it can hear you.

from oa-core.

There are a few things that could be going wrong. After seeing this issue, I got motivated to do some cleanup.. I don't think there's going to be a fix in the next few hours, but this output is interesting.

It's not really documented, but using -l <FILE> writes all the logging stuff to a file instead of stdout. There's a kind of REPL that accepts commands like <MODULE> <ARG> (e.g. voice say something.

In addition to that OA> thing (from the REPL), our load/start orders seem different. That might matter, but it shouldn't..

It might be load order or threading issues; maybe something with the ear or speech_recognition modules (or their dependencies).

from oa-core.

It might be load order or threading issues

Interesting. My laptop has an A12 cpu, an odd duck from before AMD got good (which is why I got it cheap), I have another laptop that I don't normally use because it's damaged, but it has a 7th-gen i7 with the same OS, a near-identical installation. While I've never had this kind of problem with the A12 before, based on your comment I tried on my intel laptop and it went smoothly. So it looks like we can chalk this up to hardware and close the issue.

So if I understand correctly, now I have to figure out how to program voice commands that I'll find useful? Krunner is such a useful tool as-is that if it were voice-activated, it'd be a pretty good secretary.

from oa-core.

Um, it's not obvious to me were the commands live. Is there a config file that I'm over-looking because it's in plain sight?

from oa-core.

It might be load order or threading issues

So it looks like we can chalk this up to hardware and close the issue.

Glad you got it working on a different system! Give it a try (on some later release in the future) if you're still interested, it might work. Either way, stuff's getting cleaned up.

So if I understand correctly, now I have to figure out how to program voice commands that I'll find useful? Krunner is such a useful tool as-is that if it were voice-activated, it'd be a pretty good secretary.

Yeah, that's a good place to start.

from oa-core.

Um, it's not obvious to me were the commands live. Is there a config file that I'm over-looking because it's in plain sight?

If only..

The quickest way to a custom command right now is by customizing the 'root mind' -- https://github.com/openassistant/oa-core/blob/master/oa/modules/mind/minds/root.py. I think there are a few gotchas with trying add a new mind, so you can just hack away on this one.

Make your changes, restart OA, when 'boot mind' is active say "open assistant" (command to switch to root mind, same as typing mind open assistant, iirc).

The important thing to note is the @command decorator; put that on a function you want to call, and give it a phrase or a list of phrases; it doesn't matter what the function is named, @command is for registering a phrase to a function.

When OA starts, it calls a web service to get some files for speech recognition (it takes all the phrases registered, and gets a pronunciation dictionary and ngram). That's not great, but that's how it currently works.

That's the basic idea, as brief as I could make it. Let me know how it goes!

from oa-core.

Do I understand that passing an argument is impossible right now?

Yes. And end-user ease.. isn't

I'll close this issue, and do what I can on the underlying issues.

from oa-core.