Mycroft: the open source Alexa

Mycroft AI is a voice assistant similar to Amazon's Echo/Alexa, with a speaker module that the user can shout instructions to.

In both cases the user can "wake up" the unit, and then ask a question or issue a command (e.g. "What's the weather like in Nairobi?").

But what makes Mycroft especially interesting to an old hacker like me, is that it is open source!

Not only is the software open source, but you don't need to invest in special hardware. If you are a Raspberry Pi hacker you may have most of the bits you need already. Even better, there is a Pi pre-built software image called PiCroft. You just download the image file and blast it onto an 8GB SD card in the usual way.

my Mycroft

So I just re-imaged an SD card with Picroft and plugged it into my desktop Pi-3 system. This system already has a power supply and a little audio amp with a small speaker attached. I removed the Edimax wifi dongle, plugged in Adafruit USB audio adapter and attached an electret microphone.

note the clever use of BluTak as a mic clip

When the Pi boots with the Picroft image, the onboard wifi interface of the Pi3 is automatically configured as a wifi access point called: MYCROFT

You then need to connect another device (e.g. your laptop) to the MYCROFT access point using the password: MYCROFT1

Next, use the browser on your laptop to access a page on your Mycroft device: http://start.mycroft.ai

This page takes you through the steps to configure Mycroft for internet access via your home wifi and register/pair your Mycroft device using a unique code. This process is easier than it sounds, but I did have to reboot at one stage when the on-screen instructions said that the code was displayed below ...but it wasn't.

Refer to the full procedure here:  https://docs.mycroft.ai/installing.and.running/installation/raspberry.pi

you need 'skills'

A Mycroft skill defines the ability of the system to react to a command or question.

For example; if you say "Hey Mycroft, what is your IP address?" the system has to wake up, identify your command, take some action and return a result. In this example the system is woken up by the phrase "Hey Mycroft" and then it listens to the phrase that follows. From the second part of the phrase it has to match words to determine the "intent" of the message.

We can look at the skill that enables Mycroft to identify our request for the IP address by looking in the skills directory: /home/pi/skills

I prefer to do this via my Lubuntu laptop, so I open the file manager and select Go > Connect to server...  (most file managers have a "Connect to server" option).

I use this dialog to create an SSH connection to Mycroft (host = ip of my Pi) at /home/pi (which is a good starting point) as user: pi

The default password is: raspberry

Clicking on the skills shortcut takes us to the list of currently installed skills. The one we need is called: skill-ip

Open the file ../skill-ip/vocab/en-us/IPCommand.voc in a text editor via the file manager. There are 3 lines of text:-

• ip address
• ip
• network address

So we can speak any of these 3 phrases, and Mycroft will respond by telling us the IP address. You could also say something longer, such as "would you mind telling me your IP address, please" as this contains IP address.

Take a look at the __int__.py Python code to see how Mycroft finds the IP address.

write a simple skill

Let's create a new skill, and keep it as simple as possible.

1. create a new folder in the skills directory called: a-marvin-skill
2. create 4 new folders within a-marvin-skill: dialog > en-us, vocab > en-us
3. within folder; a-marvin-skill > vocab > en-us create a file called: MarvinKeyword.voc containing the text: what's up marvin
4. within folder; a-marvin-skill > dialog > en-us create a file called: marvinspeaks.dialog containing the text: I don't know, I've never been there.
5. within folder; a-marvin.skill create a new file: __init__.py

In the new file __init__.py (that's 2 underscores before & 2 after init) add some Python code:-

from adapt.intent import IntentBuilder
from mycroft.skills.core import MycroftSkill
from mycroft.util.log import getLogger

__author__ = 'stevedee'

LOGGER = getLogger(__name__)

class MarvinSkill(MycroftSkill):
    def __init__(self):
        super(MarvinSkill, self).__init__(name="MarvinSkill")

    def initialize(self):
        marvin_intent = IntentBuilder("MarvinIntent").require("MarvinKeyword").build()
        self.register_intent(marvin_intent, self.handle_marvin_intent)

    def handle_marvin_intent(self, message):
        self.speak_dialog("marvinspeaks")

    def stop(self):
        pass

def create_skill():
    return MarvinSkill()

Now say "Hey Mycroft, what's up Marvin" and see if it works. If it doesn't work, check the Mycroft screen in case it thought you said Melvin, or didn't understand what you said for some other reason.

This is just an intro, so the next step is to read the official documentation and learn how to do the job properly.

Mycroft core overview:  https://docs.mycroft.ai/overview
Mycroft skills:  https://github.com/MycroftAI/mycroft-skills
Mycroft wiki:  https://github.com/MycroftAI/mycroft-core/wiki
Mycroft forum: https://community.mycroft.ai/

To install skills from Mycroft-skills on Github, I generally SSH into my Pi (i.e. ssh pi@{ipaddress}) and then type at the prompt:-

git clone https://github.com/CloneMMDDCVII/systemSkill-Mycroft-core-Skill-.git

...where the https address is just copy/pasted via the Git "clone or download" button. You probably need to restart Mycroft (or reboot Pi) to use a newly installed skill.

As there are a number of other privately created skills on Git, it is also worth doing an internet search for: mycroft skill git

odds 'n ends

The Mycroft screen displays the speech log by default. This is very handy when testing the system. To break out of this to use the command line just type: ctrl c

To return to speech log just type at the command prompt: view_log

The cli mode allows you to type key words or phrases to test skills. It includes log information and therefore makes it easier to debug problems.

On the bottom right corner is a simple volume level indicator, which is the most basic way to check that the microphone is working.

However, if you leave this screen (by pressing Ctrl and c) and return to the Linux command prompt, you should be able to do a better test of the microphone quality by typing at the command prompt: test_microphone
...but this doesn't seem to work for me. The screen says "Playing back the recording..." and I just hear noise.

the choice of microphone

If you are just playing with Mycroft, the performance of the attached microphone is not too important. But for best operation in a typical living room, the system needs to be able to pick out your voice over the background noise. This does not mean getting the most expensive mic available, but you may need to test one or two, or read the recommendations in the Mycroft wiki.

My small electret microphone capsule works surprisingly well. I can certainly communicate with Mycroft from a about 3m away, despite having to stand outside my man-cave and speak from the hallway.

I understand that commercial systems, like the Amazon Echo, use microphone arrays to aid better receiption. This sounds like an interesting area of experimentation. I wonder whether there is any benefit in mounting the microphone(s) remote from the systems loudspeaker?

conclusion

Obvously the number and range of skills is very limited at the moment. With the recent release of Picroft, I think that some aspects of development will now "speed up" as more and more users install the system and lend a hand by providing test data, feedback and new skills.

I don't think I want a system like this to switch my lights on/off, change the channel on my TV, or fiddle with my central heating (these are things I can easily do myself). But maybe it could be useful for finding stuff on my laptop, or answering one of the many silly questions that come into my mind during the course of an average day.

Any real benefit from robots or virtual assistants are going to come by them doing things that either I don't want to do myself or are too difficult. However, AI systems are very smart. They can now give you legal advice, but appear to be very bad at cleaning toilets  ...now that's what I call being really smart!