MirrorMirrorOnTheWallSkill

Mirror Mirror On The Wall

Build your own voice-controlled Magic Mirror Instructions

Table of Contents

• Prerequisites
• Magic Mirror
  • Installation
• AWS IoT Core
• Alexa Skill
  • Configuration
• AWS Lambda
  • Get Lambda function code ready
  • Deploy to AWS Lambda
  • Test
• Magic Mirror Module
• Have Fun!
  • Display text
  • Turn on/off Magic Mirror Modules
  • Display images
  • Display video

Prerequisites

• Amazon AWS developer account
• Alexa Skills developer account
• Install Node.js v10.x
• *Install Git

*Windows users: You must have a Linux Bash Shell to do this Workshop. When you are installing Git, please follow these instructions to use the Unix tools from the Windows Command Prompt.

Magic Mirror

repo: MichMich/MagicMirror

We are using an open source MagicMirror² software platform that will allow you to convert your hallway or bathroom mirror into your personal assistant.

To toggle the (web) Developer Tools from mirror mode, use CTRL-SHIFT-I or ALT and select View.

Installation

1. Open your command line

2. Download the repository to your laptop

   git clone https://github.com/MichMich/MagicMirror.git

3. Enter the repository

   cd MagicMirror/

4. Install node libraries

   npm install

5. Navigate to the config folder and rename config.js.sample to config.js

   cd config/

   mv config.js.sample config.js

6. Run the app

   npm start

AWS IoT Core

We need to setup an AWS IoT Device Gateway. It works as a hub that will allow an Alexa Skill to send messages to it and a Magic Mirror to subscribe to new messages from it. This will let us control our Magic Mirror with our voice.

1. Login to AWS Management Console

   

2. Choose US East (N. Virginia) region

   

3. Find IoT Core service

   

4. Click on Onboard at the left menu bar, under Configuring a device, click on Get Started

   

5. Click on Get Started

   

6. Choose Linux/OSX platform, and Node.js

   

7. Name your device MagicMirror

   

8. Download credentials, aka connection kit

   

9. Click on Next step

   

10. Follow the instructions to run the start.sh script, which will generate a root-CA.crt

    

    

    

    

    

    • Note: Don’t worry about errors as long as you get a root-CA.crt file
    • Note note: You will need to copy the credential files here for Lambda function deployment later

11. Go back to the AWS web page, click on Done

12. Open the MagicMirror Thing you just created

13. Click on Interact on the left menu bar, see the HTTPS Rest API Endpoint in the format of xxxxxxxxxxxxxxx.iot.us-east-1.amazonaws.com? You will need to copy this endpoint for Lambda function deployment later

    

Alexa Skill

In this section, we build the language model for our Alexa Skill by defining the commands that Alexa can recognize.

Configuration

1. Go to the Alexa Developer Console

2. Click on the Create Skill button.

   

3. Name your Skill (any name is fine), select Custom model, and click the Create a skill button.

   

4. Choose Start from scratch template

5. Click on Invocation on the left menu bar. Set “magic mirror” as the invocation name (or any unique phrase). This is the phrase used to activate your skill. Save Model.

   

6. Click on JSON Editor on the left menu bar. Delete everything in there.

7. Go to InteractionModel.json in the speechAssets folder, and copy everything in the file, and paste it in JSON Editor. Save Model. Build Model

   

8. Click on Endpoint on the left menu bar. Select AWS Lambda ARN. Copy your Skill ID to be used later in Lambda function deployment

   

AWS Lambda

repo: joanaz/MirrorMirrorOnTheWallSkill

In this section, we deploy our AWS Lambda function for our Alexa skill. Our Lambda function processes our voice commands and tells Alexa how to respond, and sends our commands to the AWS IoT Device Gateway.

Get Lambda function code ready

1. Download the GitHub repository to your laptop

   git clone https://github.com/joanaz/MirrorMirrorOnTheWallSkill.git

2. Install node libraries

   cd MirrorMirrorOnTheWallSkill/src/

   npm install

3. Navigate to the certs folder and rename keys_sample.json to keys.json

   cd certs/

   mv keys_sample.json keys.json

4. Copy the credential files generated in Section AWS IoT Core. Then open the local MirrorMirrorOnTheWallSkill folder you downloaded, go to src, then certs, paste your credential files here

   

5. Open MirrorMirror.js in src folder with a text editor. Copy and paste your HTTPS Rest API Endpoint to line 17, replacing YOURID.iot.us-east-1.amazonaws.com

   

6. Open index.js in src folder with a text editor. Copy and paste your Alexa Skill ID to line 18 for the variable APP_ID, replacing amzn1.ask.skill.YOURALEXASKILLID, and save the change

   

Dependencies

Here’s the list of node libraries you installed in the above step. If you are interested in learning more about them, you can go and check out their links.

• alexa-sdk

• aws-iot-device-sdk

• Google Images Search

  OPTIONAL: It is for showing images on the Magic Mirror. Follow the instructions in the link to create your own Google Custom Search Engine, and save the CSE ID and API key in certs/keys.json (see keys.json below).

• Youtube API.

  OPTIONAL: It is for showing a video on the Magic Mirror. Watch this instruction video to create your own Youtube API key, and save it in certs/keys.json (see keys.json below).

keys.json

The keys.json file is where you put your Google Images Search and YouTube API keys.

// keys.json
{
    "cse": {
        "ID": "YOUR GOOGLE CUSTOM SEARCH ENGINE ID",
        "API_key": "YOUR GOOGLE PROJECT API KEY"
    },
    "youtube": {
        "API_key": "YOUR YOUTUBE API KEY"
    }
}

Deploy to AWS Lambda

1. Go inside your local src directory, select all files and folders, then create a .ZIP file.
   • Note: Make sure the .ZIP file does not contain the src directory itself, otherwise Lambda function will not work.

   

   

2. Go to the AWS Management Console

3. Find Lambda service.
   • Note: make sure you are in the US East (N. Virginia) region or you won’t be able to use Alexa with Lambda.

   

4. Click on the Create a Function button.

   

5. Choose Author from scratch. Name the function (any name is fine).

   

6. Keep “Create new role from template(s)” and enter a role name (any name is fine)

   

7. Click on the Create function button.

8. Under Add triggers menu on the left, click on Alexa Skills Kit

   

   

9. Disable Skill ID Verification. Click Add button. Then scroll up and click the orange Save button

   

   

10. Click on the box with your function name in the Designer section. Select Code entry type as “Upload a .ZIP file”

    

11. Click on Upload button, then upload the zip file created in Step 1. Then Save

12. Copy the ARN from the top right.

    

13. Go back to the Alexa Developer Console, click on Endpoint on the left menu bar, and paste the ARN to Default Region. Click Save Endpoints

    

Test

You can now test your Alexa skill by going to the Test tab on the Alexa Developer Console and enabling testing. Type “start magic mirror” in the textbox and see what happens!

You can also test your Alexa skill on EchoSim, by saying “Alexa, ask magic mirror to say hello”.

Another testing tool is the Amazon Alexa website, where you can see each of your Alexa voice command and Alexa’s response. If you get the image command to work, you will see the image on the card too.

Magic Mirror Module

repo: joanaz/MMM-MirrorMirrorOnTheWall

We’re Finally up to the last step! We add the Magic Mirror Module to MagicMirror² which subscribes to incoming messages from the AWS IoT Device Gateway and displays text/images/video on the MagicMirror², and turns on/off other Magic Mirror Modules according to our commands.

1. Navigate to the MagicMirror repo we downloaded in Section 1

   cd MagicMirror/modules/

2. Download our Magic Mirror Module code inside the modules folder

   git clone https://github.com/joanaz/MMM-MirrorMirrorOnTheWall.git

3. Install Node libraries

   cd MMM-MirrorMirrorOnTheWall/

   npm install

4. Copy the certs folder in your Lambda function code (under MirrorMirrorOnTheWallSkill/src) to the MMM-MirrorMirrorOnTheWall folder

5. Open MirrorMirror.js in MMM-MirrorMirrorOnTheWall folder with a text editor. Copy and paste your IoT Device HTTPS Rest API Endpoint to line 18, replacing YOURID.iot.us-east-1.amazonaws.com

   

6. Copy below code and add it to MagicMirror/config/config.js

    // modules: [
    // ...
    {
        module: 'MMM-MirrorMirrorOnTheWall',
        position: "middle_center",
        config: {}
    },
    // ]
   

Have Fun!

Now you have completed all the steps! Congratulations!

You can invoke your Alexa skill by saying Alexa, start Magic Mirror. Next, you can say any of the following commands to trigger different actions on your Magic Mirror.

Display text

The text in {} will be displayed on Magic Mirror in bold.

• “say {hello}”
• “say {good morning}”
• “say {you are the fairest of them all}”
• “display text of {hello}”
• “display text of {good morning}”
• “show text of {hello}”
• “show text of {good morning}”

Turn on/off Magic Mirror Modules

To turn on/off a Magic Mirror Module, it has to already be installed and configured in MagicMirror². You also have to map its official module name to a transcribable spoken name in ModuleNames.json. For example, we can map “MMM-Globe” to “globe”, or “currentweather” to “current weather”.

To turn on a Magic Mirror Module, say:

• “start {newsfeed}”
• “start {current weather}”
• “turn on {compliments}”
• “open {smile test}”

To turn off a Magic Mirror Module, say:

• “close {newsfeed}”
• “close {current weather}”
• “turn off {compliments}”
• “finish {smile test}”

Note: To clear the text/images/video displayed by this module, you can simply turn this module off.

To turn on all Magic Mirror Module, say:

• “open all”
• “open all modules”
• “open every module”
• “open each module”
• “show all modules”
• “show me all modules”
• “show every module”
• “show each module”
• “turn on all”
• “turn on all modules”
• “turn on every module”
• “turn on each module”
• “start all”
• “start all modules”
• “start every module”
• “start each module”

To turn off all Magic Mirror Module, say:

• “close all”
• “close all modules”
• “close every module”
• “close each module”
• “hide all”
• “hide all modules”
• “hide every module”
• “hide each module”
• “turn off all”
• “turn off all modules”
• “turn off every module”
• “turn off each module”

Display images

The text in {} will be searched by Google Image Search API, and the returned images will be displayed on your Magic Mirror with the text.

• “find {snow white}”
• “find images of {hunter}”
• “find pictures of {dwarfs}”
• “show me {snow white}”
• “show me pictures of {hunter}”
• “show me images of {dwarfs}”
• “show pictures of {hunter}”
• “show images of {snow white}”
• “display pictures of {dwarfs}”
• “display images of {dwarfs}”

Display video

The text in {} will be searched by Youtube Data API, and the returned video will be displayed on your Magic Mirror, with the text. The YouTube video autoplays on a loop.

• “show me how to {make slime}”
• “show me video of {movie trailer}”
• “show me a video of {cats}”
• “show video of {volcanoes}”
• “show a video of {birds}”
• “display video of {animals}”
• “display a video of {rattlesnakes}”
• “find video of {cat}”
• “find video of {cat and dog}”
• “find a video of {snow white}”

Bonus

Instructions on Setting up on Raspberry Pi

This site is open source. Improve this page.