Making a Raspberry Pi speak: Alexa

Speaking with devices (and making them answer or do something) seems to be a trend of the time. Some up-to-date smartphones and tablets allow allow to use speech to trigger internet searches, to write short messages or e-mails (sometimes with funny results), to ask for something in the region, to turn the light of the smartphone on, … .

In addition to voice control on smartphones, well-known companies started to launch devices to enable voice control @home. However, the commercial solutions perform speech recognition on their own servers. AFAIK due to computing power requirements of the AI behind.
In this case one has to live with the fact that a constant internet connection is inevitable and that own voice samples are uploaded somewhere else for analysis.

Still, speech control can be extremely useful. My favourite example for illustration is setting a timer while being busy with something else.
Though, in a smart home there are many more applications for speech control: light, heating, media, … . Even for elderly, handicapped or visually impaired controlling everyday procedures by the own voice can be a huge advantage in the daily life.

Open Source Solution: Jasper

The open source solution for voice control, Jasper, offers the possibility to work offline, but the setup of the software is not trivial. It looks like the manual is outdated. Some experimental, but required libraries are not to be found easily anymore. This is why I turned to the API of a commercial solution to play with speech recognition on my Raspberry Pi 3.

At the moment speech recognition devices such as Amazon’s Alexa are not sold everywhere yet. It is possible to order them in Europe, but they are not shipped yet. As rumour has it: regions in which stronger accents are spoken are served first. 🙂

Amazon’s Alexa

The voice service that is used by Amazon’s Alexa devices can be relatively easy tested on a Raspberry Pi 3. Since a couple of weeks wake word detection in this solution is possible on the Raspberry Pi 3 as well.


Raspberry Pi 3 (incl. power supply, display, keyboard and mouse for setup)

USB microphone

Non-bluetooth speaker


Amazon Developer Account Settings

An Amazon developer account is required for using the voice service. The registration is free. After the registration an Alexa device has to be created along with security and web settings. On this page the required steps are explained. Save the client ID and secret for later.

Raspberry Pi

This github project contains the required installation software for download:

git clone

The setup of the software is performed running the automated_install shell script. It has to be completed with the product name, client ID and secret. The script guides through the configuration and setup.

After successful installation the companion service, the AVS client and the desired wake word agent have to be launched in three separate terminals.

The AVS client requires authorization by signing in using the Amazon developer account. On request the default browser is opened and Alexa is ready to listen in after the confirmation.

Playing around

On the Raspberry Pi Alexa starts to listen more closely either on the push of a button or by hearing the wake word ‚Alexa‘. It confirms with a sound that it is listening. The next spoken words (shoud be english) are going to be analyzed. A longer break between words marks the end of the sentence.
Alexa’s answers are returned quickly! Out of the box it is possible to ask for the current weather at a specific location, to ask for a joke, to convert unities, to look up something in wikipedia, etc . Alexa can be connected to a calendar, it can calculate and it knows its „birthday“ (being the day it was first sold). That’s not all…

Surprising was my low-cost microphone in combination with Alexa. The first tests on various operating systems were devastating: I had to speak from a distance of 1 cm to be heard at all. Independent of the recording settings. I thought it is also some kind of safety precaution if I had to be close to the microphone to use speech recognition …but Alexa immediately worked from a distance of 2 m as well. It felt a bit slower, though, but still, it worked…

When I played the video recorded of my running system telling a joke it just started itself again when hearing the wake word from the video! It has already been shown that infinite loops of voice control can be set up easily: . Alexa might also react on its wake word spoken on TV as recently learnt from the Verge’s doll house article!

Alexa is extensible with custom skills for own applications. Perhaps this is the thing to try next.


Raspberry Pi 3 Gesture controlled digital Picture Frame

The first time I realised the possibilities of gesture controlled devices I was dazzled. It was the kick-started project of a smart bike assistant, the Haiku. The Haiku’s idea is to use flick gestures to switch between functions or to handle notifications, independent of possibly covered fingertips. No direct touching is required!

How simple is it to steer something without touching? Without taking off gloves or anything that hampers control as it might with common touch sensitive devices such as smartphones or tablets? Without leaving fingerprints or accidentally painting the unlock pattern on a smudgy touchscreen (take a look at a smartphone in grazing light!)?
Gesture control sounds highly convenient and probably more safe than touch control to me. Although, one could accidentally do something by moving a bit too close on a gesture control input. That is truly a side effect…

During a hacking night with fellow workers I learnt about Pimoroni’s skywriter. Attachable to either an Arduino controller or the Raspberry Pi the skywriter allows to recognise gestures such as flicks and taps as well as X/Y/Z 3D position sensing within a range of 15 cm. I ordered one to test it for another project… so why not use a skywriter to see through photographs in a directory? Displaying images in a digital picture frame with a convenient input method.

Components used

Raspberry Pi 3

Raspberry Pi 7″ Touch Screen (any other monitor for a Pi will do)

Skywriter breakout or HAT


The wiring is described in pimoroni’s github repository. The Raspberry Pi 3 pinouts are described here.

Skywriter Raspberry Pi


At pimoroni’s github repository some very good examples are found for using the skywriter either on an Arduino controller or a Raspberry. On the Raspberry the same libraries have to be installed, e.g. with the given shell command as described in the readme.

For the Raspberry I started with the Python example Each recognised gesture and a move’s coordinates are printed on the console. The print statement in the move() method obscures the results of the recognised gestures, it can be commented.

Python’s TKinter toolkit is used to display pictures in a window. The TKinter mainloop runs in a separate thread.


The usage should be natural and intuitive.

  • Flicking from left to right will display the next image in the directory.
  • Flicking from right to left will switch back to the previous image.
  • Tapping into the centre of the skywriter will close the program
  • Tapping onto the lower end will minimise the image window.
  • Guess how to maximise the image window again!

This is the whole Python script to move through (holiday) pictures using gesture control on the digital picture frame:


Switch between different images in a directory using the skywriter.

Swipe left to right: display next image in directory
Swipe north to south: display next image in directory
Swipe right to left: display previous image in directory
Swipe south to north: display previous image in directory

Tap the lower left corner to minimize the image window
Tap the upper right corner to maximize the image window

Press CTRL+C, ESC or tap the skywriter's center to exit.
# use a Tkinter label as a panel/frame with a background image
# note that Tkinter only reads gif and ppm images
# use the Python Image Library (PIL) for other image formats
# free from [url][/url]
# give Tkinter a namespace to avoid conflicts with PIL
# (they both have a class named Image)

import Tkinter as tk
from PIL import Image, ImageTk
from ttk import Frame, Button, Style
import gtk, pygtk
import time

import sys
import os
import signal
import skywriter
import threading
import random

pathToPictures = "/home/pi/Desktop/images/"

class ImageDisplay(threading.Thread):

    def __init__(self):
        self.root = None

    def callback(self):

    def run(self):
        if self.root == None:
        self.root = tk.Tk()
        self.root.title('My Photographs')


    def setImageName(self, name): = name

    def showImage(self, path):
        self.original =

        # make the root window the size of the screen
        screen_size = getScreenSize()
        self.root.geometry("%dx%d+%d+%d" % (screen_size["width"],         screen_size["height"], 0, 0))
        #self.root.attributes("-fullscreen", False)
                       #self.root.geometry({0}x{1}+0+0".format(self.root.winfo_screenwidth(), self.root.winfo_screenheight()))
        ##self.root.geometry("{0}x{1}+0+0".format(screen_size["width"],       screen_size["height"]))
        self.root.focus_set()  # >-- move focus to this widget
        self.root.bind(">Escape<", lambda e: e.widget.quit())

        self.resized = self.original.resize((screen_size["width"],        screen_size["height"]), Image.ANTIALIAS)
        self.image = ImageTk.PhotoImage(self.resized) # keep a reference, prevent GC

        # root has no image argument, so use a label as a panel
        self.panel1 = tk.Label(self.root, image = self.image)
        self.display = self.image
        self.panel1.pack(side=tk.TOP, fill=tk.BOTH, expand=tk.YES)
        print "Display image " + path

    def updateImage(self, path):
        self.original =
        # resize
        screen_size = getScreenSize()
        self.root.geometry("%dx%d+%d+%d" % (screen_size["width"], screen_size["height"], 0, 0))

        self.resized = self.original.resize((screen_size["width"], screen_size["height"]), Image.ANTIALIAS)
        self.image = ImageTk.PhotoImage(self.resized) # keep a reference, prevent GC

        # root has no image argument, so use a label as a panel
        self.display = self.image
        print "Display image " + path

    def minimize(self):
    def maximize(self):

    def stopThread(self):
        self.do_run = False  # stop thread

def getScreenSize():
    window = gtk.Window()
    screen = window.get_screen()
    print "width = " + str(screen.get_width()) + ", height = " + str(screen.get_height())
    screen_size = {}
    screen_size["width"] = screen.get_width()
    screen_size["height"] = screen.get_height()
    return screen_size

def findImages(directory):
    imageList = []
    for file in os.listdir(directory):
        if file.endswith(('.jpg','.JPG','.jpeg','.JPEG')):
    return imageList

def increaseIndex():
    global index, images
    index += 1
    # start again with index 0
    if index >= len(images):
	index = 0

def decreaseIndex():
    global index, images
    index -= 1
    # start again with index max
    if index < 0:
	index = len(images) - 1

def nextImage():
    global imageDisplay, images, index, pathToPictures
    print "image " + images[index] + ", index=" + str(index) + "(" + str(len(images)) + ")"
    imageDisplay.updateImage(pathToPictures + images[index])

def previousImage():
    global imageDisplay, images, index, pathToPictures
    print "image " + images[index] + ", index=" + str(index) + "(" + str(len(images)) + ")"
    imageDisplay.updateImage(pathToPictures + images[index])

#---detect gestures on skywriter---#
def flick(start,finish):
    print('Got a flick!', start, finish)
    if (start == "west" and finish == "east") or (start == "south" and finish == "north"):
        print "Display next image in directory"
    if (start == "east" and finish == "west") or (start == "north" and finish == "south"):
        print "Display previous image in directory"

def touch(position):
    print('Touched!', position)
    if (position == "center"):
        print "Exit image display"
    if (position == "south"):
        print "minimize image window"
    if (position == "north"):
        print "maximize image window"

# parse picture folder
images = findImages(pathToPictures)

# reset index
index = 0

# launch image window as thread
imageDisplay = ImageDisplay()

def main():
        print "Skywriter image display launched"
        print "Images found: "
        for i in images:
            print i
        global imageDisplay, pathToPictures
        imageDisplay.showImage(pathToPictures + images[index])
    except KeyboardInterrupt:
        print "Exit"
if __name__ == '__main__':


Gesture controlled Picture Frame

Raspberry Pi 3 wears a Display-O-Tron HAT

Two weeks ago I laid my hands on a Raspberry Pi 3. For sure I did not buy the 10.000.000th one. I ordered just a week before this milestone wich was a good excuse for discounts, but anyway, a Raspberry Pi is a nice and versatile thing to play with.

Since there are already a huge number of tutorials on how to set up and configure a Raspberry Pi I will spare the details. For this example of using the Display-O-Tron HAT on a Raspberry lets assume your Pi is already set up with a Linux distribution, knows Python and is connected to the internet.


An Idea

My idea is to use the Display-O-Tron HAT as display for several purposes. The Display-O-Tron HAT comes with a three line LCD display, a row of very bright LEDs on the right and 6 touch buttons. Each button can trigger the display of different information such as

  • system statistics
  • the number of unread emails, signaling incoming messages
  • process states (dead or alive?)
  • date and time
  • head lines of a news feed

The Sources

Based on the examples for the Display-O-Tron HAT on github I created a simple, straightforward, quick and dirty Python script.

On the push of a button the LCD display will be alighted for a couple of seconds in a different colour and the desired information will be retrieved and displayed. After a couple of seconds the backlight LEDs will be turned off. This is realized using threads that will finish after the desired time has passed.
The functionality that is implemented is a check for unread emails on googlemail’s IMAP servers, a simple date and time display, a quick internet connection and system status check and a status check of certain processes.
If unread emails were detected the LEDs on the (b)right side will blink three times in a row. Every minute the display will switch back to the default view: date and time.

#!/usr/bin/env python
Switch between different views using the Display-o-Tron HAT buttons

Main button: Display date and time
Right: Display number of unread emails
Left: Display system statistics
Down: Display process status
Up: TODO find something else to display

Press CTRL+C to exit.

import dothat.touch as touch
import dothat.lcd as lcd
import dothat.backlight as backlight
import signal
import sys

import psutil
import urllib2
import subprocess
import imaplib
import time
from datetime import date
import calendar
import threading


Captouch provides the @captouch.on() decorator
to make it super easy to attach handlers to each button.

The handler will receive 'channel'( corresponding to a particular
button ID ) and 'event' ( corresponding to press/release ) arguments.

'''DISPLAY utilities'''
# store time of last button touch
# to be able to reset backlight LEDs
oldTime = time.time()
turnedOff = True

def resetBacklightLEDs():
    global oldTime
    oldTime = time.time()
    global turnedOff
    turnedOff = False

def clearDOT():

def colorDOT(r, g, b):
    backlight.rgb(r, g, b)

def turnOnLEDsDOT():
    # turn on LEDs one by one in a row
    for led in range(6):
        backlight.graph_set_led_state(led, 1)

def turnOffDOT():
    colorDOT(0, 0, 0)  # backlight off
    backlight.graph_off()  # side LEDs off
    global turnedOff
    turnedOff = True

def dotClock():

    d = time.strftime("%d.%m.%Y")
    t = time.strftime("%H:%M")
    the_date =
    day = calendar.day_name[the_date.weekday()]
    print day + ", " + d + " / " + t

    lcd.set_cursor_position(3, 0)
    lcd.set_cursor_position(3, 1)
    lcd.set_cursor_position(5, 2)

def dotMails(login, password):
    nofUnreadEmails = check_googlemail(login, password)
    if nofUnreadEmails > 0:
        lcd.write("Unread Emails: " + str(nofUnreadEmails))
        showNewMessages(200, 0, 0)

        lcd.write("No new mail")

def dotSystemStats():
    lcd.set_cursor_position(0, 0)

    lcd.set_cursor_position(0, 1)
    lcd.write("CPU: " + check_CPU())

    lcd.set_cursor_position(0, 2)
    lcd.write("Memory: " + check_memory())

def check_internet():
        # ping google to check whether internet connection works
        response = urllib2.urlopen('', timeout=1)
        return "Internet: OK"
    except urllib2.URLError as err: pass
    return "Internet connection broken"

def check_CPU():
    cpu_usage = str(psutil.cpu_percent(interval=None)) + " %"
    print "CPU usage: " + cpu_usage
    return cpu_usage

def check_memory():
    mem = psutil.virtual_memory()
    # print "Memory: " + str(mem)
    memory_used = str(mem.percent) + " %"
    print "Memory used: " + memory_used
    THRESHOLD = 100 * 1024 * 1024  # 100MB
    if mem.available >= THRESHOLD:
        print("Warning, memory low")
        return "Warning, memory low"
    return memory_used

def get_pid(name):
        pids = subprocess.check_output(["pidof", name])
    except subprocess.CalledProcessError as pids:
        print "error code", pids.returncode, pids.output
        return ""
    return map(int, pids.split())

def get_single_pid(name):
    return int(check_output(["pidof", "-s", name]))

def check_process(name):
    PID = get_pid(name)
    if len(PID) == 1:
        print "PID " + name + ": " + str(PID[0])
        p = psutil.Process(PID[0])
        status = ""
        if p.status == psutil.STATUS_ZOMBIE:
            status = "Process " + name + " died"
            print status
            status = "Process " + name + " OK"
            print status
            return status
        return ""

def dotProcessStats():
    lcd.set_cursor_position(0, 0)     
    process1 = check_process('geany')
    if len(process1) > 1 :    
        lcd.write("geany is dead.")

    lcd.set_cursor_position(0, 1)
    process2 = check_process('bash')
    if len(process2) > 1 :
        lcd.write("bash is dead.")

    lcd.set_cursor_position(0, 2)
    process3 = check_process('firefox')
    if len(process3) > 1 :
        lcd.write("firefox is dead.")

def check_googlemail(login, password):
    # if new mail return # emails
    obj = imaplib.IMAP4_SSL('', '993')
    obj.login(login, password)
    nofUnreadMessages = len(, 'UnSeen')[1][0].split())
    print "Unread emails: " + str(nofUnreadMessages)
    return nofUnreadMessages

class ShowNewMessagesThread (threading.Thread):
    red, green, blue = 0, 0, 0  # static elements, it means, they belong to the class

    def run (self):
        for i in range(0, 3):

            if i == 2:
                print "stop ShowNewMessagesThread"
                self.do_run = False  # stop thread

snmt = ShowNewMessagesThread()
def showNewMessages(r, g, b):
    global snmt
    if snmt.is_alive():
    snmt = ShowNewMessagesThread() = r = g = b
    snmt.daemon = True  # enable stop of thread along script with Ctrl+C

class AlightThread (threading.Thread):
    red, green, blue = 0, 0, 0  # static elements, it means, they belong to the class

    def run (self):
        while True:
        if turnedOff == False:
            if time.time() - oldTime > 5:
                print "stop AlightThread"
                self.do_run = False  # stop thread

at = AlightThread()
def alightDisplay(r, g, b):
    global at
    if at.is_alive():
    at = AlightThread() = r = g = b
    at.daemon = True  # enable stop of thread along script with Ctrl+C

class ClockThread (threading.Thread):
    def run (self):
        print "run update clock thread " + str(self)
        self.do_run = False  # stop thread

    def stopClockThread(self):
        self.do_run = False  # stop thread

ct = ClockThread()
def updateClock():
    global ct
    if ct.is_alive():
        print "Clock thread " + str(ct) + " is alive."
    print "Launching clock thread " + str(ct)
    ct = ClockThread()
    ct.daemon = True  # enable stop of thread along script with Ctrl+C

'''DOT touch button handler'''
def handle_up(ch, evt):
    print("Up pressed: TODO find another useful display idea")
    alightDisplay(255, 0, 255)
    lcd.write("Up up and away: TODO")

def handle_down(ch, evt):
    print("Down pressed: display process states")
    alightDisplay(255, 0, 0)

def handle_left(ch, evt):
    print("Left pressed: display system statistics")
    alightDisplay(0, 100, 200)

def handle_right(ch, evt):
    print("Right pressed, check for new email")
    alightDisplay(100, 200, 255)
    dotMails('email adress', 'password')

def handle_button(ch, evt):
    print("Main button pressed: show date and time")
    alightDisplay(255, 255, 255)

def handle_cancel(ch, evt):
    print("Cancel pressed!")
    backlight.rgb(0, 0, 0)
    alightDisplay(20, 20, 20)

if __name__ == '__main__':
    while True:
        timeDiff = time.time() - oldTime
        # print "time diff: " + str(timeDiff)

        # update clock every minute
        if timeDiff > 59 or timeDiff < 0.5:
        oldTime = time.time()
        alightDisplay(255, 255, 255)
    except KeyboardInterrupt:
        print "exit"



Raspberry Pi 3 Model B

Display-O-Tron HAT