Raspberry Pi 3 Gesture controlled digital Picture Frame

The first time I realised the possibilities of gesture controlled devices I was dazzled. It was the kick-started project of a smart bike assistant, the Haiku. The Haiku’s idea is to use flick gestures to switch between functions or to handle notifications, independent of possibly covered fingertips. No direct touching is required!

How simple is it to steer something without touching? Without taking off gloves or anything that hampers control as it might with common touch sensitive devices such as smartphones or tablets? Without leaving fingerprints or accidentally painting the unlock pattern on a smudgy touchscreen (take a look at a smartphone in grazing light!)?
Gesture control sounds highly convenient and probably more safe than touch control to me. Although, one could accidentally do something by moving a bit too close on a gesture control input. That is truly a side effect…

During a hacking night with fellow workers I learnt about Pimoroni’s skywriter. Attachable to either an Arduino controller or the Raspberry Pi the skywriter allows to recognise gestures such as flicks and taps as well as X/Y/Z 3D position sensing within a range of 15 cm. I ordered one to test it for another project… so why not use a skywriter to see through photographs in a directory? Displaying images in a digital picture frame with a convenient input method.

Components used

Raspberry Pi 3

Raspberry Pi 7″ Touch Screen (any other monitor for a Pi will do)

Skywriter breakout or HAT

Wiring

The wiring is described in pimoroni’s github repository. The Raspberry Pi 3 pinouts are described here.

Skywriter Raspberry Pi
GND GND
TRFR GPIO 27
RESET GPIO 17
SCL GPIO 3 / SCL
SDA GPIO 2 / SDA
VCC 3 V

Software

At pimoroni’s github repository some very good examples are found for using the skywriter either on an Arduino controller or a Raspberry. On the Raspberry the same libraries have to be installed, e.g. with the given shell command as described in the readme.

For the Raspberry I started with the Python example touch.py. Each recognised gesture and a move’s coordinates are printed on the console. The print statement in the move() method obscures the results of the recognised gestures, it can be commented.

Python’s TKinter toolkit is used to display pictures in a window. The TKinter mainloop runs in a separate thread.

Usage

The usage should be natural and intuitive.

  • Flicking from left to right will display the next image in the directory.
  • Flicking from right to left will switch back to the previous image.
  • Tapping into the centre of the skywriter will close the program
  • Tapping onto the lower end will minimise the image window.
  • Guess how to maximise the image window again!

This is the whole Python script to move through (holiday) pictures using gesture control on the digital picture frame:

#!/usr/bin/python

print('''
Switch between different images in a directory using the skywriter.

Swipe left to right: display next image in directory
Swipe north to south: display next image in directory
Swipe right to left: display previous image in directory
Swipe south to north: display previous image in directory

Tap the lower left corner to minimize the image window
Tap the upper right corner to maximize the image window

Press CTRL+C, ESC or tap the skywriter's center to exit.
''')
# use a Tkinter label as a panel/frame with a background image
# note that Tkinter only reads gif and ppm images
# use the Python Image Library (PIL) for other image formats
# free from [url]http://www.pythonware.com/products/pil/index.htm[/url]
# give Tkinter a namespace to avoid conflicts with PIL
# (they both have a class named Image)

import Tkinter as tk
from PIL import Image, ImageTk
from ttk import Frame, Button, Style
import gtk, pygtk
import time

import sys
import os
import signal
import skywriter
import threading
import random

pathToPictures = "/home/pi/Desktop/images/"

class ImageDisplay(threading.Thread):

    def __init__(self):
        threading.Thread.__init__(self)
        self.root = None
        self.start()

    def callback(self):
        self.root.quit()

    def run(self):
        if self.root == None:
        self.root = tk.Tk()
        self.root.title('My Photographs')

    self.root.mainloop()

    def setImageName(self, name):
        self.name = name

    def showImage(self, path):
        self.original = Image.open(path)

        # make the root window the size of the screen
        screen_size = getScreenSize()
        self.root.geometry("%dx%d+%d+%d" % (screen_size["width"],         screen_size["height"], 0, 0))
        #self.root.attributes("-fullscreen", False)
        #self.root.overrideredirect(True)
                       #self.root.geometry({0}x{1}+0+0".format(self.root.winfo_screenwidth(), self.root.winfo_screenheight()))
        ##self.root.geometry("{0}x{1}+0+0".format(screen_size["width"],       screen_size["height"]))
        self.root.focus_set()  # >-- move focus to this widget
        self.root.bind(">Escape<", lambda e: e.widget.quit())

        self.resized = self.original.resize((screen_size["width"],        screen_size["height"]), Image.ANTIALIAS)
        self.image = ImageTk.PhotoImage(self.resized) # keep a reference, prevent GC

        # root has no image argument, so use a label as a panel
        self.panel1 = tk.Label(self.root, image = self.image)
        self.display = self.image
        self.panel1.pack(side=tk.TOP, fill=tk.BOTH, expand=tk.YES)
        print "Display image " + path

    def updateImage(self, path):
        self.original = Image.open(path)
        # resize
        screen_size = getScreenSize()
        self.root.geometry("%dx%d+%d+%d" % (screen_size["width"], screen_size["height"], 0, 0))

        self.resized = self.original.resize((screen_size["width"], screen_size["height"]), Image.ANTIALIAS)
        self.image = ImageTk.PhotoImage(self.resized) # keep a reference, prevent GC

        # root has no image argument, so use a label as a panel
        self.panel1.configure(image=self.image)
        self.display = self.image
        print "Display image " + path

    def minimize(self):
        self.root.iconify()
    def maximize(self):
        self.root.deiconify()

    def stopThread(self):
        self.root.quit()
        self.do_run = False  # stop thread

#---Utilities---#
def getScreenSize():
    window = gtk.Window()
    screen = window.get_screen()
    print "width = " + str(screen.get_width()) + ", height = " + str(screen.get_height())
    screen_size = {}
    screen_size["width"] = screen.get_width()
    screen_size["height"] = screen.get_height()
    return screen_size

def findImages(directory):
    imageList = []
    for file in os.listdir(directory):
        if file.endswith(('.jpg','.JPG','.jpeg','.JPEG')):
            print(file)
            imageList.append(file)
    return imageList

def increaseIndex():
    global index, images
    index += 1
    # start again with index 0
    if index >= len(images):
	index = 0

def decreaseIndex():
    global index, images
    index -= 1
    # start again with index max
    if index < 0:
	index = len(images) - 1

def nextImage():
    global imageDisplay, images, index, pathToPictures
    increaseIndex()
    print "image " + images[index] + ", index=" + str(index) + "(" + str(len(images)) + ")"
    imageDisplay.updateImage(pathToPictures + images[index])

def previousImage():
    global imageDisplay, images, index, pathToPictures
    decreaseIndex()
    print "image " + images[index] + ", index=" + str(index) + "(" + str(len(images)) + ")"
    imageDisplay.updateImage(pathToPictures + images[index])

#---detect gestures on skywriter---#
@skywriter.flick()
def flick(start,finish):
    print('Got a flick!', start, finish)
    if (start == "west" and finish == "east") or (start == "south" and finish == "north"):
        print "Display next image in directory"
        nextImage()
    if (start == "east" and finish == "west") or (start == "north" and finish == "south"):
        print "Display previous image in directory"
        previousImage()

@skywriter.touch()
def touch(position):
    print('Touched!', position)
    if (position == "center"):
        print "Exit image display"
        imageDisplay.stopThread()
        sys.exit()
    if (position == "south"):
        print "minimize image window"
        imageDisplay.minimize()
    if (position == "north"):
        print "maximize image window"
        imageDisplay.maximize()

# parse picture folder
images = findImages(pathToPictures)

# reset index
index = 0

# launch image window as thread
imageDisplay = ImageDisplay()

def main():
    try:
        print "Skywriter image display launched"
        getScreenSize()
        print "Images found: "
        for i in images:
            print i
        global imageDisplay, pathToPictures
        imageDisplay.showImage(pathToPictures + images[index])
    except KeyboardInterrupt:
        print "Exit"
        imageDisplay.stopThread()
        sys.exit()
    pass
if __name__ == '__main__':
    main()

Result

skywriter_photo
Gesture controlled Picture Frame
Advertisements