android cardboard app using camera and OpenGL ES

Some time ago I started to program Android apps for my smartphone. This year I got my hands on a cardboard. My first idea was to use the smartphones camera. Stereo vision with a single camera is a fake, but still it was tempting.

I was searching for examples of cardboard apps and stumbled across this example. For a start this was very useful. The example app shows how to create a stereo view and how to display a live camera image. The app is using Open GL ES which won’t work on all smartphones.

After having played around a little with the app I realized that if I want to change the live image only the fragment shader has to be replaced. This can happen  while clicking on the display (difficult when the smartphone is in the cardboard) or when using the magnet switch.

cardboardView.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View v) {
        switchShader();
    }
});


@Override
public void onCardboardTrigger() {
    vibrator.vibrate(50);
    switchShader();
}

Excerpt of switchShader():

private void switchShader() {
    shader_selection++;
    if (shader_selection > 31)
        shader_selection = 0;

    if (shader_selection == ORIG)
        overlayView.show3DToast("Shader: original. " +     shader_selection);
    else if (shader_selection == NEG)
        overlayView.show3DToast("Shader: negative. " +     shader_selection);
    ...
}

To display a negative image the fragment shader looks like this:

#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec2 textureCoordinate;
uniform samplerExternalOES s_texture;
void main() {
    vec4 color = texture2D( s_texture, textureCoordinate );
    float inverted = 1.0 - color.r;
    vec4 inverted_vec = vec4( vec3(inverted), 1.0);
    gl_FragColor = clamp(inverted_vec, 0.0, 1.0);
}

To invert RGB images:

#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec2 textureCoordinate;
uniform samplerExternalOES s_texture;
void main() {
	vec4 color = texture2D( s_texture, textureCoordinate );
    float invertedr = 1.0 - color.r;
    float invertedg = 1.0 - color.g;
    float invertedb = 1.0 - color.b;
    vec4 inverted_vec = vec4( vec3(invertedr, invertedg, invertedb), 1.0);
    gl_FragColor = clamp(inverted_vec, 0.0, 1.0);
}


A nice sepia effect:

#extension GL_OES_EGL_image_external : require
#ifdef GL_ES
precision mediump float;
#endif

varying vec2 textureCoordinate;
uniform samplerExternalOES s_texture;

void main()
{
    vec4 color = texture2D( s_texture, textureCoordinate );
    vec4 sepia = texture2D( s_texture, textureCoordinate );
    sepia.r = (color.r * .393) + (color.g *.769) + (color.b * .189);
    sepia.g = (color.r * .349) + (color.g *.686) + (color.b * .168);
    sepia.b = (color.r * .272) + (color.g *.534) + (color.b * .131);
    
    gl_FragColor = vec4(sepia.rgb, 1);
}

Several good examples for shaders which can be adapted I found here:

http://littlecheesecake.me/blog1/2013/08/15/pretty-geeky-codes.html
http://littlecheesecake.me/blog1/2013/04/19/more-filters.html
http://littlecheesecake.me/blog1/2013/01/31/image-processing.html

It is quite important to stop the camera when the app is paused. Otherwise other apps could not use the camera in the meantime.

    @Override
    public void onPause() {
        super.onPause();
        stopCamera();
    }

    private void stopCamera() {
        if( camera!=null ) {
            camera.stopPreview();
        }
    }

The result looks like this (computer mouse in negative mode):

androidcardboardnegativemouse

The smartphone gets pretty warm while using the app…

Advertisements

blinking bike helmet

A bike helmet could also light signal left and right on the push of a button. In the dark visibility is improved. As more and more car drivers don’t use their blinkers when changing directions it is also a nice statement.

components used
Adafruit Gemma
4 Neopixel
conductive thread, needle for sewing
small LiPo
two mini switches
hook-and-loop fastener
fabric
for testing: alligator clips

make it happen
Sew the components on fabric as shown on the pictures. Alternatively test everything using the alligator clips before sewing.

blinkinghelmet_circuitblinkinghelmet_blink

Important: the ‚data‘ thread of the neopixels should not continue below the pixels.

To place this on a helmet hook-and-loop fastener may be the right choice.

the source code
Load this program on the Adafruit Gemma. When the LiPo is powering the microcontroller push the switches to see either the left or the right LEDs blinking.

leftright.ino

The source code is based on the works of Leah Buechley, 2008.

how it works

When one of the switches is pressed the two pixels beside blink 10 times (adjustable in the source code).

blinkinghelmet_blink2

Please keep in mind that such an add on is never a replacement for proper lighting of the bike or showing your directions using your hands!

The Gemma and the Neopixels are also available in starter kits.

Of course there is no guarantee this description is free of bugs!

enlightened bag

Why not „enlighten“ a bag when opening it? This comes in useful especially in the dark.

components used
Adafruit Gemma
4 Neopixel
conductive thread, needle for sewing
small LiPo
soldering kit, small plate, cables
bag with metal zipper
for testing: alligator clips

making it happen

Sew the components in a bag as shown on the pictures. Alternatively test everything using the alligator clips before sewing.

enlightenedbag

enlightenedbag_inside

bag_plate

Realize the zipper switch to turn on the Gemma MicroController when the bag is open. A small plate with soldered cables is used to make the connection only when the zipper is open (=switched). Ensure that there is no connection on the plate for the bridged cable! Transparent nail polish helps to keep the knots from undoing. Needs several hours to dry!

the source code

Load this program on the Adafruit Gemma.

enlightenedbag.ino

how it works

When the zipper switch is used the microcontroller is powered on and turns on the LEDs one by one.

The Gemma and the Neopixels are also available in starter kits.

Of course there is no guarantee this description is free of bugs!

garden watering system

Since I have a garden regular watering is essential. But this can easily be automated with a micro controller. Here is a short description how the system is realized.

components used

Arduino Uno (or replacement)
12 V magnet valve
pressure sprayer, 5 l
flexible hoses, matching adapters
PIR sensor
IR receiver
TO 220 MOSFET
TIP 120 transistor
(red) LED
1 kOhm resistor
12 V power supply
old (plastic) box
wooden lath, screws
soldering kit, small plate, cables

making it happen

Bring the components together as briefly depicted below.

wetcat_circuit_wateringsystem_inside

wateringsystem_

The most time consuming is to find the proper hoses and adapters so that no water is leaking and the pressure lasts.

For the water output simple bend the end of a hose and prepare some holes with a hot needle.

the source code

Prepare this program for upload on the Arduino.

wetcat.ino

Adapt the code for your preferred IR remote control. It is possible to find out which codes they are with this example from an Arduino IR library.

how the system works
The red LED blinks several times when the system is powered on. The red LED is lighted when the water is running. The water runs when either the PIR sensor is activated for 5 s or as long as the matching remote control button was pressed.

The beautiful side effect of this watering system is that the neighbour cats suddenly decide to find another loo.

Of course there is no guarantee this description is free of bugs!