Tutorial: Build your own Smart TV Using RaspberryPi, NodeJS and Socket.io

Codementor Donald Derek is a conference speaker, an open-source advocate, a maker and an activist. He organizes, facilitates, and mentors startup events, and has also personally won several awards for his projects, including n!mp, a SoundCloud + Jamendo Music player with a web mobile remote app, and meeneeme, a Tamagotchi game based on your Foursquare checkins. He is currently a researcher at the MIT Media Lab – Responsive Environments.

Donald has also created the RaspberryPi-Tv, in which he shares with us a tutorial on how to build your own Google TV with it. This article was originally posted at his blog.

---

Hardware:

• The RaspberryPi (Tested on Raspberry Pi model I-B, I-B+ and II-B)
• A USB WiFi dongle or Ethernet cable (Tested on Edimax WiFi Dongle)
• SD/MicroSD card (8 GB+) (check out NOOBS)

Software Stack:

• Raspbian – a fork of Debian designed for the Raspberry Pi
• Node.js
  • Socket.io – to handle the connection between our remote and our TV via websockets
  • Express – to handle some basic http requests
  • Omxcontrol – a simple module to control the OMXPlayer which is the best video player on the rPi
• Chromium Browser
• OMXPlayer
• Youtube-dl – a script that let you download youtube videos
• Quo.js – to handle swipe gestures on the mobile web app
• HTML5, CSS3 transitions, Javascript, and Moustache as a template engine
• Youtube API

The end result

Raspberry Pi TV with its special remote controller

Outline

• Installing software and packages.
• Basic shellcode
• Server-side scripting: Node.js, Express.js, and Socket.io
• Client-side scripting: Dashboard and remote mobile-app

1. Installing software and packages

Installing Raspbian and Node.js

Follow this tutorial to install Raspbian and Node.js on your Raspberry Pi

Installing Chromium browser and Youtube-dl

Build from source or use apt-get

sudo apt-get install chromium-browser

In order to have a better display you can also install MC core fonts using

sudo apt-get install ttf-mscorefonts-installer

Install and update Youtube-dl script

sudo apt-get install youtube-dl 

sudo youtube-dl -U

Chromium on the RaspberryPi is not hardware accelerated for the moment, thus streaming video into the browser is a bad idea. Youtube-dl comes as a quick alternative, the video is downloaded and/or streamed instead in OMX-player which is hardware accelerated on the rPi!

OMX-player is installed by default Raspbian OS.

2. Basic shellcode

If you’re connecting to your rPi via SSH, you’d have to update your DISPLAY environment variables by issuing

export DISPLAY=:0.0

The graphical output of this session now points to the first display connected to the rPi. To check all your environment variables issue:

env

Test Chromium in Kiosk Mode:

chromium --kiosk http://www.google.com

Test Youtube-dl

youtube-dl youtube_video_url

I’ve added few arguments to the Youtube-dl command

• Changed the default name of the downloaded file to be: -o youtube ID [dot] file extension
• Force 720p mode: -f /22/18
• Check out the full list of supported youtube formats here

youtube-dl -o "%(id)s.%(ext)s" -f /22/18 youtube_video_url

After downloading the video, try playing it using OMX-player

omxplayer youtube_video_file

Have fun exploring the keyboard shortcuts! More shortcuts can be found here or by checking the help menu in OMX-Player.

Fancy! Let’s automate this process using Node.js

3. Server-side scripting: Node.js, Express.js, and Socket.io

The source code is intended to be simple for the sake of the workshop. Here’s the project file hierarchy:

• public
  • js
  • css
  • images
  • fonts
  • index.html
  • remote.html
• app.js
• package.json

Package.json – A JSON file needed by npm to auto-install dependencies and store other meta data about your project.

{
    "name": "RasPi.TV",
    "version": "0.0.1",
    "private": false,
    "scripts": {
        "start": "node app.js"
    },
    "dependencies": {
    "express": "3.1.1",
    "socket.io":"0.9.14",
    "omxcontrol":"*"
    }
}

In your terminal issue the following command to install the dependencies :

npm install

Notice that a folder called _ node_modules _ will be created in your project directory, if you like using git, don’t forget to create a .gitignore file and simply write in it “ node_modules ” this will ignore the folder node_modules from being added to your git project

Create the app.js file and lets start by creating our basic HTTP server with Express.js (v3.x.x)

var express = require('express')
  , app = express()  
  , server = require('http').createServer(app)
  , path = require('path')

// all environments
app.set('port', process.env.TEST_PORT || 8080);
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(express.static(path.join(__dirname, 'public')));

//Routes
app.get('/', function (req, res) {
  res.sendfile(__dirname + '/public/index.html');
});

app.get('/remote', function (req, res) {
  res.sendfile(__dirname + '/public/remote.html');
});

server.listen(app.get('port'), function(){
  console.log('Express server listening on port ' + app.get('port'));
});

This is a basic Express HTTP server configuration. To test it out, first must create static files: index.html and remote.html into the public dir. Write your favorite “Hello, World” messages into these pages, then go back to your terminal and execute

node app.js

or

npm start

Will only work if you have added the following snippet to your package.json file

...
"scripts": {
        "start": "node app.js"
    },
...

On server init you’ll get this on your stdout Express server listening on port 8080
To test your static pages run this application in the background by adding &.

node app.js &

Now this is the most primitive way to launch a Node.js app in the background, while learning Node.js you might bump into some modules that can daemonize processes for you, just like PM2

Now we have a basic Node.js Web-server that can serve static files, let’s open Chromium in –kiosk mode and test the pages.

chromium --kiosk http://localhost:8080

Integrating Socket.io

When AJAX first popped out, old skool developers felt its magic, but they’ve encountered many problems due to how different browsers handle Asynchronous Javascript and XML requests. jQuery came with the solution and provided a minimal and cross-platform abstraction for XHR. Socket.io does the same thing but for Web-sockets and can fallback into other bidirectional Web-protocols if the clients browser doesn’t support Web-sockets yet.

In order to provide realtime connectivity on every browser, Socket.io selects the most capable transport at runtime, without affecting the API. Of course, some of the technologies below will make both your network and server suffers, specially the Adobe® Flash® Socket fallback, it should be disabled or deprecated really.

1. WebSocket
2. Adobe® Flash® Socket
3. AJAX long polling
4. AJAX multipart streaming
5. Forever Iframe
6. JSONP Polling

In order to integrate Socket.io we add the following to the app.js file:

var express = require('express')
  , app = express()  
  , server = require('http').createServer(app)
  , path = require('path')
  , io = require('socket.io').listen(server)
  , spawn = require('child_process').spawn

and to minify the logs add this:

//Socket.io Config
io.set('log level', 1);

When developing with socket.io always have the mindset of a basic chat application. I’ve added a simple chat web-app done with Node.js & Socket.io on a github repo for the sake of this tutorial!

Our Socket.io Server is ready, but it doesn’t do anything yet. We implement how we process messages and events sent from the client to the server next.

Server side Socket.io integration below:

io.sockets.on('connection', function (socket) {
    socket.emit('message', { message: 'welcome to the chat' });
    socket.on('send', function (data) {
        //Emit to all
        io.sockets.emit('message', data);
    });
});

Now our server Emits the message “message” whenever a new client is connected, and waits for an event name “send” to process the data and emit it back to all connected clients

In our case, we have two types of clients: The rPi display (the TV) and the mobile Web-app (the remote)

var ss;
//Socket.io Server
io.sockets.on('connection', function (socket) {

 socket.on("screen", function(data){
   socket.type = "screen";
   //Save the screen socket
   ss = socket;
   console.log("Screen ready...");
 });

 socket.on("remote", function(data){
   socket.type = "remote";
   console.log("Remote ready...");
   if(ss != undefined){
      console.log("Synced...");
   }
 });
)};

Client-side Web-socket implementation

Add the following to remote.html:

<script src="/socket.io/socket.io.js"> </script>
<script>
  //use http://raspberryPi.local if your using Avahi Service 
      //or use your RasperryPi IP instead
      var socket = io.connect('http://raspberrypi.local:8080');
  socket.on('connect', function(data){
      socket.emit('screen');
  });
</script>

Add the following to index.html:

<script src="/socket.io/socket.io.js"> </script>
<script>
  //use http://raspberryPi.local if your using Avahi Service 
      //or use your RasperryPi IP instead
      var socket = io.connect('http://raspberrypi.local:8080');
  socket.on('connect', function(data){
    socket.emit('screen');
  });
</script>

Executing shellcode from Node.js

Node.js enables us to run system commands within the given privilleges of a child process. This includes being able to pass arguments to the command, and even pipe the results of one command to another similar to UNIX.

One way of executing shell commands from Node.js Child Process

spawn('echo',['foobar']);

But if you want to pipe the response to another call, you should provide the following callback function to the function:

//Run and pipe shell script output 
function run_shell(cmd, args, cb, end) {
    var spawn = require('child_process').spawn,
        child = spawn(cmd, args),
        me = this;
    child.stdout.on('data', function (buffer) { cb(me, buffer) });
    child.stdout.on('end', end);
}

Adding OMXControl – the OMX-player controller for Node.js

Great things can be found on npm: OMXControl module will allow you to control OMX-player over HTTP.
Require this module intp your main project file.

var omx = require('omxcontrol');

//use it with express
app.use(omx());

OMXControl module create the following routes to control the video playback:

http://localhost:8080/omx/start/:filename
http://localhost:8080/omx/pause
http://localhost:8080/omx/quit

Putting it all together

Our evolved app.js file

/**
 * Module dependencies.
 */

var express = require('express')
  , app = express()  
  , server = require('http').createServer(app)
  , path = require('path')
  , io = require('socket.io').listen(server)
  , spawn = require('child_process').spawn
  , omx = require('omxcontrol');

// all environments
app.set('port', process.env.TEST_PORT || 8080);
app.use(express.favicon());
app.use(express.logger('dev'));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(express.static(path.join(__dirname, 'public')));
app.use(omx());

//Routes
app.get('/', function (req, res) {
  res.sendfile(__dirname + '/public/index.html');
});

app.get('/remote', function (req, res) {
  res.sendfile(__dirname + '/public/remote.html');
});

//Socket.io Congfig
io.set('log level', 1);

server.listen(app.get('port'), function(){
  console.log('Express server listening on port ' + app.get('port'));
});

//Run and pipe shell script output 
function run_shell(cmd, args, cb, end) {
    var spawn = require('child_process').spawn,
        child = spawn(cmd, args),
        me = this;
    child.stdout.on('data', function (buffer) { cb(me, buffer) });
    child.stdout.on('end', end);
}

//Save the Screen Socket in this variable
var ss;
//Socket.io Server
io.sockets.on('connection', function (socket) {

 socket.on("screen", function(data){
   socket.type = "screen";
   ss = socket;
   console.log("Screen ready...");
 });
 socket.on("remote", function(data){
   socket.type = "remote";
   console.log("Remote ready...");
 });

 socket.on("controll", function(data){
    console.log(data);
   if(socket.type === "remote"){

     if(data.action === "tap"){
         if(ss != undefined){
            ss.emit("controlling", {action:"enter"}); 
            }
     }
     else if(data.action === "swipeLeft"){
      if(ss != undefined){
          ss.emit("controlling", {action:"goLeft"}); 
          }
     }
     else if(data.action === "swipeRight"){
       if(ss != undefined){
           ss.emit("controlling", {action:"goRight"}); 
           }
     }
   }
 });

 socket.on("video", function(data){

    if( data.action === "play"){
    var id = data.video_id,
         url = "http://www.youtube.com/watch?v="+id;

    var runShell = new run_shell('youtube-dl',['-o','%(id)s.%(ext)s','-f','/18/22',url],
        function (me, buffer) { 
            me.stdout += buffer.toString();
            socket.emit("loading",{output: me.stdout});
            console.log(me.stdout)
         },
        function () { 
            //child = spawn('omxplayer',[id+'.mp4']);
            omx.start(id+'.mp4');
        });
    }    

 });
});

4. Client-side scripting: Dashboard and Remote mobile-app

Raspberry Pi TV Screen Front-end

Describing in details how I built the front-end is out-of-scope in this tutorial, however I would like to point out few tips that I discovered while doing this project.

When designing for the 10 foot screen there’s some design tips to consider, Google compiled a nice set of these tricks on their Developers Site

raspberry pi remote

Instead of creating a classical remote full of buttons, I decided to give Quo.js a try, it’s a fantastic cross-platform swipe gestures js library!

$$(".r-container").swipeLeft(function(){
socket.emit('control',{action:"swipeLeft"}); 
});

Here’s an example of how I send the message “Control” back to the server with the data action: swipeLeft

The server will handle that message by sending it to the screen, the screen client will handle this message by moving the selected square to the next app (Watch, Listen, Play)

I’ve also compiled a list of meta trick that will let your iPhone mobile web app look like a native one with a nice icon and a splash screen.
Just add the following to your HTML <head></head> blocks

<link rel="apple-touch-icon" href="images/custom_icon.png"/>
<link rel="apple-touch-startup-image" href="images/startup.png">
<meta name="viewport" content="width=device-width initial-scale=1, maximum-scale=1, user-scalable=no" />
<meta name="apple-mobile-web-app-title" content="Remote">
<meta name="apple-mobile-web-app-capable" content="yes">

---

Special Thanks

This workshop was given at Lamba Labs Beirut First Hackerspace after a series of lightning talks check out the presentation here If you’d like to bypass the tutorial and jump into the fun stuff, you can always fork the code on Github

This workshop is also powered by GDG Beirut and the idea was baked during a lightning talk that I gave at the Google IO Extend Beirut 2013