About a month ago, I finally had the time to publish a small side-project of mine: https://webchan.nl. Webchan.nl is a simple service that everyone can use to augment a website with push notifications.

You simply include a small JavaScript file in your web pages that connects to the Webchan.nl server, reserves a channel and receives messages from that channel. From that point on your server-side components can push to that channel any information you want your users to see in realtime.

The software is far from being feature-complete in my sense of the word but it does what I needed when I started to code on the project.

It all started with an idea I had for my LinuxTag talk this year. I wanted to pitch the web platform to the audience as an alternative to native platforms like iOS or Android (which I try hard since about 1,5 years now) and I wanted to show them something that was really cool and caused that “wow” moment; and it should be practical in some way. I thought it would be really cool to be able to control the presentation itself (which is composed of just HTML, CSS and JavaScript, of course) using the browser running on my smartphone.

So I immediately bootstrapped a project which I called web remote control, or webrc for short. It should consist of a server component mediating between the laptop and the smartphone and two client components: the presentation itself running in a browser on the laptop and the controller running in a browser on my phone.

At the time I knew about several push mechanisms browser vendors were working on. I didn’t want to use hacks such as long-polling using XHR so I actually had the choice between these two: Web Sockets and Server-sent events (or EventSource for short). From some experiments with web sockets before I knew that these were kind of a pain to setup on the server side (at least on Apache Httpd) and for my use case probably oversized. So I opted for EventSource which is a really neat mechanism to use on the client. It’s just these 4 lines of JavaScript code:

var evsrc = new EventSource("/endpoint");
evsrc.addEventListener("myevent", function(ev) {
    ...
});

Can’t be easier, right? Have a look at this page for more info.

So the client side is really easy to setup. But what about the server side?

To make things as easy as possible I opted for Python and the Flask web microframework for the server component. With Flask you’re up and running in a few minutes so it’s perfect for prototyping and stuff. Since EventSource is really just a neat API around the well-known long-polling mechanism and doesn’t require a specific protocol (except for the correct MIME type and the “two newlines separate events from each other” mechanism) you just have to implement some mechanism in the application that doesn’t close HTTP connections.

Now if that’s not a use case for our friend, the “event driven server”, made popular to the world by Node.js! The pain when building something with Python is that you have so many choices. There are fully-fledged event driven servers such as Tornado, there are low-level libraries like Twisted and there are pure Python libraries such as Greenlet and Gevent. After looking at some code and playing around with all the alternatives I choose Gevent as a base for my web app.

After having experimented with how to integrate Gevent into my Flask app I finally came to this solution:

@APP.route('/channel/', methods=["POST","GET"])
def api_channel(channel_id):
    try:
        iterator = CHANSERV.open_channel(channel_id)
    except KeyError:
        abort(404)
    def json_iterator():
        for event in iterator():
            yield unicode("event: {0}\ndata: {1}\n\n").format(event.name, event.data)
    return Response(json_iterator(), headers=[('cache-control',
        'no-cache'), ('connection', 'keep-alive'),
        ('Access-Control-Allow-Origin', get_origin_header(request))],
        content_type='text/event-stream')

The magic happens in the return statement which returns a Response object carrying an iterator. This iterator in turn iterates over the iterator returned by CHANSERV.open_channel() which - simply spoken - is just a blocking queue provided by Gevent. And since we have a blocking queue the HTTP connection is actually never closed (until we hit the timeout set by the web server or the TCP stack). Now I was ready to build my slideshow remote control around this construct.

  • I included a little JavaScript into my slideshow which creates an EventSource object listening for events from the server

  • Then I built a simple controller app consisting of two buttons “left” and “right” for changing slides. When one of the buttons is pushed it sends an HTTP POST request to the server (which is actually the endpoint to my channel). The server then decodes the event data (“left” or “right”) and pushes an event to the channel so that the client can react and change to the next or previous slide.

You can see a demonstration of this mechanism here: http://linuxtag12.makk.es. Each channel is identified by a random string. For more comfort I built-in a QR code that you can scan with your phone which leads you to the controller of the page you’re currently viewing on your desktop browser.

The logical next step was to generalize this mechanism, and this is when Webchan.nl was born. I abstracted some of the code, separated the channel management from the web server part and implemented a small JavaScript library for channel acquiring, event handling and QR code generation, added some code to make this all work cross-site and ready is the general-purpose realtime channel comprised of only web standards technology.

Since this is just a side project there are buggers still: Cross-site EventSource only work in Firefox at the moment. There is a polyfill for EventSource that makes cross-site EventSource objects work in every major browser but I was too lazy to integrate it into my client library, yet. The good thing is that you can include the polyfill into your page and then use the Webchan.nl client library without changes because the polyfill overwrites the whole EventSource object in the global namespace (haven’t tried it, though).

I’m willing to make the source code of all the components of Webchan.nl available as soon as there’s interest. So just drop me a line (or a comment here) and I’ll start to put the code somewhere.