For October 17, when cannabis was legalized in Canada for recreational use, Emerald Health wanted a live map of tweets about Cannabis on their Discuss Cannabis website.
Using the Twitter streaming API I very quickly had something working: listen for tweets matching any of a list of filters, filter out anything not geotagged or not in Canada, and then broadcast the coordinates to all viewers via websocket.
It became more complicated when the requirement was added that a certain number of tweets should already be visible on the map. This immediately necessitated storage, and so I added a MySQL database (via a Heroku addon), and a Rest endpoint to retrieve a report of the latest tweet locations. Data reaching into the past was also then required, so the client paid for the premium Twitter API, which allows retrieval of tweets from the last 30 days and more advanced filters (including filtering to tweets geotagged with a particular country). I added a routine for this search to be preformed again a few times a day to ensure no tweets were missed by the listener (which could happen if it restarted or lost its connection for a while).
The Twitter API is frustrating in a number of ways, not least because it’s thin in a number of places and simply contradictory in others. It’s also misleading in some areas. For example, the documentation suggests that the premium API will unlock more filter types on the streaming API, but this is not the case. You’d also think that the premium API’s 30-day tweet search endpoint is similar to the regular API’s tweet search endpoint, just with more features available, but you’d be mistaken: the parameters are quite different in terms of function, naming, and expected values.
There was a disappointing amount of traffic on the original set of keywords during development – I’d have to watch the map for twenty seconds or more to see a tweet appear. This led to the impression that the map was static, whether or not it included existing tweets.
My solution was to “replay” the existing tweet data for a minute or so after page load. Time is compressed and the tweets are revealed gradually, while retaining their real cadence. I didn’t like the idea of giving the impression that these tweets are actually happening right now during this playback phase, so I added a readout of the time in the corner. Once it catches up to the present moment the word “live” is added and it ticks up as a regular clock. On legalization day it was really fun to watch the tweets light up! And I had to remind myself that there were a lot of other tweets happening across Canada – we were only seeing those with geo information, which is a pretty small percentage.
The most time-consuming part of the development, other than getting Twitter’s approval for the developer account, was the tweet reveal animation. It seemed like this should be simple – reveal a central circle and then render an expanding and fading ring from that. Sure enough, I had a hand-coded SVG with SMIL animation ready and looking great within just a few minutes, but having it play well with the Google Maps API was not so easy. The Google Maps API will happily render an SVG marker, but every time another marker is added to the map, all markers restart their animation! I tried a lot of different approaches to get around this. One attempt involved waiting for the animation to finish before switching the icon for a static version, but this was still problematic since if another animation started before an existing one finished, the existing one would still restart. The eventual solution was to use custom overlays rather than Google Maps’s built-in markers. The custom overlays allow arbitrary markup, so for each tweet a tiny SVG is added, and animated with CSS.