I spent this weekend at BL-NK at Build The News, a hackathon laid on by The Times, The Sunday Times and The Sun which my team and I won. Loads of teams entered and they were some pretty cool submissions, too much to do justice by covering here. This post is an account of our team’s journey over the last two days. If you want to see what everyone else has been up to, a good place to start is the event’s hashtag, #BuildTheNews.
Noise
There were four challenges presented to us. In brief they were: Stretch, presenting longform journalism in an engaging way via a digital medium; Crowd, communicating and generating participation in a campaign using digital resources; Tactile, reimagining the sunday paper experience for the 21st century and Noise, making sense of the riot of noise on social media and pulling news from it.
I was initially pulled towards the longform challenge, recalling this Daft Punk feature by Pitchfork. Ever since I saw that, I have wanted to have a go at building something similar. But the three journalists of the team were more interested in the noise challenge, and given the other hacker on our team has got mad algorithm skills, I was kinda out voted. I didn’t mind. Following a big ol’ google hangout and going into the Saturday, I was happy to be part of quite a promising team.
Saturday: Biting off more than we could chew
//platform.twitter.com/widgets.js
Fresh faced and eager to get going, Saturday began with the standard routine of intro talks, swag bags and free coffee. Drawing from a presentation outlining some of the problems of live-blogging events that explode through social media, we devised a prototype. It was brilliant, quite fantastic. Forget the silver bullet, this was a gold plated machine warship of an idea. Just imagine: A live blogging platform that brought together multiple content streams (video, radio, text based blogging) into a single online location that simultaneously encouraged reader participation. Broadly speaking, a clever bit of software would filter out 99% of the crap coming from twitter leaving a journalist with a few tweets and posts to skim through which she could then present to her audience and ask for help in verifying the credibility of the new information. All this was incentivised through the age old trick of awarding reputation (read: fake internet points – Jeff Atwood writes hilariously cynically on this topic) to those who contributed best. If the subheading and sarcasm haven’t made clear how this idea panned out, let me take this opportunity to say that our choice, our vision for a Brave New World ®, was a massive error of judgement. This took us all of day one to realise, but at the sae time it would be unfair to say this was time wasted. Without stumbling down this dead end and constantly trying to articulate exactly what our output would be, both to the journalist and the reader, we would never have discovered the great myriad of ways in which our ideas wouldn’t work. With each set back, the desire to find cement an idea grew and grew until the small phrase ‘tweet density’ sparked a pivot towards what would become our winning entry: Low Pass.
Low Pass
Saturday evening: With laptops packed up and own table a strewn mess of rejected ideas and deflated enthusiasm, what felt like the final conversation between Gio and Kadhim before someone declared we should throw in the towel and drown our sorrows with the hefty bar tab (provided courtesy of the event’s sponsors), spawned a simple idea for a graph. On the vertical axis was tweets per second on a given topic/hashtag, and along the horizontal was time.
Consider an event such as the Boston bombings. For such an event, one would expect the graph for #bostonbombing to be completely flat until the point at which the bomb goes of. At that point there is a huge spike as everyone starts talking about what’s going on: speculation, misinformation and opinion all but eliminate any chance that the first few commentators on the ground will be heard. The signal is completely lost in an ocean of noise. (Very impressed with that last sentence; I managed a hat tip to Nate Silver AND quote an Arcade Fire lyric). Low Pass is a tool that presents this graph to you, taking a hashtag and a point in time you know to be close to the time of the event as the only inputs. You then select the period of time you want to see tweets from (as a journalist, you most likely want to see tweets from the low activity time period just before the event spreads across social media) and Low Pass gives them to you. That’s it. It’s quite simple, but we thought we were onto something with it.
A dash to the finish
We weren’t nearly as punctual on Sunday morning as we were the previous day. This had everything to do with that fact that if you invite only students to a bar in London and have a large newspaper bankrolling the drinks, 8am starts become hard to pull off. Nevertheless, with a clear goal in sight, Gio and myself rushed to produce and MVP we could present while Kadhim, Ashley and Philip researched case studies for the tool’s use and compiled a slick presentation.
I worked on the front end. Using Twitter Bootstrap and jQuery, I didn’t take long to assemble the standard UI elements required for the app. The challenge was an interactive line graph that would scale to the data provided and offer a quick an idiot proof means of selecting a data range. This isn’t a simple jQuery job, there is no $.plotResponsiveLineChartWithSelectableDateRange(data) method, much to my dismay. This called for D3.js. I have a love hate relationship with D3. It can do incredible things, and when you do things with it, they’re usually pretty good. (One example is the One Young World flight visualisation I built over the summer: demo). But to understand how D3 works takes a lot of time, and I was rusty. That said, I managed to combine examples online to good effect and the final outcome was exactly what we needed. (At this point I’d love to tell you how Gio built the backend but I fear that a) it’s pretty tricky to explain in layman’s terms and b) I don’t understand the full algorithm myself. Suffice to say, he maxed out the Twitter API in developing a very smart piece of software.)
And the winner is…
We didn’t know our app worked for certain until 5 seconds before we began our presentation. That is no exaggeration. Thankfully we were able to give a working live demo, using the protests in Ukraine and their sudden explosion through western European media as an example of finding the people who were talking before the noise.
I’d like to mention again that there were some really cool ideas generated, prototyped and executed by all the teams taking part. Most can be found on the event’s hackdash page here.
It was a shakey 25 minutes following the presentations as I came down from the coffee and adrenaline coursing through my body. Following prizes for the individual categories, we were thrilled to learn we were the overall victors of what I hope will be the first in a series of journalism-fueled hack events.
I will update this post when a live demo of our app is available for you to try. In the mean time feel free to browse the code on GitHub If you so desire.
Jeshua Maxey 
Brilliant! Looking forward to trying out the live demo.