Open data: of illusions and conjuring tricks

This entry is part 3 of 6 in the series Open data: magic from the inside out?

This is part 3 in a series thinking about whether the magic of open data in local government might be found from the inside out. In part 1, I considered the phrase ‘open data’ and pointed to thoughts elsewhere and part 2 suggested we needed to start from inside our organisations. In today’s third part I’m thinking about those who make magic.

Of illusions and conjuring tricks

In the last post I said that thinking about open data needed to start with how it improves what we do within our organisations because then we might understand it, recognise the value people might add to it and therefore properly champion the concept of ‘open data’.

It’s all very well saying that but if the narrative about exposing public data is difficult then an internal conversation which talks about what data could do for us is perhaps going to be thwarted before it gets off the ground anyway.

Part of the issue is that without concrete examples conversations can tend far too often towards the technicalities. The most helpful conversations aren’t comparing SOAP and RESTful APIs or talking about integration, nor will they bring up open standards or this protocol or that data format with the layman. Phil Jewitt recently wrote a couple of blog posts (1, 2) about how those beyond the project team didn’t need to know about SCRUM they just needed to know what was necessary. The most helpful conversations have at their heart somebody enthusiastically committed to sharing the secret of what’s possible.

Arthur C Clarke’s third law of prediction says that

“any sufficiently advanced technology is indistinguishable from magic”

When you go to see someone like Derren Brown you will leave wondering about how on earth they did something amazing with the everyday. And as I sit on this train with my laptop wirelessly tethered to my mobile phone connected to the internet you might as well just accept that it is magic before you try to understand how it actually works.

When it comes to seeing what can be done with open data we need members of the public to be magicians (even if that’s just by using Facebook in a slightly different way) but even before we get to that point we need someone to manipulate raw data internally and use that everyday, local data to demonstrate what could happen if we made our data public; not just talk about it.

There are a number of things which have given that treatment to data for the public but which hint at what you might be able to do internally.

Do not take me for some conjurer of cheap tricks by Jonathan_W on Flickr
Do not take me for some conjurer of cheap tricks by Jonathan_W on Flickr

Birmingham’s Civic Dashboard, for example, gives the public an overview of feedback reports but there must be richer data behind the scenes which could provide real time, location rich insight into what’s going on. We’ll have that exact same data at HCC but at the moment it languishes in spreadsheets waiting for painstaking manipulation to reveal its secrets.

Those clever folk at GOV.UK have also blogged a number of times about how they’ve started from the premise of data reuse because that simplifies everything else. To take two of those – James Stewart explains the principle of ‘building APIs, building on APIs‘ whilst John Sheridan gives an insight into the structure behind legislation.gov.uk in his post ‘Putting APIs first‘.

In Hull we are very possessive of our platforms and understandably committed to ensuring we squeeze value for money from our existing systems (Oracle didn’t/doesn’t come cheap after all). We don’t have any public APIs or publish a wide range of open data. But we do do a pretty good job with the publishing of our financial spend (everything above £1). Unfortunately there’s no narrative associated with it, we haven’t been held to any greater account because of it and nor is there suddenly a proliferation of citizen produced effort using it so does it actually add value to anybody?

The answer is yes, it is in the process of adding really serious value to Hull City Council because it helped us find our magician.

Tomorrow you’ll get two posts – one looking at why what is in motion gives me hope about the direction of open data in Hull and the other attempting to pull together a conclusion.

About Benjamin Welby

Hi, I'm Benjamin Welby. I'm a displaced northerner currently living in Croydon, I church with a group of Christians who meet in a Soho nightclub on Wednesdays and I support Bradford City. I've an academic background in History, Politics and International Development. I work for the Government Digital Service but I left my heart in local government. This blog is infrequently updated and may feature any, all or none of these things...