Treading the cowpaths of the information superhighway

Treading the cowpaths of the information superhighway.

Chris&an Heilmann, Ignite, London, 2nd of March 2010

Today I will talk about a way of trying to get the web to be more inclusive to people who really should use it, but cannot be bothered. I will show you how you can easily use information created by those who know to create information and put it on the web without spending any money.

To the interwebs!

Right now we are trying to get everybody to use the internet. We are very excited about it and everybody else should be, too.

The revolution of social media changed our lives completely and it is tough to understand why not everybody is as excited about it as we are.

Hamstering.

However, when companies start to build web systems you will find something like this. Instead of embracing the simplicity of the web and give people a chance to digest information piece by piece we overload them with interfaces that look exactly like the ones they use on the desktop.

In essence, it is tough to explain our passion and get people to embrace the web the way we do - enough to turn you gaga.

The thing is though that companies do have already a working and very much used IT infrastructure - one that people got trained in using and are happy to use. That these pieces of software are Desktop-driven shouldn’t bother us.

What we should concentrate on is the data created in these systems and if they are useful for web consumption. If they are, we should try to find a way to convert them and get them out there. Most of the time a lot of valuable information goes into spreadsheets but only a fraction of it makes it into the CMS.

We then use our own toys and building blocks to get this information out as a web-friendly interface.

http://winterolympicsmedals.com

This is winterolympicsmedals.com - a search interface for winter olympics medals from 1924 up to now, searchable by country, medal type, sport, discipline and allowing you to compare the success of different countries side-by-side.

The data that drives the whole site and builds the navigation and the interface is all stored in an Excel sheet the Guardian released on their data blog. Every week the Guardian releases the information behind their articles as spreadsheets - which is an amazing service.

I took the Guardian spreadsheet, created a copy in Google docs and published it to the web as a CSV file. CSV is a terribly easy to parse data format. Now what I needed to do was accessing this data by the different cells and columns.

select * from csv where url="http://spreadsheets.google.com/pub?key=tpWDkIZMZleQaREf493v1Jw&output=csv" and columns="Year,City,Sport,Discipline,Country,Event,Gender,Type" where Year="1924"

And this is where Yahoo’s YQL is an amazingly useful tool. Using the CSV as a data source and giving it a list of columns you can easily filter by each of the columns and create complex queries. The query gets sent to a web API and you get the data back either as XML or a JSON.

Building the interface was as easy - using the YUI grids builder I put together a HTML skeleton and simply added my data where I wanted it to appear. The CSS is even hosted by Yahoo on a server network for me.

This made me think that it would be useful to have this as a generic solution which is why I put together csvtowebservice.php which allows you to create a filtering form and result datatables from a CSV on the web.

http://data.gov.uk/The Guardian is not the only resource that releases interesting spreadsheets with information that can be converted with a script like that. Data.gov.uk is still quite new and offers thousands of government data sets as excel sheets.

http://uk-house-prices.comFor the release of data.gov.uk I was asked to build a demo and this is what I did: http://uk-house-prices.com - an interface to compare house prices throughout England from 1996 up to now - again driven by a single CSV.

*bleep*

I think it is time we stop being fanboys of our own environment and instead see how we can communicate and link with the other - already existing - networks to take their data and bring it onto the web.

And maybe, just maybe - if we manage to build very compelling interfaces really quickly we give corporate IT the hint that upgrading is a great plan.

If you want to know more, there is a video of me talking for a whole 48 minutes about using these systems to build uk-house-prices.com and other solutions available in German and English on the Yahoo User Interface blog.

  Chris&an Heilmann  h?p://wait‐&ll‐i.com   h?p://developer‐evangelism.com  h?p://twi?er.com/codepo8   

Cheers

That’s it! Now you go and build something!