I saw a presentation recently on a small Melbourne start-up kaggle.com.

Basically, kaggle helps organisations solve data-related problems via crowd-sourcing and competitions.  These ‘problems’ aren’t the far-fetched conceptual ones you’d find in a stat textbook like ‘if Mary had 2X pieces of pie multiplied to the power of a rainbow…’

…although the main ingredients sound eerily similar: datasets, graphs, formulas, numbers….more numbers *eyes glazing over*…

WAIT!

Real world is calling and it would like your help.

Kaggle helps sort out the BIG, relevant issues.  For instance, is there is a genetic link to how rapidly HIV progresses to full-blown AIDS?

A biochemist in the US was working on this problem, released the question and data sets via Kaggle and started receiving answers from all over the globe.

In the end, a mathemetician from a neighbouring state won the competition and came up with an awesome model to predict this (with no biochemistry experience at all).

Why wouldn’t you get the world’s best statiticians working against the clock to solve your problem, rather than just Bruce and the geek squad you currently employ?

At the end of the day, with unlimited submissions, the most accurate model wins and you get to keep it.  The rest are kept hidden.  So all seems fair.

What is neat is that Kaggle goes beyond the basics and incorporates a collaborative/community element too – there are forums to chat about the problem/competition, which opens up the opportunity for members to team up with others working on the same problem.

Anthony, the company’s founder, told us that they get a lot of University academics submitting theories/answers.  For them, it’s far more exciting to be working on real-world issues than re-hashing old textbooks.

I couldn’t agree more.  Power to the statisticians, throw out the books, there’s plenty of real-world problems that need answers.

Image via Wikipedia

“I keep saying that the sexy job in the next ten years will be statisticians”

– Hal Varian, Google’s Chief Economist

Well Hal, as much as this project rocks, only an Economist would say that.