“Can you tell us a little bit about your role as data editor for The Economist?
The position is a new one, but it stems from the idea that the new wealth of data and new tools to process and visualise it means that we as journalists can tell stories in new ways. Instead of basing stories on a string of anecdotes with a single statistic dropped in, we can invert the form and make the data the story, while just using a judicious anecdote to illustrate the information. In this respect, The Economist can be said to have been practising data journalism for 170 years; we’re known for our data-driven content like the Big Mac index to compare currencies.
How do you characterise big data?
There is no concrete definition and that is probably a good thing since to define is also to limit. But it’s not woolly either. We can understand big data by its features, and the central one is this: we can do things with a huge corpus of data that we are unable to do with smaller amounts, to extract new insights and create new sources of value. This encompasses things like machine learning, in which we have self-driving cars and decent language translation. This is not because we have faster chips or cleverer algorithms, but because we have more data (and the tools to process it at a vast and affordable scale).
In what ways is big data transforming the world?
It is on track to touch all aspects of society. We will go from a world that we understand by experiencing it on an individual level, to one we comprehend on a more universal level. By that, I mean that we tend to base decisions on small amounts of data that are usually just a simulacrum of the complex reality we are trying to deal with, and tailored to our cognitive limitations to make sense of it. Tomorrow, we will use big data to surpass our faith in our individual powers and instead place trust in the data (though not blind trust).
Take medicine. Today, doctors make diagnoses based on their judgement. Sounds reasonable? In time, this will probably be considered as barbaric as bloodletting. Why not use big data? We could enshrine the experience of all doctors, and of hundreds of millions of patients over decades, to identify the best treatments to achieve the best outcomes and spot hidden adverse drug side effects. After all, the sum of all medical knowledge isn’t in the possession of any single physician. But if we aggregate vast troves of healthcare information, we may learn what works best, just as Amazon recommends books not based on the inklings of a literary critic but from correlating sales data. This will mark a revolution in how society uses information.
Concerns have been raised regarding the privacy implications of big data. What is your view of this?
Privacy is a big problem today and it will be a bigger problem tomorrow. We need to improve the legal regime to govern privacy to move beyond the system of notice-and-consent (that is, companies inform users what data they collect and how it’s used, and people give their okay). In reality it means that people tick a box agreeing to 60 pages of legal jargon with nary a glance. Instead, we need to consider the use and misuse of the data, not just the collection. We need to focus on the area of harm and not just on the inert, potential harm.
Are there any other challenges associated with big data?
While privacy is a problem, a newer issue is ‘propensity’. This refers to the idea that algorithms may be making predictions about what we are likely to do, and we may find that we’re penalised before we’ve actually committed the infraction. So big data may assign a 95% likelihood that a certain person will shoplift, or default on a loan, or fail to survive a surgical operation. We’ll need to sanctify human agency and freewill. At the same time, we’ll need a new class of professional the “algorithmists” to review big data analyses and provide society the same transparency and accountability that we have today, to ensure that big data is not a black box that obviates the public interests.“
These reflections by Kenneth Cukier of The Economist, were published in The Guardian Media Professionals Network by Adam Davidi on Friday 12 April 2013.