Big Data R&D for the newsroom

US newspaper circulation To grasp the scale of the challenge that faces the newspaper industry take a look at this chart. As the red line shows, daily circulation figures in the United States have declined to levels not seen since World War 2. For an even grimmer narrative visit Paper Cuts where Erica Smith chronicles layoffs at newspapers as they occur. However, the future for the newspaper will be brighter than these things might appear to suggest.

There are two reasons to be optimistic. First, the emerging information environment suits the high-end newspaper, although that might be difficult to believe today. Second, big data in the newsroom can help deliver precisely what the reader needs. At The Irish Times we are working to develop big data capabilities in our newsroom that do exactly this.

The information environment: White Noise 

Three in every five of our users believe that they receive too much information every day in the form of news, e-mails, newsfeeds, and social media updates, according to a survey I recently ran. (Sample: just under 1,800 respondents from among users of Rewarding Times, The Irish Times’ upmarket group discount service, 94% of whom are regular Irish Times readers online or in print).

Irish Times readers tend to be adept information consumers, well-educated and well-read. So when a large proportion of them say they are receiving too much information that suggests the problem affects everyone. The crux is that information, which was once scarce, expensive, and manageable, is now the opposite: plentiful, cheap, and overwhelming. This change was apparent before I joined The Irish Times, and it has become increasingly visible since. Reuters Oxford Institute’s Digital News Report 2013 reports that as people “acquire more devices they also consume more news … but also access news more often throughout the day”. And the result is a white noise effect.

The idea of ‘information overload’  gained currency among researchers from the 1960s onward. An  example worth reading is Streufert, Suedfeld, and Driver’s studyConceptual structure, information search, and information utilizationpublished in 1965. It tracks the decisionmaking ability of 185 students as they receive increasing quantities of information, and proved that humans continue to crave information even when they have more of it than they can handle.

There is an analogy with food, put simply in Clay A. Johnson’s Information diet (2011). As dietary calories became cheap and plentiful in the West people were forced to be selective in their diet in order to maintain their health. Information, like energy in food form, is something that humans have an insatiable appetite for.

Ryan's loose theory of too much information

As the ratio between quantity of available information and our capacity to absorb it changes, we must be more selective.

The newsroom: Big Data R&D 

Although information of varying quality and provenance has become cheap, trustworthy insight is increasingly precious. The white noise creates an acute need for thoughtful analysis and verification. And the market reflects this. Publications that provide this, such as The EconomistThe New York TimesThe Financial Times, and so on, are prospering.

The lesson is that the value of insight has survived the disruption, and the reputation for trustworthiness that reputable newsrooms can attach to that insight is more valuable than ever. This is good news for The Irish Times. It is governed by a trust whose articles mandate that it exists ‘to enable readers of The Irish Times to reach informed and independent judgements and to contribute more effectively to the life of the community’. In other words, its sole reason for operating is to turn people into thinking citizens rather than to make a profit. (Profits generated through things like Rewarding Times are reinvested back into the newspaper’s mission).

Earlier this year at the annual conference of the World Association of Newspapers and News Publishers in Bangkok we told our colleagues around the world that we had begun to develop a multi-million, multi-annual, big data R&D programme focussed on radical innovation in the newsroom. Over the coming months I will post updates on this blog. You can also see video diaries of our innovation initiatives with startups (Irish Times Digital Challenge and Irish Times FUSION) at

We have been working with Enterprise Ireland, Science Foundation Ireland, and with the leading computer science research centres in the country to build up a world class consortium for big data R&D. And crucially, where many organisations have made customer data the primary focus of their big data work, our focus at the outset is to find and deliver the critical insights that separate a story worth reading from just another distracting tid bit.

The Irish Times is re-discovering the spirit of invention that made it one of the first half dozen newspapers in the world to launch an online edition in 1994. We are pioneering new ways to deliver insight and cuts through the white noise. If you would like to help please get in touch.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s