HP researchers can predict trending news on Twitter

12 Jun 2012

A new study from HP Labs claims to predict the popularity of news stories on Twitter with remarkable accuracy – before they’re even published.

Previous HP Labs studies have shown that mainstream media influences the majority of Twitter’s trending topics, and that the subject matter of a tweet is more important than who tweeted it and how often.

Now, HP Labs claims to have invented a technique that can predict if an article will be popular on Twitter before it has even been published, with up to 84pc accuracy.

Analysing more than 40,000 news articles published over nine days in August 2011, Sitaram Asur, Roja Bandari and Bernardo Huberman confirmed a news story’s source is a crucial factor in determining how many tweets will link to a given article, but other influential factors give a more complete perspective on what stories are likely to trend.

Using publicly available tools, the research team assigned a score to each news article, ranking its source, the category of news it belonged to, the language (whether emotional or subjective) and mentions of celebrities, famous brands or notable institutions. Standard statistical models were then used to calculate the number of tweets the article would receive.

Results were surprisingly accurate but, conclusions serve only to reiterate what many of us already know: stories that mention celebrities, come from credible sources and belong to popular categories (like technology) are more likely to generate tweets.

The factor with the least influence on distribution was the use of emotional over objective language, however, which suggests that the use of link-baiting hysterical headlines isn’t enough to see a story take off on Twitter.

“The tool we’ve created is not just useful to news organisations that want to increase their stories’ distribution on Twitter,” said Huberman. “For example, activists and politicians are increasingly using social media to influence public opinion. By testing their messages using our algorithm, they may be able to improve the visibility of their cause.”

Elaine Burke is the host of For Tech’s Sake, a co-production from Silicon Republic and The HeadStuff Podcast Network. She was previously the editor of Silicon Republic.

editorial@siliconrepublic.com