Big data will embrace small businesses

21 Jun 2012

Image: © your123/

Large organisations from airlines to publishing giants are snapping up big data technologies. But when will big data get real for the smaller business? Very soon, says John Kennedy.

The words ‘big data’ are probably the best way to describe this new technological discipline set to transform business as we know it. Ever since businesses have come to depend on computers, a massive data trail has been built up over the years.

Until lately no one knew really what to do with the data warehouses and databases stuffed with hordes of data – most of it private and personal – to entire digital trails left by visits to websites, bills, transactions and much more.

Throw in social media like Twitter and Facebook and there’s a lot of data that should be gold to businesses but in reality is hard to make sense of.

But the quest to make sense of this information, to make split-second real-time decisions and fix the bottom line, boost profitability and more, is leading to the next evolution in business technology.

Big data careers

Big data gets real – analysing the next big business trend

$3.2bn: Value of big data market worldwide in 2010

$7bn: Projected value of big data in 2015

40pc: Growth rate of big data technologies between now and 2015

Big data technology growth to 2015:

Software – 34.2pc

Storage – 61.4pc

Servers – 27.3pc

Analytics and business intelligence have been around for some time and quintessentially provide firms with the equivalent of a car’s dashboard to get a real sense of how the business is doing.

But the quest to know more about what the customer is thinking, why products don’t sell very well, why a store or airline has experienced a huge drop in sales, is giving rise to new career opportunities in jobs such as data scientists and analytic experts, who can crunch this data.

Firms like Google and Facebook can’t get enough of these people who can sift through a variety of data sets and come up with real answers to business problems.

But for ordinary firms in ordinary, traditional businesses there is the rise in new technologies and tools to understand what’s happening.

In recent weeks, Dublin-based technology company PolarLake was acquired by global business publishing giant Bloomberg, which is launching a new enterprise data management (EDM) service for its customers. Prior to the acquisition PolarLake raised €850,000 from Enterprise Ireland and had previously been referred to by Oracle CEO Larry Ellison as one of the world’s best emerging technology companies.

The new EDM service from Bloomberg will help companies acquire, manage and distribute data across their organisations. Bloomberg’s new EDM unit, with the addition of PolarLake, will allow firms to increase their efficiency by better managing high volumes of data from multiple sources, either third parties or proprietary sources.

“Our customers are looking to find new ways to reduce the cost of managing data while adhering to increasing regulatory requirements for transparency,” said Dan Doctoroff, Bloomberg president and CEO at the time of the PolarLake acquisition.

Customer sentiment analysis

The world of big data when it meets ordinary business and social media analysis was brought alive to me last week when I spoke to Richard Owen, CEO of SatMetrix.

SatMetrix, which developed an online sentiment analysis technology called SparkScore, last week launched a free online tool called Net Promoter Score, which allows firms to analyse customer sentiment from sites like Facebook and Twitter and align this with brand performance.

The company’s technology is used by Fortune 500 companies to understand why brands aren’t performing as well as they should. “It’s no longer just a matter of surveying customers to see what they think. There is a new generation of technologies that are capable of analysing data sources and we can get this in real-time to company employees so they can act on a situation,” said Owen.

He demonstrated by analysing Twitter comments on how erratic pilot behaviour on a low-fares airline that became headline news had a direct correlation with a sudden decrease in trust and footfall. He was also able to demonstrate how another airline’s meal choices and staff attitudes were impacting on customer loyalty.

“It is important to ensure that this data can be made available to everyone in an organisation in a way that they can understand it. If all employees can see the score as well as the reasons behind the poor performance it leads to action taking and behavioural modification.

“If you have an army of employees capable of doing sophisticated analysis at an operational level that applies to them, then you get quick, actionable answers to the big questions.”

In one novel example of how the system can function, a single tweet from reality TV star Kim Kardashian about her luggage being opened knocked British Airways’ social media reputation to the bottom of the rankings.

Making big data real and applicable to ordinary firms was brought to life in Ireland last week when telecoms giant BT launched an Asssure Analytics security data analysis service which helps organisations collect, arrange and evaluate big data sets, presenting them in visually insightful ways which can improve decision making.

It enables businesses to make informed, split-second decisions and develop effective long-term policies to govern their use of resources and response to potential risks and security threats across their infrastructures and operations.

The service, which instantly fuses together structured and unstructured data from any number of sources – such as emails, reporting systems, databases and internet news feeds – brings the information to life for businesses through insightful imagery and mapping, which highlights potential patterns, threats, interdependencies and outcomes across their organisation.

Fighting thievery


BT is already using Assure Analytics in its battle to protect the UK telecoms network from copper theft. The service is helping the company analyse crime statistics, fault reporting and geographical information-highlighting patterns and theft hotspots to inform its prevention and response policies. It also has the potential to be connected into the company’s ‘Rabit’ tool, which gives BT and the police an early warning of cable thefts in progress.

“Organisations are facing an explosion in data generation and use, and they need to detect patterns or find answers faster than ever before to drive business and detect risks,” explained Richard Villars, vice-president for data centre and cloud research at IDC.

“Solutions like BT Assure Analytics provide organisations with access to tools and skills that were once too complex and inaccessible for use outside BT’s own facilities.

“BT’s new technology transfer model makes it easier for customers to focus on improving security and business value, while BT focuses on technology adoption and operational efficiency.”

Jeff Schmidt, global head of business continuity, security and governance, BT Global Services, said: “BT Assure Analytics is a real game changer which will allow businesses to visualise their complex ‘big data’ as well as the diverse security threat spectrum of data and voice events.

“It gives organisations fresh and unique clarity and allows them to rethink the risk they face in defending against new and sophisticated cyber threats.”

Education for emerging areas in tech

A report last year from McKinsey & Company showed the demand for deep analytical skills in the US could outstrip supply by 50pc to 60pc in 2018. EMC predicts that Ireland will follow a similar pattern if steps are not taken to drive more students to computer and engineering courses adapted to emerging areas such as big data analytics.

“If this pattern is emerging in the US, it will not be long until it is repeated in Ireland and we need to be ready,” said Jason Ward, EMC’s country manager in Ireland. “If we are not, Ireland will be overtaken by other economies and we will miss out on the big-data opportunity and cost the country jobs.”

EMC makes three recommendations towards fostering the development of these skills in Ireland. The first is to tailor educational programmes to draw together multi-disciplinary strands across mathematics, computing, science and sociology.

Secondly, EMC believes that applied science should be added to the Leaving Certificate curriculum, with a specific industry focus and cross references with other subjects.

Finally, EMC believes that a strong Government-led effort is required to generate greater public awareness of IT-related careers, and to demystify science, technology, engineering and maths (STEM) programmes.

“In Ireland, as EMC supports companies through their IT and business transformation journey, we will need more cloud architects and data scientists. Big data analytics can become a new jobs-rich source of activity for our economy, enabling huge data sets to be compared side by side at lightning-fast speeds to generate business insights,” said Ward.

John Kennedy is a journalist who served as editor of Silicon Republic for 17 years