When you use Netflix or Hailo or other well-known apps that influence what you are doing right now wherever you are, the likelihood is that these and thousands of other businesses are likely to be running real-time from your mobile over the cloud upon infrastructure from Amazon Web Services, possibly here in Dublin.
It is actually worth arguing that Amazon, better known for its prominence as an e-commerce destination, has laid the foundations for much of the technology revolution of the last eight years.
Think about it: a decade ago, when internet entrepreneurs or software companies in general needed to expand their services, they had to buy more servers or build their own data centres. Amazon changed all of that, making on-demand cloud infrastructure a reality – all you needed was a credit card and a world of infrastructure was available to you.
Today Amazon Web Services (AWS) has thousands of customers in 190 countries, including more than 800 government agencies and 3,000 education institutions.
The company’s S3 (Simple, Storage, Service) platform holds trillions of objects and peaks at 1.5m requests per second.
Every day AWS adds enough new server capacity to support Amazon.com’s entire global infrastructure when it was a US$7bn enterprise.
In terms of big data, customers of AWS have launched more than 5.5m Hadoop clusters on Amazon EMR (Elastic MapReduce) so far.
According to Gartner’st most recent Magic Quadrant report on infrastructure as a service (IaaS) AWS is the market leader with more than five-times the compute capacity in use than the aggregate total of the other 14 providers in this space.
The company maintains a significant portion of its European data centre activities in Dublin, employing an army of technical specialists, account managers, customer support and engineers to maintain its vast infrastructure and serve European customers.
Pillars of the digital Earth
Keeping the lights on for such a considerable chunk of the world’s existing compute power would give most of us sleepless nights, but for the managing director of AWS UK and Ireland Iain Gavin the challenge is invigorating.
“When you look at Amazon there are three pillars: the main pillar is consumer retail which everyone knows in terms of e-commerce, Kindle devices; there’s the seller business where Amazon works with physical retailers to sell stock it doesn’t hold itself; and the third is the developer business, AWS.
“AWS is an enabling technology and we treat Amazon consumer retail just like any other customer we would have. Services like Cloud Drive and WhisperNet which backs up Kindle books all of that is stored on S3. Scratch under the surface and you’ll see that Amazon Prime Video and its rival Netflix are built on AWS. We treat them all equally as customers.”
Amazon’s origins extend back to 1994 when founder and CEO Jeff Bezos, a computer science graduate, worked at a Wall Street investment bank and began reading analyst papers about this emerging technology known as the internet which would one day electrify commerce. Determined to follow his self-ordained regret minimisation framework, Bezos left New York and moved to Seattle where he built one of the world’s first e-commerce mega giants, initially selling books before moving to DVDs, music and now virtually anything.
A computer science major, Bezos and his colleague Andy Jassy learned many tricks about running their own infrastructure, how to scale up and down, how to maintain complex systems at a lower running cost and ultimately guarantee reliability.
They became so good at it that they decided to make a business out of it and eight years ago AWS was born.
“We were scaling up for Christmas – always busy for Amazon.com – and then after that we were scaling down and so we built our own scalable platform to handle that,” Gavin said.
“We realized other businesses needed the same flexibility and quite rightly we decided there was an opportunity.”
Mission critical
Relying on feedback to inspire a continuous cycle of product development AWS has grown to be available in every region of the world Amazon RDS (Relational Database) is being used in mission critical deployments by tens of thousands of businesses of all sizes, processing trillions of I/O requests each month.
The product development cycle is equally breadthtaking with 280 new services and features released only last year – by the end of May 182 features and services would have been launched so far this year.
The AWS partner ecosystem has sprawled to include more than 5,000 consultants and system integrator partners and 3,000 independent software vendors.
In Ireland the company’s technology has been used since day one by Boxever, a software start-up focused on big data for airlines and which uses the platform to continually expand its infrastructure.
Smyths Toys uses the AWS infrastructure for its e-commerce infrastructure, enabling it to scale up for busy shopping periods like Christmas and scale back down afterwards.
Insurance company Laya Healthcare uses the AWS technology to interface with Oracle JD Edwards ERP and serve its 475,000 members.
Ireland’s largest telecoms operator Eircom is an AWS Advanced Consulting Partner, working with companies to design cloud systems to meet their business needs.
The challenge of keeping up with different categories of technology, such as variants of Oracle, SAP or MySQL keeps AWS on its toes, but Gavin says feedback from users and customers decides what services it adds.
“People ask do we have a 10-year roadmap, but in reality we look at what the short-term requirements are and develop against them. It’s really about trying to simplify the problem – a lot of our products have the word ‘simple’ in them.
“If you look, however, at the evolution of our services in the last five years since I started it began with computing, reasonable storage, physical RAM, but we’ve now developed into a product family that lets you do big data, scientific research and even the rendering of Hollywood movies. We worked with Time Fiction who did the special effects for Transformers and used our cluster to render them.
“All of Netflix’s encoding for devices is done across our cloud. The BBC – one of the biggest video on demand services in the world – does its rendering for iPlayer on AWS.”
While AWS dominates the on-demand computing space, it has to be aware of players like Microsoft, Google, HP, IBM and others trying to catch up.
“It’s really interesting. The attitude towards competition at Amazon is really quite refreshing. Jeff’s belief on this is quite true: ‘you don’t obsess about competitors, you obsess about the customer at the end of the day.’
“We find that the majority of the time customers come to us because ultimately they want to be able to focus on their own business. If you analyse IT operations, traditionally 70pc of the work involved stuff that wasn’t adding to the business in any form. All that other stuff doesn’t get you anything other than heartaches and headaches. Amazon learned early on it needed to free itself of that muck and AWS exists to keep things really simple.”
Dublin at the heart of the digital economy
Amazon has had a base in Ireland since 2005 and AWS came to Dublin in 2007 to focus mainly on technical operations and running its handful of data centres in the city. The company won’t reveal its actual headcount or the location of its data centres but it has been reported in recent months that 100 software development staff are moving to a new 69,000 sq ft premises in Burlington Plaza, Dublin 4.
“When you look at the places we locate it is for reasons of depth of technical expertise, the power available, the internet capacity, available structures that have opportunity for us to scale. All of this played a role in us coming to Dublin and of course, we’re close to where the customers are.
“We are constantly hiring. We have technical operations, a regional support centre and technical account managers who go and visit clients all over Europe and advise on architecture and infrastructure design. We also have customer service to handle billing enquiries. In terms of data centre operations we have hired a variety of people focused on things like building operatinos, security and of course professionals with deep networking experience and who understand things like peering and long haul data transit.”
I point out to Gavin that AWS has covered a lot of ground since being a spin-out of a company better known for e-commerce to becoming the go-to resource for computing power for the plethora of apps and online services we consume today.
“Oh hell yeah,” he concludes.
“The five years that I have been here has been a whirlwind. There’s no time compression algorithm for the experience that we have gotten, the lessons we have learned and the challenge of doing computing at this scale. But yeah, it has been phenomenal and last five years just superb.”
Amazon Web Services is a Silicon Republic Featured Employer, comprised of top tech companies that are hiring now
