Spotify has signed a major infrastructure deal that will move Spotify's infrastructure from different data centres into Google's cloud.
Swedish music streaming giant Spotify has signed a major infrastructure deal with Google that will see it move its infrastructure from a hodge-podge of data centres dotted around Europe and the US entirely to Google’s cloud.
“We’re working with the Google Cloud Platform team to provide platform infrastructure for Spotify, everywhere,” said Nicholas Harteau, VP of engineering and infrastructure at Spotify.
Harteau said that, historically, Spotify leased or bought data centre space, server hardware and networking gear to give users music instantly.
“But in a business growing quickly in users, markets and features, keeping pace with scaling demands requires ever-increasing amounts of focus and effort. Like good, lazy engineers, we occasionally asked ourselves: do we really need to do all this stuff?
‘Google, in our experience, has an edge here, but it’s a competitive space and we expect the big players to be battling it out for the foreseeable future’
– NICHOLAS HARTEAU, SPOTIFY
“For a long time the answer was ‘yes’. Operating our own data centres may be a pain, but the core cloud services were not at a level of quality, performance and cost that would make cloud a significantly better option for Spotify in the long run. As they say: better the devil you know.
“Recently, that balance has shifted. The storage, compute and network services available from cloud providers are as high quality, high performance and low cost as what the traditional approach provides. This makes the move to the cloud a no-brainer for us. Google, in our experience, has an edge here, but it’s a competitive space and we expect the big players to be battling it out for the foreseeable future.”
Harteau said that Google currently has the edge because of the sophistication of its data services, from batch processing with Dataproc to event delivery with Pub/Sub and the abilities of BigQuery.
“We have a large and complex backend, so this is a large and complex project that will take us some time to complete,” Harteau said.
Irish engineers have set the benchmark for Google’s cloud ambitions
Ireland could become a beneficiary of this alliance between Google and Spotify as Google has built two massive data centres in Dublin. The most recent, a €150m project, was announced in August 2015 and resulted in 400 construction jobs.
At a recent Google open day at its European headquarters in Dublin, the head of Google’s engineering team in Dublin, Terrence McGoff, said that 400 engineers in Ireland have set a global benchmark for data centre design, software-defined networks and site reliability engineering.
“In Dublin, we have a long history of doing disruptive things on an engineering front, building the core technologies that support search, ads, infrastructure, storage layers and all the technology that underpins Gmail [and] the Google Cloud,” McGoff said in recent weeks.
McGoff and his Irish team have also spearheaded unique engineering projects aimed at power usage efficiency (PUE), which involves reducing the amount of energy used to cool data centres.
“That is the holy grail – zero power to cool computers – and we have embarked on unique projects for data centres all over Europe. For example, in Finland, we have used undersea cooling to cool data centres and pump slightly warmer seawater back into the Gulf of Finland.”
Spotify image via Shutterstock
