“We have our own custom dashboard so that literally every bet that’s placed I can tell you how many were done in the last minute,” says Paddy Power CIO Fin Goulding.
This week, Irish punters will place €100m worth of bets at the Cheltenham horse-racing festival. At the heart of the action is Paddy Power, the Irish bookmaker whose operations extend from retail on the high street to the web and apps on smartphones in your pocket.
Paddy Power, which recently reported pre-tax profits of €167m on revenues of €882m, operates 243 shops across Ireland and opened 20 new shops last year. The company’s retail stakes exceeded €1bn for the first time.
Ensuring all these systems, from the stores to the apps, as well as the internal reporting systems, perform smoothly and that every bet is placed securely is Goulding.
Prior to joining Paddy Power, Goulding held senior technology leadership positions at Lastminute.com, Sabre, Visa, HSBC and Nat West.
Can you outline the breadth and scope of the technology rollout across your organisation and what improvements it will bring to the company?
As CIO I am in charge of all of technology; everything from portfolio and project management to architecture and solutions design, development, technical operations and the support of the production systems.
We are a digital business. We are an e-commerce business but we are also a retail business and we are proud of our retail tradition and so we have a large retail real estate. And then we have our own internal systems, as well, so I’ve got internal stakeholders in finance, HR, etc, so you’ve got a whole gamut of things which for me is great so I am not just specialising in one area or another.
We are a mobile-first business at the moment. Everybody in Paddy Power outside of my team has some technical capability or profession that they bring to our organisation. There are people who are not in my team but are former technologists who have a lot of context and it makes it good on one hand but does also mean you really have to be on top of your game. It’s healthy, it’s fun.
What are the main points of your company’s IT strategy?
We have a new CEO Andy McCue and so he is doing a reboot of business strategy and I would say previously our strategy was around coping with this phenomenal growth. People talk about transformation but I always talk about evolution, how we evolve really quickly to keep up with the pace.
Andy’s view is what we need to do is be a bit more focused. Let’s place our bets in certain areas, have a really strong flow of innovation and ideas for the next thing that’s coming along. But we as a company need to focus on certain things and that means our technology strategy is adapted towards scale, efficiency, speed, pace, delivery. And so when I talk about other aspects of what we are up to and the big challenges it is meeting that demand and making the company even more successful than it currently is.
Can you give a snapshot of how extensive your IT infrastructure is?
Cloud is important to us at an enterprise level but we are not really necessarily running our production systems in the cloud. What is happening with cloud is that it is taking care of some of our corporate stuff like HR systems, finance systems, even our own ticketing systems and our helpdesk systems are all cloud based because really it’s a core competency in terms of delivering the solution but not necessarily needing to manage all of that infrastructure. My guys need to be managing the website, the mobile sites and so on. I think it is good that we have people working in those areas. Cloud is really taking on all of that kind of corporate work for us, which is great.
We see cloud on a production side to be moving through this phase of hybrid cloud, giving you capacity on demand and giving you recovery as a service and things like that. Going with that we take security really, really seriously – just like a bank – we just can’t just pick a cloud supplier and use our credit card and start using the capacity, you need to go through the whole accreditation checking and everything else to make sure it is truly enterprise level.
We have a credit card cloud for certain applications to give small capacity.
Enterprise level is a different ballgame, now we are talking about software defined networking, software defined data centres and that is one of my challenges – how you can actually flip your internal systems and extend them into the cloud and then scale the back again once the demand is not there.
We have big events like Cheltenham, the Grand National, the World Cup, the Olympics, everything. You need to be able to scale up and scale back again. Traditionally we just bought a lot of hardware as much as possible ahead of time, which is highly inefficient but it was the only way of doing that then.
I now see a different way forward but we have regulations in compliance and security to worry about and that’s why cloud adoptions is a bit more tricky but not impossible.
Do you have a large in-house IT team, or do you look to strategically outsource where possible?
Currently we have got 400 people across the different areas of the profession and we have a team also from a games company we bought some time ago that builds games for our websites and mobile platforms and they are really a great bunch of people with great skills. We decided to extend to them so we are growing to about 70 or 80 there, but they are part of our team. I wouldn’t call it outsourcing or offshoring, but tapping into talent because here in Dublin we are playing against some serious providers of technology and some really good companies to work for and we like to think that we are one of those companies.
We don’t own our own data centres, we actually rent the space and manage it ourselves, but with such rapid growth there isn’t really a need for us to do any true outsourcing at the moment.
We use all the cloud providers in some form or another but they are very much for targeted solutions and use cases. We would use Amazon, Microsoft, but because we are a big virtual machine (VM) shop we have the ability to use any provider that we want.
What are some of the main responsibilities of your own role, and how much of it is spent on deep technical issues compared to the management and business side?
Everything. That’s why a CIO role is very hard. A lot of people ask me what is the difference between a CIO and a CTO and I don’t think there really is much of a difference. A CIO is a CTO that can explain in business terms what they are trying to do.
I don’t have any credibility if I can’t talk to my team about deep technical stuff and I love it, that’s my profession. But I also have to allow my people to deliver those things and I need to go back to the business quite frequently and in fact “the business” is the wrong word, we are the business – IT is the business.
I spend my time dealing with all the senior leaders talking about the business side of our organisation but then at the same time I’m a big fanboy when it comes to things like DevOps and container type technologies. That’s why I am in the profession.
Paddy Power has a huge engine behind it and we are continually adding to the capabilities and also becoming more efficient as we go along. The nice thing about being a CIO is that in previous jobs I always worked for other senior technologist in some form or another. Even as CIO at LastMinute.com there was a global CTO and so here it is just me and so that’s scary opportunity also to make decisions and take a large group of people in a certain direction. That is where the business and leadership skills come in – it’s like bringing people on the journey and selling it and making them understand it. I think also people here know I am a technologist at heart.
What are the big trends and challenges in your sector, and how do you plan to use IT to address them?
A lot of the big changes I see are already maturing. Big data is maturing, it is becoming real.
I see software-defined data centres where there is a challenge in infrastructure – that is taking routers and switches and turning that into software so that the data centre itself could be on a hard disk essentially and replicated into the cloud.
SDX as they call it, the whole software-defined world, is newish and we are very much in the forefront of doing stuff in that area.
The other area is more around agility and you hear a lot of conversations out there about DevOps, lean, kanban stuff like that, and all of these things no one has put them all together until now which is what we are doing.
It’s just so exciting. We can build a feature on some platforms and you can have the idea now and be in production today. You could never do that before and we have some systems that are pretty agile in how they operate but they are delivering in two or three weeks, not good enough. It has to be pretty much same day and some projects, even big projects, it is incremental benefits. Even though here might be 100 things in that project, as soon as the first 10 are ready get them out, start building on it, that’s the challenge, the pace of business is completely changed and I find it exciting.
What metrics or measurement tools do you use to gauge how well IT is performing?
We monitor everything, literally from the entire applications stack to the virtual machines but also we are logging everything and reviewing the logs all the time so we have a lot of Splunk usage and we have our own custom dashboard so that literally every bet that’s placed I can tell you how many were done in the last minute.
There is so much data, our challenge is making that into a form that you can digest. But certainly building stuff and getting it out fast is something we use our lean practices for. Once systems are in production there is an awful lot of data around availability and meantime between failures.
I don’t want to sound too much like a bank because we are actually a cool place, but we have all the large systems. In a DevOps world developers should build code, test, deploy and support. In our world we have a separate support function as well which you wouldn’t have in a smaller DevOps shop. We just can’t afford to have any downtime, it is 24×7 support. People on site, we have a full Network Operations Centre (NOC) and those guys keep us in business because it is fast response. I call them the Navy SEALs, they are the first ones on the beach to take care of stuff.
When the developers, engineers, architects and everybody else gets involved, the triage has been done, the repair has been put in place, now let’s work out what happened and in our own time refine the process.
You have to go through a lot of pain. We still have legacy, things to remove, still things from the past which have to be repaired but we are on that journey and it’s exciting and it is fun.