Search professionals in internet firms from Netflix to Facebook and LinkedIn are focusing on a new generation of contextual algorithms that better match users with the content they will enjoy from video to music and news, the director of product innovation at Netflix Carlos Gomez Uribe told Siliconrepublic.com.
Uribe was in Dublin recently at the Recommender Systems Conference, where more than 300 professionals working in different aspects of personalisation from organisations like Facebook and LinkedIn, were gathered.
Not only is Netflix available across 800 connected devices, but it tries to read users’ minds from the moment they sign in based on 20 questions that match them with genres like action or romance that match their tastes.
As time goes on, Netflix gets to know users better by analysing what they’ve watched, how long they watched it and how quickly they jumped to the next episode of a show. This additional information is added to algorithms to enhanced members’ personal recommendations.
The process involves dozens of algorithms that result in recommendation sets, such as members’ top 10, content similar to films they’ve watched and personalised genres.
The tricky part is personalisation because it involves large amounts of data to respond in real-time.
Netflix calculates new personalisation for every single user all the time – an immense engineering challenge. Netflix’s integration with Facebook simplifies much of this by allowing the video site to show members what their friends had been watching.
“I work with several teams who are responsible for all the algorithms that match users with the shows and movies they will enjoy,” Uribe explained.
“The ultimate vision is that as soon as you log into your Netflix account the best video for you will start playing and you’ll be thrilled because we got it exactly right.
“This ultimately will be depending on the time of the day and all the other interaction data we have from your account – what you watched before, what you searched for and how you rated it. The key is to know your video tastes and know what is exactly right for you.
“We will probably never get there exactly but the key is to get to the truth.”
The optimisation paradox
There is a certain paradox in how algorithms are being developed to know as much about users tastes as possible, yet Uribe insists as far as the algorithms are concerned it’s not people they are studying, but statistics.
“Our algorithms are never trying to put labels on people of any time – they are mostly statistical algorithms that look for patterns. But never to say this is the child, mother or father on an account – it is better to assume that this account likes videos about trains, enjoys French thrillers and so on. The algorithm doesn’t really care if this means there’s one person who really likes all those things or that there’s one person for each of the themes in the account. We care that we identify all those themes and make sure we have good recommendations for each of them.”
Having to serve 800 different media devices complicates things. “Different devices rely on different technologies and it is difficult to support them all at once. It’s a challenge but at the same time we want to make our services as accessible and easy to use as possible.
“The reality for our members is that it is very important to be able to access the service from any of the devices they have and so we will continue to push for being on as many devices as possible.”
At the recent conference, the professionals from the major new media houses concentrated on sharing knowledge on new mathematical techniques and algorithms that have been developed.
“There are lots of new algorithms to figure out, for example, how to use contextual information like time of day, the day of week, to improve recommendations. There are vectoring and mathematical problems and we have to match new algorithms against historical data.
“Some people have found better ways to retrain those algorithms using historical data. A key aspect is optimisation criteria; there have been developments around optimising different criteria to be more in line with the real performance of the algorithm and everyone is real excited about trying them out,” Uribe explained.