1 year, 5 months ago

Rhyme or Reason – The Logic of Netflix

@GuyLongworth, who teaches philosophy at Warwick, is puzzled by the Netflix recommendation algorithm.

Having seen both, I can only think that this must have to do with rhyme.

— Guy Longworth (@GuyLongworth) June 29, 2017

Philosopher Guy’s appeal to rhyme rather than reason seems to be based on the view that the two films have nothing else in common. But this is rather contradicted by the fact that he has actually seen both. Netflix has correctly surmised that people like Guy might possibly be interested in both films.

The first thing to understand about recommendation algorithms is that they are not solely (if at all) based on the intrinsic similarity of two products, but on what we might call relational similarity. If I tell you that people who like pizza also like ice-cream, that is primarily a statement about the “people who like”. You might try to explain this statement by observing that pizza and ice-cream both have a high fat content, but then so do lots of other foods.

And when someone has just eaten a pizza, it is perhaps more likely that they will go on to eat ice-cream next, rather than eating another pizza straightaway.

Would it be virtue signalling of me to reveal that I resisted the lure of the second pizza?

— Guy Longworth (@GuyLongworth) June 22, 2017

The second thing to understand is that recommendation algorithms work by trial and error. Netflix wants to know if Guy will accept its suggestion to re-watch Annie Hall, and this feedback will add to its knowledge of Guy as well as its knowledge of relational similarity between films.

Trial and error works better if you have a diverse range of trials. If you watch a couple of films in a particular genre, and then Netflix only ever shows you suggestions within that genre, it will never discover that you might be interested in a completely different genre as well. And you will never discover the full range of Netflix offerings, which could result in your abandoning Netflix altogether.

Diversity of suggestion adds to the richness of the experimental data that are generated. How many members of the “people like Guy” category respond positively to suggestion A, and how many to suggestion B? Todd Yellin, Netflix VP of Product, told journalists in March that “we are addicted to the methodology of A/B testing”.

What is genre anyway? In the past, genres (in book publishing, music, film, video games) were defined by the industry or by experts. In 2013, Netflix employed over 40 people hand-tagging TV shows and movies. But a data-driven approach allows genres to emerge organically from the patterns of consumption. Netflix (and Amazon and the rest) will be much more interested in data-defined genres than in industry-defined genres.

In her rant against the Netflix algorithm, @mehreenkasana makes two apparently contrary complaints. On the one hand, Netflix offers her content that is nothing like anything she has ever watched. She dismisses one suggestion with the words “I’ve never watched a show in a remotely similar vein.” On the other hand, she doesn’t see how Netflix can offer her challenging experiences. “Intensely curated experiences, whether you’re looking to explore movies or to meet people to date, remove one of the most critical aspects of a rich experience: risk, as in going out of your comfort zone.”

But as @larakiara explains, “personalization is key to ensuring users keep coming back. But there’s also the problem of over-personalization, so Netflix has to introduce variants.”

Thus we can see Netflix as an embodiment of at least three of @kevin2kelly’s Nine Laws of God.

  • Control from the bottom up
  • Maximize the fringes
  • Honor your errors

“A trick will only work for a while, until everyone else is doing it.” (Remember Blockbuster.)


Mehreen Kasana, Netflix’s recommendation algorithm sucks (The Outline, 24 March 2017)

Kevin Kelly, Nine Laws of God. Chapter 24 of Out of Control (1994)

Lara O’Reilly, Netflix lifted the lid on how the algorithm that recommends you titles to watch actually works (Business Insider, 26 February 2016)

Janko Roettgers, Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Downs (Variety, 16 March 2017)

Tom Vanderbilt, The Science Behind the Netflix Algorithms That Decide What You’ll Watch Next (Wired, 7 August 2013)

3 years, 11 months ago

From Coincidensity to Consilience

In my post From Convenience to Consilience – “Technology Alone Is Not Enough”  (October 2011), I praised Steve Jobs for his role in the design of the Pixar campus, whose physical layout was intended to bring different specialists together in serendipitous interactions.

Thanks to @jhagel and @CoCreatr, I have just read a blogpost by @StoweBoyd commenting on a related project at Google to build a new Googleplex. Because this is Google, this is a bottom-up data-driven project: it is based on a predicted metric of coincidensity, which is sometimes defined as the likelihood of serendipity.

With the right technology (for example, electronic monitoring of the corridors and/or tagging of employees), a corporation like Google can easily monitor and control “casual collisions of the work force”.

But as Ilkka Kakko (@Serendipitor) points out, such measures of coincidensity cannot be equated with true serendipity. I wonder whether Google will be able to correlate casual meetings with enhanced knowledge and understanding, and measure the consequent quantity and quality of innovation? And then reconfigure the campus to improve the results? Hm.

However, the principle of designing physical spaces for human activity rather than for visual elegance is a good one, as is the notion of evidence-based design. Form following function.


Stowe Boyd, Building From The Inside Out (February 2013)

Paul Goldberger, Exclusive Preview: Google’s New Built-from-Scratch Googleplex (Vanity Fair, February 2013)

Ilkka Kakko, Are we reducing the magic of serendipity to the logic of coincidence? (April 2013)

3 years, 11 months ago

From Coincidensity to Consilience

In my post From Convenience to Consilience – “Technology Alone Is Not Enough”  (October 2011), I praised Steve Jobs for his role in the design of the Pixar campus, whose physical layout was intended to bring different specialists together in serendipitous interactions.

Thanks to @jhagel and @CoCreatr, I have just read a blogpost by @StoweBoyd commenting on a related project at Google to build a new Googleplex. Because this is Google, this is a bottom-up data-driven project: it is based on a predicted metric of coincidensity, which is sometimes defined as the likelihood of serendipity.

With the right technology (for example, electronic monitoring of the corridors and/or tagging of employees), a corporation like Google can easily monitor and control “casual collisions of the work force”.

But as Ilkka Kakko (@Serendipitor) points out, such measures of coincidensity cannot be equated with true serendipity. I wonder whether Google will be able to correlate casual meetings with enhanced knowledge and understanding, and measure the consequent quantity and quality of innovation? And then reconfigure the campus to improve the results? Hm.

However, the principle of designing physical spaces for human activity rather than for visual elegance is a good one, as is the notion of evidence-based design. Form following function.


Stowe Boyd, Building From The Inside Out (February 2013)

Paul Goldberger, Exclusive Preview: Google’s New Built-from-Scratch Googleplex (Vanity Fair, February 2013)

Ilkka Kakko, Are we reducing the magic of serendipity to the logic of coincidence? (April 2013)