Summary:  In this multi-part series we walk through the full landscape of Recommenders.  In this article we cover business considerations as well as issues for Recommenders as a group.  In the next articles we’ll discuss the details of the five major types of recommenders, improving their performance, and finally the coming impact of deep learning on Recommenders.

For all types of ecommerce it’s likely that recommenders are the single most important analytic tool you can deploy.  Although hard data is difficult to come by, many informed sources estimate that for the major ecommerce platforms like Amazon and Netflix that recommenders may be responsible for as much as 10% to 25% of incremental revenue.

With that much at stake financially, not only do you need to optimize their performance but you can also justify putting significant investment behind getting the resources necessary to do so.  Recommenders is just one of the many fast evolving specialties within predictive analytics and you’ll want to find experienced data scientists to help you.

If you’re new to data science or if you’ve never had to deal with optimizing recommenders before you may be confused as where to start.  Pretty much every major analytic platform and cloud service offers some form of an out-of-the-box recommender you could implement.  But that doesn’t answer the questions:

• Is this the best one for my circumstances?
• What’s the algorithm at the core of this particular recommender?
• How do I optimize its performance?
• Can I use more than one type?

What Are Recommenders?  Simple and Not So Simple.

Recommenders are simple at least in their objective.  They are the applications which personalize your customer’s shopping experience by recommending next best options in light of their recent buying or browsing activity. 

However, the question of exactly what alternatives or recommendations to present requires you to also evaluate the following issues:

1. Substitutes: Does your visitor know what he wants to buy but can’t find it?  Are there suitable substitute products you could present?
2. Complements: Your visitor knows what he wants to buy and has it in his basket.  Are there complementary add-on products you should be presenting to increase his satisfaction and your sales?
3. Ideas: Your customer is browsing for new items in which he might be interested.  What are the products that he’s most likely to purchase?
4. Sequence:  Frequently the order in which multiple recommendations are presented is important in maximizing satisfaction.  This is called order of consumption.  What is the right order of consumption for your items?  Order also encompasses emphasis.  If you are presenting pictures of 10 alternate shirts or shoes, which should be given prominence on the page?

Recommenders can answer all these issues but how you select and implement them will be different.

What can be a little confusing is that many applications of predictive analytics seem to be doing the same thing.  That’s true.  The many scoring models we use for outreach and in-bound marketing to target up sell, cross sell, and even first sell offers are in a sense recommenders.  But for a clearer understanding we’ll adopt the more parochial definition that recommenders are meant to personalize web interactions to encourage your customer to consume more or linger longer.

In Very Different Environments

One of the considerations that you should keep in mind as you read on is the really tremendous differences between seemingly similar ecommerce environments.  Consider and compare:

• Movies, books, and music – huge rapidly changing inventories for immediate consumption.
• Electronics or hardware - large but more stable inventories with very large subcategories of complimentary and supplementary choices and many different buyer objectives and motives. 
• Women’s clothing – large fast changing inventories with far fewer attributes for making objective or hard comparisons but a very large number of sub-attributes like color and size.

We’ll use recommenders to maximize sales in all of these environments but their implementation will clearly be very different.

Just to be clear, while recommenders obviously apply to ecommerce apps where your customer is buying something from you, they’re equally applicable to applications where you’re trying to maximize your user’s time on your site for the purpose of increasing advertising exposures.  Good examples are news sites like Yahoo that personalize your feed so that you spend more time on the site.

Also, while we’re going to talk about ecommerce applications, be on the lookout for real-world applications of these same techniques.  Some of these recommender categories like Association and Market Basket are just as useful for outreach and in-bound marketing as they are on line.

Recommenders Are Not All the Same

Recommenders are not all the same.  If you are implementing a recommender for the first time it’s very likely you’ll be starting with an out-of-the-box solution from a vendor though you may also elect to write one from scratch using R or Python.  In addition to the tactical issues described above about what exactly is it that you need to help your customer accomplish, you will want to focus on optimizing overall sales.  That means looking deeply into the different kinds of recommenders and understanding the strengths and weaknesses of each.

• They rely on different algorithms. 
• Some have a greater focus on customer similarities, some on content similarities, and some a blend of the two.
• Different algorithms vary in their utility based on how much you know in advance about your customers, your content, and whether you can get your customers to give you feedback about their purchase.
• All must meet the general requirement of serving a personalized result within about 50 ms. 
• The scoring tags you place on customers and content records are seldom actually real-time and are typically updated overnight, sometimes more or less frequently depending on your circumstance.  So off-line scoring updates may rely on in-recommender calculations but they can be supplemented with scoring and analysis performed separately on a longer time scale.

5 Types of Recommenders

In the second article in this series we’ll talk in detail about the five main types of recommenders, how they work, and their relative strengths and weaknesses.  Those five are:

1. Most Popular Item
2. Association and Market Basket Models
3. Content Filtering
4. Collaborative Filtering
5. Hybrid Models

Advances in basic recommender analytics has not progressed as rapidly as some other areas of data science.  The most complex of these options, Collaborative Filtering has been around for five or ten years.  From a business perspective the good news is that you can feel comfortable building a portfolio from existing technology.  Most of the advances in the data science around recommenders have been focused on incrementally improving their performance in the specific domains in which they’ve been deployed.  That will undoubtedly be your focus also.

 Other Articles in this Series

Article 2: “5 Types of Recommenders” details the most dominant styles of Recommenders.

Article 3: “Recommenders: Packaged Solutions or Home Grown” how to acquire different types of recommenders and how those sources differ. 

Article 4: “Deep Learning and Recommenders” looks to the future to see how the rapidly emerging capabilities of Deep Learning can be used to enhance performance.

About the author:  Bill Vorhies is Editorial Director for Data Science Central and has practiced as a data scientist and commercial predictive modeler since 2001.  He can be reached at:

[email protected]