Most common questions about recommender systems…
I get ten to fifteen questions a week on recommender systems from entrepreneurs and engineers. Sometimes, I help people find their way in the literature. On occasion—for a consulting fee—I get my hands dirty and evaluate, design or code specific algorithms. But mostly, I answer the same questions again and again:
1. How much data do I need?
Given your data, you can use cross-validation or A/B testing to measure objectively the effectiveness of a recommender system.
2. We have this system in place, how do we know whether it is sane?
See previous question.
3. My online recommender system is slow!
Laziness is your friend: don’t recompute the recommendations each time you have new data.
4. My customers don’t like the recommendations!
- Keep expectations in check: recommending products is difficult and even human beings have trouble doing it,
- Explain the recommendations: nobody trusts a black box,
- Allow your users to freely explore your data and products in convenient and exciting ways.
5. Which algorithm is best?
You should start with simple algorithms: it worked well enough for Amazon. To do better, a mix of different algorithms is probably best. You can combine them using ensemble methods.
Montreal, Canada 
Follow on
My answer to #5 in commercial settings is “whichever makes you the most money”.
Measuring profitability is no trivial task. Conversion rates, usage, churn and word of mouth can all be affected by the quality of the recommendations.
There are hints in the literature about ways that recommenders can indirectly add revenue, see e.g. http://portal.acm.org/citation.cfm?id=1454054&dl=GUIDE&coll=GUIDE&CFID=77217953&CFTOKEN=60022310
Comment by Daniel Haran — 22/2/2010 @ 12:48
Excellent point, profitability is important.
Comment by Daniel Lemire — 22/2/2010 @ 13:18
Nice post.
Mixing methods is a good strategy, because the system is able ‘to learn’ (user feedback), which strategy fits best to customers (on a user level).
However, it is important to ‘justify’ and to explain to users, why a particular item is recommended.
Comment by Marcel Blattner — 23/2/2010 @ 2:21