Skip to main content

Posts

Showing posts with the label collaborative filtering

Designing The Next-generation Review And Recommendation System

It's unfortunate that despite of the popularity of social networks and plenty of other services that leverage network effects, the review and recommendation systems that are supposed to help users make the right decisions haven't changed much. Thumbs-up and thumbs-down or likes and unlikes signal two things: popularity and polarization. If a YouTube video has 400 thumbs-up and 500 thumbs-down it means that the video is popular as well as polarized, but it doesn't tell me whether I will like it or not. The star review system also signals two things - on average how good something is and whether it's significant or not. There are multiple problems with this approach. An item with 8 reviews, all 5 stars, could be really bad compared to an item that has 300 reviews with 3.5 stars. Star ratings alone, without associated descriptive reviews, wouldn't make much sense if there aren't enough people who have reviewed the item. Also, relying on an average rating alone coul...

Data Is More Important Than Algorithms

Netflix Similarity Map In 2006 Netflix offered to pay a million dollar, popularly known as the Netflix Prize , to whoever could help Netflix improve their recommendation system by at least 10%. A year later Korbel team won the Progress Prize by improving Netflix's recommendation system by 8.43%. They also gave the source code to Netflix of their 107 algorithms and 2000 hours of work. Netflix looked at these algorithms and decided to implement two main algorithms out of it to improve their recommendation system. Netflix did face some challenges but they managed to deploy these algorithms into their production system. Two years later Netflix awarded the grand prize of $1 million to the work that involved hundreds of predictive models and algorithms. They evaluated these new methods and decided not to implement them . This is what they had to say: "We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineer...

Learning From Elevators To Design Dynamic Systems

Elevators suck. They are not smart enough to know which floor you might want to go. They aren't designed to avoid crowding in single elevator. And they make people press buttons twice, once to call an elevator and then to let it know which floor you want to go to. This all changed during my recent trip to Brazil when I saw the newer kind of elevators. These elevators have a common button panel outside in the lobby area of a high rise building. All people are required to enter their respective floor numbers and the machine will display a specific elevator number that they should get into. Once you enter into an elevator you don't press any numbers. In fact the elevators have no buttons at all. The elevator would highlight the floor numbers that it would stop at. That's it! I love this redesigned experience of elevators. It solves a numbers of problems. The old style elevators could not predict the demand. Now the system exactly knows how many people are waiting at what floor...

Amazon's Re-designed Review System Generates More Revenue But Has Plenty Of Untapped Potential

Amazon's design tweaks to its review system has resulted into $2.7 billion of new revenue argues Jared Spool. Other people have also picked up this story with their analysis . I am wary of absolute revenue numbers tied to a feature to derive lost opportunity cost since a variety of other things could have driven the sale. It is wrong to assume that people would not have bought the products had the feature not existed. However I do believe it is a great step in the direction of making the review system more useful and drive more clickthroughs and conversions. Simply the presence of the reviews, magic number 20 in this case , motivates consumers to drill down into the details of a product and its reviews. Amazon has made significant progress in collaborative filtering through their review system and it is an exemplary of a long tail business model. It has helped consumers to gain transparency and has also helped expose issues with the products. This is not enough. As an e-commerce m...

Does Cloud Computing Help Create Network Effect To Support Crowdsourcing And Collaborative Filtering?

Nick has a long post about Tim O'Reilly not getting the cloud . He questions Tim's assumptions on Web 2.0, network effects, power laws, and cloud computing . Both of them have good points. O'Reilly comments on the cloud in the context of network effects: "Cloud computing, at least in the sense that Hugh seems to be using the term, as a synonym for the infrastructure level of the cloud as best exemplified by Amazon S3 and EC2, doesn't have this kind of dynamic." Nick argues: "The network effect is indeed an important force shaping business online, and O'Reilly is right to remind us of that fact. But he's wrong to suggest that the network effect is the only or the most powerful means of achieving superior market share or profitability online or that it will be the defining formative factor for cloud computing." Both of them also argue about applying power laws to the cloud computing. I am with Nick on the power laws but strongly disagree with ...