Recommender systems typically are optimized to produce a top-N list reflective of the most-highly recommended items a user has not yet rated. However, there are many reasons to believe that this order may not be the best order to present items to users, either within or across sessions.
First, top-N does not consider whether a recommendation has already been displayed to the user before, that is, whether it is fresh vs. potentially stale. Second, presenting the standard top-N list may create an experience where continued exploration results in a sense of finding ever-worse alternatives recommended. In this paper, we explore two alternatives to the standard top-N approach designed to address these concerns.
Cycling recommendations demotes recommended items after they have been viewed several times, while promoting fresher recommendations from the lower portions of the list. Serpentining displays a “zig-zag” order, in which the best recommendations (i.e., the top recommendations from a rating prediction model) are spread across several pages, offering high-quality items on each page as a user continues to explore. Cycling may happen within the same visit or across multiple visits, which we call intra-session or inter-session cycling. Intra-session cycling creates a more immediate and noticeable change but may cause confusion because potentially interesting recommendations may disappear when a user goes back to the previous page. Inter-session cycling is less likely to have this problem but may not be noticeable because users have forgotten what they saw previously.
We conducted a field experiment in MovieLens (https://movielens.org) to test these two approaches of re-thinking about Top-N recommendation list. We found interesting tensions between opt-outs and activities, user perceived accuracy and freshness. Intra-session cycling might be a “love it or hate it” recommender property, because users in it have a higher opt-out rate, but also engage in more activities such as page views, ratings, clicks and wishlistings, especially for those who stay. Intersession cycling and serpentining increase activity without significantly increasing opt-out rate. Users perceive more change and freshness on cycled recommendations and less accuracy, familiarity on both cycled and serpentined recommendations. Combining cycling and serpentining does not work as well as each individual manipulation.
Here are two interesting overall messages from our research (paper link):
- Not always showing the best content in Top-N recommendation increases user activities and hence is a promising direction to help improve user temporal experience in recommender systems.
- Measuring user subjective perception on recommendations is important because increased objective activity metrics do not necessarily indicate better user experience.
