Lazy collaborative filtering with dynamic neighborhoods

Abstract

Purpose

The purpose of this paper is to address the scalability issue and produce high-quality recommendation that best matches the user’s current preference in the dynamically growing datasets in the context of memory-based collaborative filtering methods using temporal information.

Design/methodology/approach

The proposed method is formalized as time-dependent collaborative filtering method. For each item, a set of influential neighbors is identified by using the truncated version of similarity computation based on the timestamp. Then, recent n transactions are used to generate the recommendation that reflect the recent preference of the active user. The proposed method, lazy collaborative filtering with dynamic neighborhoods (LCFDN), is further scaled up by implementing in spark using parallel processing paradigm MapReduce. The experiments conducted on MovieLens dataset reveal that LCFDN implemented on MapReduce is more efficient and achieves good performance than the existing methods.

Findings

The results of the experimental study clearly show that not all ratings provide valuable information. Recommendation system based on LCFDN increases the efficiency of predictions by selecting the most influential neighbors based on the temporal information. The pruning of the recent transactions of the user also addresses the user’s preference drifts and is more scalable when compared to state-of-art methods.

Research limitations/implications

In the proposed method, LCFDN, the neighborhood space is dynamically adjusted based on the temporal information. In addition, the LCFDN also determines the user’s current interest based on the recent preference or purchase details. This method is designed to continuously track the user’s preference with the growing dataset which makes it suitable to be implemented in the e-commerce industry. Compared with the state-of-art methods, this method provides high-quality recommendation with good efficiency.

Originality/value

The LCFDN is an extension of collaborative filtering with temporal information used as context. The dynamic nature of data and user’s preference drifts are addressed in the proposed method by dynamically adapting the neighbors. To improve the scalability, the proposed method is implemented in big data environment using MapReduce. The proposed recommendation system provides greater prediction accuracy than the traditional recommender systems.

Citation

G., S., S.P., S.I. and Li, G. (2018), "Lazy collaborative filtering with dynamic neighborhoods", Information Discovery and Delivery, Vol. 46 No. 2, pp. 95-109. https://doi.org/10.1108/IDD-02-2018-0007