2. Non personalized recommender
Non-personalized recommendations are those that will recommend the same thing no matter which user is consulting the recommender. These can include recommending a given item of interest, a list of most popular items or even a list of random items. The purpose of including such recommendations in this work is to obtain a baseline on how well such a recommendation works and see whether personalized recommenders will improve the recommendations.
The script basic_recommenders.py provides a diverse list of recommendations per user that includes recommending a set of random items, a set of the works with the largest number of words, a set of the most recent works and a set of popularity based recommendations that take into account number of comments, number of hits, number of bookmarks and total number of likes. Note that the total number of likes include those of non-registered users and as such are larger than the ones we use in the dataset. In order to obtain such recommendations we need to execute:
python3 ./2_non_personalized_recommender/basic_recommenders.py -t train.u2i.txt -i metadata_fics.clean.txt -o recom.Random.txt -m Random -k 50
Where -t is the user to item table. In this case, as we will want to evaluate the performance of the different methods we use the training table. The -i contains the clean meatadata, -o will be the name of the output and -k the number of recommendations. -m determines which of the recommendations we will receive and as such it can take the following values: Random,Longest,MostRecent,NumComments,NumHits,NumBookmarks,NumLikes.
Note that while the recommendations in all cases except the random will be the same for all users, works that have already been read by a particular user will be removed for their recommendation so slight differences are to be expected.
I applied the different recommenders on our training dataset which is formed by 10,000 users and about 121,000 works. I evaluated their performance by calculating the following metrics: f1@k and map@k using the script provided in 7_evaluation. This gives the base metrics which the personalized methods will need to improve.
As seen in the graph, the best non-personalized recommenders are those that recommend based on popularity. The best recommender is the one based on the number of likes with a f1@k of 0.0364 and a map@k of 0.0090 and is followed by the one based on number of bookmarks (f1@k 0.0334 and map@k 0.085). As expected, the wrost recommender is the one that provides the most recently published works, because there's not been time for those works to have been read and liked, followed recommending the longest works. Both methods perform worse than the random. These results show that personalized recommenders will have to do better than the recommender based on likes.