What we saw at RecSys 2017 Conference

Marco Creatura

d&a blog

RecSys, which took place in Como last 27th August 2017, is one of the largest academic conferences on Recommender Systems (RecSys) and has reached this year its eleventh edition with an all time record of attendees (627), proving the rising importance of Recommender Systems in the current digital agenda.

At BBVA Data & Analytics, we believe that Recommender Systems can make the lives of our clients easier, and have been exploring how Recommender Systems could make more appropriate recommendations for pension plans to BBVA clients.

RecSys 2017: A brief overview

The eleventh RecSys conferences aimed at portraying recent developments in the field, trends and challenges in providing recommendation components in a range of innovative application contexts. In addition to the main technical track, RecSys 2017 featured keynote speakers from other disciplines, such as George Loewenstein (Carnegie Mellon University) who stressed that curiosity is massively under-exploited in recommender systems.

The conference also featured tutorials covering state-of-the-art in this domain, workshops, special sessions for industrial partners from different sectors such as travel, gaming and fashion industries, and a doctoral symposium. Most notably, for the first time in its history the conference was preceded by a five-day Summer School that brought together experts in the field covering a broad range of topics such as “Tools and Replication of RS Research” (Alan Said & Alejandro Bellogín), “Scalable RS in Industry” (Xavier Amatriain) or “Evaluation of RS research” (Guy Shani).

This article summarises our main takeaways from RecSys 2017:

“Accuracy does not matter, what it matters is impact” (J. Kolen)

Evaluation of offline experiments is still far away from the goal of the final application. As several speakers from industry have stated there is still a gap between academia and industry.

“There is still a gap between most collaborative filtering models and the actual goal of recommender system”

— Noam Koenigstein, Microsoft

Following this issue I have remembered that a similar topic was discussed on RecSys 2016 — see the section “Ghosts of the past (10 years)” of B.Hidasi blog post.

Analyzing the most common evaluation metric used at RecSys 2017 in the main conference papers, it is possible to see that there are still publications that optimize their model for rating prediction task (i.e. RMSE, MAE and MSE), which is distant from most real-world scenario.

In the plot above the metrics have been grouped in “Other” if they are used in less than 3 publications, e.g. hit rate, item coverage, G-score, etc.

Deep Learning for Recommender Systems

A.Karatzoglou and B.Hidasi provided a complete overview of Deep Learning (DL) and its application within the Recommender Systems domain. Their presentation showed the most relevant techniques that have been used so far for recommendation, and provided an interesting parallelism between the history of aviation and Deep Learning, both inspired by Biology.

Panelists highlighted some best practices, such as: favouring open source code, experimenting on public datasets, not using small datasets (do not use MovieLens 100k) nor working on irrelevant tasks such as rating prediction. Their tutorial is available here.

What’s Next for RecSys conference

The plenary panel, composed by J.Konstan, X.Amatriain, P.Brusilovsky and G.Karypis suggested several points about the future direction of #RecSys conference:

  • knowledge should be open to everyone : open datasets, open source code/initiatives and open publication.
    • Following the open knowledge point I checked how many papers share their code at RecSys 2017, obtaining 8 out of 46 (approx 18%). Not bad but we still need to push more in this direction!
  • keep innovation inside the conference judging a paper not only on novelty, diversity, and serendipity
  • gender diversity
  • ethics

Highlighted papers

  • Getting Deep Recommenders Fit: Bloom Embeddings for Sparse Binary Input/Output Networks [J.Serra et al.]
  • Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks [M.Quadrana et al.]
  • A Gradient-based Adaptive Learning Framework for Efficient Personal Recommendation [Y.Ning et al.]

See you next year in Vancouver!


Please contact me about any errors in summarization or attribution.