KAIST CCC-Project FM & Online FM for Recommendation Systems

In this project, we study factorization machine and its variations for recommendation systems. Here, we consider the two problem settings for recommendation systems: (1) Offline setting that assumes that given sufficient amount of data, the model trains the given dataset and then conducts the recommendations tasks, (2) Online setting that assumes that as data points are given sequentially, the model sequentially trains the data and recommends the items.

What we have done for this project can be summarized as :
(1) We study the previous models and implement them based on pytorch for offline setting and online setting.
(2) We try to propose Online Neural Factorization Machine.
(3) We propose Meta Embedding Factorization machine designed for the specific dataset which requires the large number of feature embeddings for some of the features.

For more details, you can check it in models and jupyters directory.

Models for Offline Setting

• Factorization Machine
• Meta Embedding Factorization Machine (proposed)
• Deep Factorization Machine
• Neural Factorization Machine
• Attentional Factorization Machine

Models for Online Setting

• Online Factorization Machine by FTRL, Sketched FTRL
• Online Deep Factorization Machine by SGD
• Online Neural Factorization Machine by SGD
• Online Deep Factorization Machine by Hedge Backpropagation (proposed)
• Online Neural Factorization Machine by Hedge Backpropagation (proposed)

Dataset

Real Valued Dataset

[1] movielens 100k
[2] YearPredictionMSD

Discrete Valued Dataset

[3] cod-rna
[4] Frappe
[5] Criteo

Reference Code

[1] PyTorch Implementations of Factorization Machines
[2] Factorization Machine Review Paper
[3] A PyTorch Implementation of DeepFM for CTR Prediction Problem
[4] Online Deep Learning: Learning Deep Neural Networks on the Fly
[5] Sketched Follow-The-Regularized-Leader for Online Factorization Machine
[6] Large-scale Online Kernel Learning with Random Feature Reparameterization

Funding

This research was supported by the Korean MSIT (Ministry of Science and ICT), under the National Program for Excellence in SW (2016-0-00018), supervised by the IITP (Institute for Information communication Technology Planning and Evaluation)

Author

Yohan Jung (becre1776@kaist.ac.kr)
Yeji Han (yejihan@kaist.ac.kr)