Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regarding the issue with reproduction #4

Open
zhaijianyang opened this issue Dec 14, 2023 · 4 comments
Open

Regarding the issue with reproduction #4

zhaijianyang opened this issue Dec 14, 2023 · 4 comments

Comments

@zhaijianyang
Copy link

Hello. I have encountered an issue with reproducing the results. I replicated two methods, Recformer and UniSRec, using default parameters. The results for Recformer seem reasonable, but the performance of the reproduced UniSRec is significantly better than reported in the paper. What could be the reason for this?
image

@JiachengLi1995
Copy link
Owner

Are you using UniSRec with text only or text+IDs? In this paper, we tested the text-only version of UniSRec.

@zhaijianyang
Copy link
Author

I conducted experiments by incorporating IDs, and the results are as follows. What confuses me even more is that I did not pre-train either of the methods; I only utilized the pre-training parameters of the language model.
image

@JiachengLi1995
Copy link
Owner

Which category are you working on? Do you mean you didn't pretrain Recformer and UniSRec?

@zhaijianyang
Copy link
Author

I'm sorry, I misspoke earlier. UniSRec actually performed worse after pretraining.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants