New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RWKV-v5] use register_buffer instead of frozen params #213

Open

kashif wants to merge 1 commit into BlinkDL:main from kashif:buffer

kashif commented Dec 19, 2023

also fixed a bug due to this in MishGLU


          use register_buffer

7dc6c0c

Owner

BlinkDL commented Dec 20, 2023

these parameters are trainable
so we use nn.Parameter

Author

kashif commented Dec 20, 2023

ah ok then what does the with torch.no_grad(): do?

Author

kashif commented Dec 29, 2023

@BlinkDL happy to close this PR or should i remove the torch.no_grad()?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet