Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backfill runs into rate limits #157

Open
hbh7 opened this issue Sep 2, 2024 · 0 comments
Open

Backfill runs into rate limits #157

hbh7 opened this issue Sep 2, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@hbh7
Copy link

hbh7 commented Sep 2, 2024

Hi,

I've been playing around with Matrix and this bridge recently, and I think I ran into a bug.

{"level":"error","user_id":"@hbh7:hbh7.com","component":"discordgo","caller":"restapi.go:298:RequestWithLockedBucket()","time":"2024-09-02T00:32:03.941281651Z","message":"rate limit unmarshal error, invalid character '<' looking for beginning of value"}
{"level":"error","channel_id":"<id>","channel_receiver":"<id>","room_id":"!<id>:hbh7.com","action":"initial backfill","room_id":"!<id>:hbh7.com","limit":1000000000,"error":"invalid character '<' looking for beginning of value","time":"2024-09-02T00:32:03.941323181Z","message":"Error collecting messages to forward backfill"}

Many of these messages are seen in the log shortly after starting an initial backfill, and it leaves many of the chats with entirely missing or partially incomplete histories.

I think it's related to this upstream issue: bwmarrin/discordgo#659

Seems like some sort of retry system should be implemented to work around rate limits, and/or some sort of built-in/configurable rate limiting within the bridge like I've seen in other bridges. The current path seems to be to ignore the failure and pretend everything imported successfully, which is definitely not ideal.

Hoping someone knowledgeable can build a solution. Thanks!

@hbh7 hbh7 added the bug Something isn't working label Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

1 participant