Scheduled queries not refreshing

[galenspikes]

We have about 550 scheduled queries. We noticed that some queries in a dashboard have not refreshed for a couple weeks, even though they are supposed to refresh at least daily. Looking at one query for example in the Redash application database in the queries table, it shows 0 for schedule_failures. Any ideas on how to handle this? I also don't see any errors in the log.

When I search the docker log in redash_scheduled_worker_1 for the query_id it also shows the schedule seems to be running every day. It isn't refreshing in the GUI though.

In these log entries from yesterday you can see it executes and finishes

[2024-09-18 12:31:14,206][PID:4536][INFO][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=11557841-20fa-4d86-b9d6-7b1df1f22e1f job=execute_query state=executing_query query_hash=38834c78782bbc1bb442e233ae03186e type=redshift ds_id=47 job_id=11557841-20fa-4d86-b9d6-7b1df1f22e1f queue=scheduled_queries query_id=5227 username=Scheduled
[2024-09-18 12:32:12,740][PID:4536][INFO][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=11557841-20fa-4d86-b9d6-7b1df1f22e1f job=execute_query state=checking_alerts query_hash=38834c78782bbc1bb442e233ae03186e type=redshift ds_id=47 job_id=11557841-20fa-4d86-b9d6-7b1df1f22e1f queue=scheduled_queries query_id=5227 username=Scheduled
[2024-09-18 12:32:12,741][PID:4536][INFO][rq.job.redash.tasks.queries.execution] job.func_name=redash.tasks.queries.execution.execute_query job.id=11557841-20fa-4d86-b9d6-7b1df1f22e1f job=execute_query state=finished query_hash=38834c78782bbc1bb442e233ae03186e type=redshift ds_id=47 job_id=11557841-20fa-4d86-b9d6-7b1df1f22e1f queue=scheduled_queries query_id=5227 username=Scheduled

[gaecoli]

Maybe u should restart scheduler service every day. U can set a corntab only restart scheduler. Because the python rq is not stabled.

[gaecoli]

And another reason is u use Date parameter, and Data parameter time zone is UTC.

[justinclift]

@gaecoli Nah, this seems like a different thing. It seems like the query itself is running, but it's not being cached in a way that the gui is noticing.

If that's the case, it might actually be related to the weird embeds problem I was seeing after upgrading from Redash 10.x to modern Redash.

@galenspikes Which version of Redash is this happening on? Also, as a diagnostic step, for one of the affected queries try setting it's scheduled refresh to "Never", then manually refresh the query using the gui refresh button, then try setting the scheduled refresh back to what it was. For a problem I was having that got things unstuck (for that particular query).

If that works for you, then it means it's more of a widespread problem than I was expecting and we'll need to look into it better.

[galenspikes]

@justinclift

Version: 10.0.0 (9c928bd)

I tried your diagnostic step and again I see it refresh in the scheduled worker log, but the GUI does not reflect that refresh time, it shows 8 minutes ago when I manually refreshed it.

What can be done to alleviate this? The documentation seems to be old so when I tried following steps to restart workers they didn't work. What will work with v10?

[galenspikes]

@justinclift It actually looks like it did refresh on the GUI side. I will continue to test and get back to you. If this is the case, what does that mean?

[galenspikes]

This appears to do the trick. Thanks @justinclift. With so many queries to keep track of, hopefully a fix can be pushed to the application?

[justinclift]

@arikfr I'm thinking the issue you started looking at yesterday might turn out to be whatever is causing the problem here too. Though it's happening for version 10.0.0 apparently here as well. 😬

[arikfr]

@galenspikes does this query have parameters or just a regular scheduled query? Anything else that might stand out with the queries that aren't refreshing?