Where could we run nixpkgs-swh?

[nlewo]

Until recently, the nixpkgs-swh script was executed every day by a buildkite agent. However, this agent no longer exists.

Would it be possible to run a daily systemd cron jobs?
Since it evaluates the whole nixpkgs repository, it takes a bit of time (about 20min on my laptop i guess).
Moreover, we would need to expose the generated JSON file. We would then have to spawn a nginx server to serve it. Note this file doesn't strictly need to be persisted since it is generated each day.

Note that using hydra.nix-community.org seems to be difficult because the script calls hydra.nixos.org to get latest evaluated nixpkgs commits.

WDYT?

[zowoq]

Would it be possible to run a daily systemd cron jobs?

Sounds okay.

Moreover, we would need to expose the generated JSON file. We would then have to spawn a nginx server to serve it. Note this file doesn't strictly need to be persisted since it is generated each day.

Skimming though the scripts in https://github.com/nix-community/nixpkgs-swh, looks like could also serve the analysis from nginx and drop the github pages publish steps?

[nlewo]

Sounds okay.

Cool! I will start working on it then.

Skimming though the scripts in https://github.com/nix-community/nixpkgs-swh, looks like could also serve the analysis from nginx and drop the github pages publish steps?

Yes, that was included in my plan :)

[zowoq]

Just curious, is this going to be running here indefinitely?

[nlewo]

@zowoq yes, i don't have any other plan (and i don't have too much time to spend on this project atm for a bigger move).

[zowoq]

@nlewo The nixos org infra team was recently split and expanded to better handle running this sort of thing, I'd suggest opening an issue in nixos-org-configurations to see if there is any interest in running it on official infra, might also find some people willing to work on implementing it given the recent s3 cache problems.

NixOS/foundation#79 (comment)
Edit: actually, the more significant problem that I see is that people tend to host things left and right, and then the NixOS project starts becoming dependent on those single-deployer services. Eg: I use https://nixpk.gs/ quite a bit. In order to combat this, the best is to have a common infrastructure where it's relatively easy to deploy onto.

cc @zimbatm

[zimbatm]

It's going to take a while for the infra team to scale up. Most of the focus will probably be on the core for a while.

If you're up for hosting the service here that's fine.