ccache & stockfish builds

[amotmot]

Recently, fishnet was updated such that Stockfish and Fairy-Stockfish are build locally. Thus omitting the need to build externally (for each supported platform), package and making them available to fishnet code. With the change the build times for fishnet have increased.

1. What possible ways could we reduce build times? For example, could ccache[1] be introduced?

2. Could the SF and Fairy-SF benchmark step be removed?

[fishnet 2.3.4-dev] Step 2/4. Running benchmark for pgo-build ...
[...]
[fishnet 2.3.4-dev] Position: 47/47 (bb1n1rkr/ppp1Q1pp/3n1p2/3p4/3P4/6Pq/PPP1PP1P/BB1NNRKR w HFhf - 0 5)
[fishnet 2.3.4-dev]
[fishnet 2.3.4-dev] ===========================
[fishnet 2.3.4-dev] Total time (ms) : 9502
[fishnet 2.3.4-dev] Nodes searched  : 4039197
[fishnet 2.3.4-dev] Nodes/second    : 425089

3. Also, I've noticed that when building on x86-64 architecture multiple executables are build? The following was observed seven times ...

[fishnet 2.3.4-dev] Step 1/4. Building instrumented executable ...
[...]
[fishnet 2.3.4-dev] Step 2/4. Running benchmark for pgo-build ...
[...]
[fishnet 2.3.4-dev] Step 3/4. Building optimized executable ...
[...]
[fishnet 2.3.4-dev] Step 4/4. Deleting profile data ...

[1] https://lib.rs/crates/ccache

[niklasf]

Some options:

• The fishnet binary still bundles Stockfish and Fairy-Stockfish for various x86-64 CPUs (bmi2, avx2, sse41-popcnt, and generic x86), so that users downloading the release do not have to figure out the best fit. That means each of the engines is built 4 times. This could potentially be reduced for non-release builds.
• Both engines are built with profile-guided optimization. The benchmark output you see is part of that. Skipping pgo at the cost of some performance for non-release builds could be an option.
• Ccache (the compiler cache, not the crate you linked) should already work, if it is installed on the build system.

I guess it depends in which context you want to reduce the build times. For development? To use fishnet after building it from source?

[amotmot]

1. When I build locally and afterwards fishnet initializes which of 4 SF builds is used in traditional chess game analysis? In other words, are there steps fishnet users need to take to select the ideal SF build for their CPU?

2. I assumed modification of build.rs would have been required to inform compiler cache of what artifacts to cache. The GitHub CI build times could be reduced through https://github.com/marketplace/actions/ccache-for-gh-actions

[niklasf]

1. It automatically selects the best of the 4 Stockfish and the best of the 4 Fairy-Stockfish builds, but the build process builds and bundles them all.
2. It looks like ccache can be used as a drop-in replacement for g++ that handles the caching transparently. The GitHub action you linked might need some changes to build.rs to respect the environment variables.