Python submodule requirements

[FHusko]

I think it's not clear what combination of required submodules (swiftpipeline, velociraptor, scipy) is required to successfully run the pipeline.

The ones listed in requirements.txt appear to be out of date. I tried instead using the latest for all three submodules (swiftpipeline==0.3.5, velociraptor==0.16.1, scipy==0.13.1), but there is a conflict that pops up:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
numba 0.57.0 requires numpy<1.25,>=1.21, but you have numpy 2.0.0 which is incompatible.

This appears to get resolved if I instead install an older version of scipy, e.g. scipy==0.8.0).

[james-trayford]

I think numpy 2.0.0 release is new and breaks a lot of packages, so specifying a 1.X.X version of numpy might make sense for now? see https://numpy.org/devdocs/numpy_2_0_migration_guide.html

[FHusko]

Indeed. I guess numpy gets installed to such a late version through scipy. But even if we avoid installing numpy 2.0.0 by e.g. installing scipy 1.10.0, something like the above error still pops up because numba 0.57.0 seems to want numpy<1.25 (which is only satisfied with scipy 1.8.0).