Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ndarray-linalg as unused dependency breaks build #1356

Open
grothesque opened this issue Feb 3, 2024 · 2 comments
Open

Adding ndarray-linalg as unused dependency breaks build #1356

grothesque opened this issue Feb 3, 2024 · 2 comments
Labels

Comments

@grothesque
Copy link

The following program builds and runs with cargo run. This is on current Debian stable with current Rust from rustup. The project was started from a fresh cargo new.

src/main.rs:

use ndarray as nd;

fn main() {
    let a = nd::arr2(&[[1.0, 2.0], [3.0, 4.0]]);
    let b = a.dot(&a);
    dbg!(b);
}

Cargo.toml:

[package]
name = "matmul"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
ndarray = "0.15.6"

However, when the following dependency line is added to Cargo.toml

ndarray-linalg = { version = "0.16.0", features = ["openblas-system"] }

cargo build fails with the following linker error:

error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/cwg/.cargo/bin:/home/cwg/local/bin:/home/cwg/sys/bin:/home/cwg/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games" VSLANG="1033" "cc" "-m64" "/tmp/rustcvTd5ry/symbols.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.12rijkjv0x41cjbz.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.13zpo2a8nbmhezo6.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.18mwf38722fgdy2r.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1aulkdo8yfo6ihe3.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1ecocs5yuozxilmd.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1f7nyf4sn87xzaeq.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1fscnlrszql1dx22.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1g9982xymiazplzt.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1gn7rlw3e0d00jy7.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1h75op7sa9k7iynp.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1htow7tq8fl054u1.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1mpbuhv5lz3hq8vc.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1pq7vses7fawrpw5.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1q2dfthqogtb4973.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1rei7w7vfghwcm06.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1s0cev0krhm1nhli.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1v4mntwzpnbzlj7e.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1x8dahilfep9z896.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.1z9wqh15t745i2lu.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2476p408ijlio2aw.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.25zmx5z4ite7p93j.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.26zmyc0tqt5w8kp.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.295wsp3neqca3vq1.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2ehfwqz7fchblmoj.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2gj3qnetq34b5t25.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2llxqdmvpmyaw5tu.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2pr2xhkd4kum3ou0.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2y0vmyequpityodr.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2yfjbf68nlk12tkk.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2yi0z30et38wrvz3.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.2yj1k6s5zr8tke29.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.323o19iavd61zegj.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.33diw2qg26c5h3fa.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3bfbbia7767bgi9q.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3chmf4xfctqyqoxz.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3cxwcdlcjhvofqjw.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3er1zjezmko0ldc6.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3iwgrdt7o7qs655c.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3ojuwlzkujzr5ofk.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.42vkbpy0wns7q6b6.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.43sivngp0qeg9fo2.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.47fffcu23i8420ji.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.4lpp2rihs6fjz7uk.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.4ooads6hzvl3x9my.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.4tezgwp05waflge3.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.4tg3ihci1x6vfruf.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.4z4566ttclw6xyvv.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.52uo2hu8276a261m.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.55ukw19rwbfqk3ff.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.587cg4f0hsp9jvt7.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.5e71hbpcqf1hqyrd.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.785rbqjn06o4vau.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.9cssaprb5odwrpa.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.d6j127m8bb6cvpe.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.dnksvzmb461opu9.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.ft41rhofy1okclq.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.g0qqcxss02unjop.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.jo240miv0pmekn6.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.m6kcwgjgcso7rkw.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.qhkcypped3s2239.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.qkck8ljwzx8hq7p.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.rrl7vpmrjcugxxp.rcgu.o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3bx3kcxg016rjf94.rcgu.o" "-Wl,--as-needed" "-L" "/tmp/matmul/target/debug/deps" "-L" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/tmp/matmul/target/debug/deps/libndarray-023ca636ca46b8eb.rlib" "/tmp/matmul/target/debug/deps/libmatrixmultiply-c16e1aba180dc4e2.rlib" "/tmp/matmul/target/debug/deps/libapprox-8319bcbed4fd7e02.rlib" "/tmp/matmul/target/debug/deps/libnum_complex-65da99b25716bc26.rlib" "/tmp/matmul/target/debug/deps/libserde-1c762bfd80303691.rlib" "/tmp/matmul/target/debug/deps/librand-4959d6cec0f8d469.rlib" "/tmp/matmul/target/debug/deps/librand_chacha-b22197ade9f74797.rlib" "/tmp/matmul/target/debug/deps/libppv_lite86-9f808eb911999ef2.rlib" "/tmp/matmul/target/debug/deps/librand_core-fe980624cf1fe49e.rlib" "/tmp/matmul/target/debug/deps/libgetrandom-70ffb9e79a3d1149.rlib" "/tmp/matmul/target/debug/deps/libcfg_if-61dc7a6799e72b18.rlib" "/tmp/matmul/target/debug/deps/libnum_integer-6761fe06578db2d2.rlib" "/tmp/matmul/target/debug/deps/libnum_traits-aa75f3964f70d2b4.rlib" "/tmp/matmul/target/debug/deps/librawpointer-fd99f7762ca119b4.rlib" "/tmp/matmul/target/debug/deps/libcblas_sys-33b52a1f1355ee0d.rlib" "/tmp/matmul/target/debug/deps/liblibc-a9fa19dd18f65bfc.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-90f6ddbf82de36ec.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-480c41e9d4e1f677.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-db744c0ca03eed1d.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-8ba652dc7d4b285a.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-d5f4520a9ebc0d58.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-4ad84de58f0cb463.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0d10aae2e0f38735.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-0c17cff739e6745b.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-b29e17139dde1aa8.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-1f975299829cc7bd.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-86bae3bc3079f89b.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-321b82c30dffdf5f.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-3dd8a6810a0bdfef.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-eb39a61c0c879984.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-aa769569f91c3548.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-510a192a50a983ed.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-0577018320f99037.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-193cf992125ccd4c.rlib" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-8e138eaf26ebb4a8.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/cwg/cb/autostow/rust/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: /usr/bin/ld: /tmp/matmul/target/debug/deps/matmul-df71ff3b22c784a0.3bfbbia7767bgi9q.rcgu.o: in function `ndarray::linalg::impl_linalg::mat_mul_impl':
          /home/cwg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/linalg/impl_linalg.rs:484: undefined reference to `cblas_zgemm'
          /usr/bin/ld: /home/cwg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/linalg/impl_linalg.rs:483: undefined reference to `cblas_cgemm'
          /usr/bin/ld: /home/cwg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/linalg/impl_linalg.rs:481: undefined reference to `cblas_dgemm'
          /usr/bin/ld: /home/cwg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/linalg/impl_linalg.rs:480: undefined reference to `cblas_sgemm'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `matmul` (bin "matmul") due to previous error

This problem disappears when the following use declaration is added to the program

use ndarray_linalg;

(However rustc warns about an "unused import"...)

I reckon that ndarray and ndarray-linalg have a tricky job here dealing with different sources of BLAS/LAPACK and a fallback implementation of matrix multiplication provided by the matrixmultiply crate. Still, it seems to me that the problem described above should be addressed in some way, at least by improving documentation.

It is very surprising when a program stops working after an unused dependency is added, and then can be made to build again by adding a needless use declaration.

@nilgoyette nilgoyette added the bug label Feb 23, 2024
@bluss
Copy link
Member

bluss commented Mar 9, 2024

It's the same problem as the one described here https://github.com/rust-ndarray/ndarray?tab=readme-ov-file#how-to-enable-blas-integration in the Readme - at least I think it is. It says "your program must also link to blas_src by using it".

This is not great, but that's ndarray's design for blas right now to enable loose coupling between ndarray and blas versions. So we have some tangential docs. It happens that using ndarray_linalg is interchangeable with using blas_src.

Not sure how to improve this, probably add back a fixed requirement on blas src version and so on?

@grothesque
Copy link
Author

grothesque commented Apr 16, 2024

Thanks for the reply. I do not yet understand the problem completely, so I'm unable to suggest a fix at this time.

I spent some time trying to understand what's going on. Starting from the toy project described in the topmost post, I can obtain the same error message by replacing the following chunk in Cargo.toml

[dependencies]
ndarray = "0.15.6"

by

[dependencies.ndarray]
default-features = false
features = ["blas", "approx", "std"]
version = "0.15.2"

This latter chunk is taken directly from Cargo.toml of ndarray-linalg.

It is indeed the "blas" feature of ndarray that triggers the problem.

Now looking, as you suggest, at the readme file of ndarray (I note in passing that the subsection title "How to enable blas integration" does not render correctly on github), it suggests to use

[dependencies]
ndarray = { version = "0.15.0", features = ["blas"] }
blas-src = { version = "0.8", features = ["openblas"] }
openblas-src = { version = "0.10", features = ["cblas", "system"] }

for system openblas.

However, this does not work on my box (neither as the only dependencies, or together with

ndarray-linalg = { version = "0.16.0", features = ["openblas-system"] }

So, disregarding ndarray-linalg for a moment, is the suggestion in the readme file of ndarray on how to use system openblas out of date, or am I missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants