NIX_CC_WRAPPER_TARGET_HOST_${stdenv.cc.suffixSalt} is not set when setting languages.c.enable = true

[kloenk]

Describe the bug
NIX_CC_WRAPPER_TARGET_HOST_${stdenv.cc.suffixSalt} should be set to 1 so the gcc/clang wrapper picked up NIX_CLFAGS_COMPILE/NIX_LDFLAGS. Without this, I noticed that the wrapper does not accept libraries provided in those environment paths.

To Reproduce

Version
devenv: 0.5

[domenkozar]

Where is this used exactly?

❯ git grep NIX_CC_WRAPPER_TARGET_HOST_
pkgs/applications/audio/faust/faust2.nix:          # e.g. NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu
pkgs/build-support/build-fhs-userenv-bubblewrap/env.nix:    export NIX_CC_WRAPPER_TARGET_HOST_${stdenv.cc.suffixSalt}=1
pkgs/build-support/build-fhs-userenv/env.nix:    export NIX_CC_WRAPPER_TARGET_HOST_${stdenv.cc.suffixSalt}=1
pkgs/development/idris-modules/idris-wrapper.nix:      --set 'NIX_CC_WRAPPER_TARGET_HOST_${stdenv.cc.suffixSalt}' 1 \

[kloenk]

The last time I played with the clang wrapper, it was required. But not sure, maybe that changed

[Mic92]

It's set by the cc wrapper to prevent including flags twice in forked processes.
see build-support/cc-wrapper/add-flags.sh. It's actually the opposite, if you set this value than NIX_LDFLAGS and co gets ignored.

[Mic92]

However with the introduction of nakedshell for devenv is pretty much broken as it was in devshell before, because there are no setup hooks that will setup NIX_LDFLAGS or NIX_CFLAGS_COMPILE in the first place correctly.

[thenonameguy]

This is fair criticism, but it ties IMO too much of the expected behaviour of devenv to the nix-shell one.

We have two main options here:

1. (preferred) encapsulate building of your softwate in a nix build expression. This way the setup hooks work and your C compiler works transitively.
   You get reproducible builds as a plus.

2. Patch the 'devenv shell' time build of your software to work. This is what your issue description aimed at. This requires a minimal repro case and what tooling specific configuration is required to get it to work (without nix specifics)

[Mic92]

For context I described some strategies how C/C++ could be integrated, if you prefer to not use hooks: numtide/devshell#104 and here numtide/devshell#103

For my part I cannot use devenv at the moment because the experimental nix fork used in devenv is breaking and it does not work with ~/.config/nixpkgs/config.nix being a symlink.

[domenkozar]

For my part I cannot use devenv at the moment because the experimental nix fork used in devenv is breaking and it does not work with ~/.config/nixpkgs/config.nix being a symlink.

How does it break?

[Mic92]

@domenkozar opened #260

[kai-tub]

Hey, sorry for potentially adding noise to the discussion, but I am unsure what I should take away from the thread.
As far as I can tell, the thread extended to the question if and how devenv should work with gcc/clang wrappers to provide some help when linking.

If I get the point of @Mic92 correctly:

However with the introduction of nakedshell for devenv is pretty much broken as it was in devshell before, because there are no setup hooks that will setup NIX_LDFLAGS or NIX_CFLAGS_COMPILE in the first place correctly.

devenv is not designed to provide an environment where the shared libraries are provided/patched for the development of packages that rely on those, correct?

@thenonameguy disagrees but I have trouble wrapping my head around the answer (especially because I am a C/C++ gcc/clang noob). According to the first part:

1. (preferred) encapsulate building of your softwate in a nix build expression. This way the setup hooks work and your C compiler works transitively. You get reproducible builds as a plus.

I can see how it is desirable to build the software as a nix build expression, but what if I am developing, for example, a rust package that relies on bindgen and requires those C compiler flags to be set to build the package?
If I am actively working on it, I want to be able to use rust-analyzer for example, that continuously tries to run the build instructions to provide inline error (lsp) messages.
So targeting a 'static' build expression won't help me during the development of the package.
This leads to the second option:

2. Patch the 'devenv shell' time build of your software to work. This is what your issue description aimed at. This requires a minimal repro case and what tooling specific configuration is required to get it to work (without nix specifics)

I guess this is the part I am trying to wrap my head around as I have trouble finding any resources where to start.
Should I ask look for support from the 'bigger' nix community and/or the language specific nix communities (rust in this case)?

Concrete example

The following is potentially outside of the scope of the current issue, but I wanted to provide a real, reproducible example for better context.

I would like to work on a rust package that includes georust/gdal which provides bindings for gdal. GDAL is known for not being easy to compile.
Or to make it a bit more approachable, let's start with georust/proj that provides bindings for proj.

proj has been packages in nixpkgs, so I would assume/hope that it should be fairly easy to generate the bindings for it, but I haven't been able to get that working.

I know that it relies on pkg-config and that the output of the proj-V-dev/lib/pkgconfig/proj.pc lists the static libraries it requires. In this case, the output is:

prefix=/nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1
libdir=/nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib
includedir=/nix/store/y73zc288npjq5irbxfa4cvs8vlng4m07-proj-9.1.1-dev/include
datarootdir=${prefix}/share
datadir=${datarootdir}/proj

Name: PROJ
Description: Coordinate transformation software library
Requires:
Version: 9.1.1
Libs: -L${libdir} -lproj
Libs.private: -lstdc++ -lm -ldl
Requires.private: sqlite3 libtiff-4 libcurl
Cflags: -I${includedir}

but when I try to compile it with caro build, I get:

error: linking with `cc` failed: exit status: 1
[...]
= note: /nix/store/r2b9k28c6aghczpqfvh71y9xavm7rr68-binutils-2.39/bin/ld: warning: libsqlite3.so.0, needed by /nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib/libproj.so, not found (try using -rpath or -rpath-link)
          /nix/store/r2b9k28c6aghczpqfvh71y9xavm7rr68-binutils-2.39/bin/ld: warning: libtiff.so.6, needed by /nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib/libproj.so, not found (try using -rpath or -rpath-link)
          /nix/store/r2b9k28c6aghczpqfvh71y9xavm7rr68-binutils-2.39/bin/ld: warning: libcurl.so.4, needed by /nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib/libproj.so, not found (try using -rpath or -rpath-link)
          /nix/store/r2b9k28c6aghczpqfvh71y9xavm7rr68-binutils-2.39/bin/ld: warning: libstdc++.so.6, needed by /nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib/libproj.so, not found (try using -rpath or -rpath-link)
          /nix/store/r2b9k28c6aghczpqfvh71y9xavm7rr68-binutils-2.39/bin/ld: /nix/store/ck9ljp9x3w13jiqqsmpdc4rmj3q41ck1-proj-9.1.1/lib/libproj.so: undefined reference to `typeinfo for std::invalid_argument@GLIBCXX_3.4'
[...]

which suggests that the libraries from Libc.private aren't added to the compiler's path.
But at this point, it feels like I am stabbing in the dark and have no idea where I should go from here.
I guess my next step would be to move away from devenv and trying to build the package as a nix build expression?

I hope this wasn't too confusing and all over the place...
My goal was to share my experience as a beginner that is trying to disentangle all the different layers and how difficult it is to understand where to ask what questions. 😅

Here is the environment if it helps: https://github.com/kai-tub/devenv-proj

[Mic92]

You might be able to link libraries and header files using the current devenv implementation but than the linker won't add libraries to the RPATH of a binary because devenv does not bring any support for that for the reasons mentioned above. In the current state you are better off not using devenv for these types of projects and use mkShell instead and wait until support in devenv gets added. I have a here a tutorial that explains most of the mechanics of how Nix deals with compilers: https://nixos.wiki/wiki/C

[domenkozar]

Can you test #745