Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adios2-config cxxflags output missing "-I" or "-isystem" #4350

Open
giltirn opened this issue Sep 18, 2024 · 1 comment
Open

adios2-config cxxflags output missing "-I" or "-isystem" #4350

giltirn opened this issue Sep 18, 2024 · 1 comment

Comments

@giltirn
Copy link

giltirn commented Sep 18, 2024 •
edited
Loading

Describe the bug
I built adios2@2.10.0 via spack on Sunspot using the icpx compiler and found adios2-config does not include the "-I" or "-isystem" prefix to the include path:


ckelly@uan-0002:~/bld/chimbuko_spack_env_9_12_24> /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/adios2-2.10.0-cj255dyxpy3joolxnthsq3rlwqrfavfc/bin/adios2-config --cxx-flags
-DADIOS2_USE_MPI /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/adios2-2.10.0-cj255dyxpy3joolxnthsq3rlwqrfavfc/include

A build with gcc and openmpi in a Docker test environment running on a local machine produces the following


root@fe0859b75c2c:~# /opt/software/linux-ubuntu18.04-broadwell/gcc-7.5.0/adios2-2.10.0-tqtfqjay6ikguvo2bckeoykz7f3i3gfn/bin/adios2-config --cxx-flags
-DADIOS2_USE_MPI -isystem /opt/software/linux-ubuntu18.04-broadwell/gcc-7.5.0/adios2-2.10.0-tqtfqjay6ikguvo2bckeoykz7f3i3gfn/include

which has the "-isystem" prefix.

This missing prefix is preventing libraries using adios2 from being built with spack.

To Reproduce

Here is a complete spack environment:


spack:
  specs:
  - adios2 % oneapi
  concretizer:
      unify: true
      reuse: false
  compilers:
  - compiler:
      spec: gcc@=12.2.0
      paths:
        cc: /opt/aurora/24.086.0/spack/gcc/0.7.0/install/linux-sles15-x86_64/gcc-7.5.0/gcc-12.2.0-fxd4zwa64mfswv34rol52zamj7c6r3cz/bin/gcc
        cxx: /opt/aurora/24.086.0/spack/gcc/0.7.0/install/linux-sles15-x86_64/gcc-7.5.0/gcc-12.2.0-fxd4zwa64mfswv34rol52zamj7c6r3cz/bin/g++
        f77: /opt/aurora/24.086.0/spack/gcc/0.7.0/install/linux-sles15-x86_64/gcc-7.5.0/gcc-12.2.0-fxd4zwa64mfswv34rol52zamj7c6r3cz/bin/gfortran
        fc: /opt/aurora/24.086.0/spack/gcc/0.7.0/install/linux-sles15-x86_64/gcc-7.5.0/gcc-12.2.0-fxd4zwa64mfswv34rol52zamj7c6r3cz/bin/gfortran
      flags: {}
      operating_system: sles15
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: "oneapi@2024.04.15.002 ~envmods"
      paths:
        cc: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym/compiler/2024.04.15.002/bin/icx
        cxx: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym/compiler/2024.04.15.002/bin/icpx
        f77: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym/compiler/2024.04.15.002/bin/ifx
        fc: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym/compiler/2024.04.15.002/bin/ifx
      flags:
        cflags: -fp-model=precise
        cxxflags: -fp-model=precise
        fflags: -fp-model=precise
      operating_system: sles15
      target: x86_64
      modules:
        - intel_compute_runtime/release/821.36
        - oneapi/eng-compiler/2024.04.15.002
        - gcc/12.2.0
      environment: {}
      extra_rpaths: []

  repos: []
  packages:
    all:
      compiler: [oneapi]
      target: [x86_64]
      providers:
        mpi: [mpich]

    intel-oneapi-compilers:
      externals:
        - spec: intel-oneapi-compilers@2024.04.15.002 ~envmods %gcc@12.2.0
          prefix: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym
          modules:
            - oneapi/eng-compiler/2024.04.15.002
            - gcc/12.2.0
        - spec: intel-oneapi-compilers@2024.1 ~envmods %gcc@12.2.0
          prefix: /opt/aurora/24.086.0/spack/oneapi/0.7.0/oneapi_sym
          modules:
            - oneapi/release/2024.1
            - gcc/12.2.0
      buildable: false

    mpich:
      require:
        - spec: "@:20231026 %oneapi pmi=pmix ~hydra"
          when: "%oneapi@2024.04.15.002"
        - spec: "@:20231026 %oneapi pmi=pmix ~hydra"
          when: "%oneapi@2024.1"
      externals:
        - spec: "mpich@20231026 %oneapi pmi=pmix ~hydra"
          prefix: /opt/aurora/24.086.0/CNDA/mpich/20231026/mpich-ofi-all-icc-default-pmix-gpu-drop20231026
          modules:
            - mpich/icc-all-pmix-gpu/20231026
      buildable: false

    oneapi-level-zero:
      require:
        - spec: "@:821.36"
          when: "%oneapi@2024.04.15.002"
        - spec: "@:803.29"
          when: "%oneapi@2024.1"
      externals:
        - spec: "oneapi-level-zero@803.29"
          prefix: /opt/aurora/24.086.0/intel-gpu-umd/803.29
          modules:
            - intel_compute_runtime/release/803.29
        - spec: "oneapi-level-zero@803.45"
          prefix: /opt/aurora/24.086.0/intel-gpu-umd/803.45
          modules:
            - intel_compute_runtime/release/803.45
        - spec: "oneapi-level-zero@821.30"
          prefix: /opt/aurora/24.086.0/intel-gpu-umd/821.30
          modules:
            - intel_compute_runtime/release/821.30
        - spec: "oneapi-level-zero@821.35"
          prefix: /opt/aurora/24.086.0/intel-gpu-umd/821.35
          modules:
            - intel_compute_runtime/release/821.35
        - spec: "oneapi-level-zero@821.36"
          prefix: /opt/aurora/24.086.0/intel-gpu-umd/821.36
          modules:
            - intel_compute_runtime/release/821.36
      buildable: false

    libfabric:
      externals:
        - spec: libfabric@1.15.2.0
          prefix: /opt/cray/libfabric/1.15.2.0
      buildable: false

    mpi:
      buildable: false

    perl:
      externals:
      - spec: perl@5.26.1
        prefix: /usr
      buildable: false
      version: []

    python:
      version: []
      buildable: false
      externals:
      - modules:
        - python/3.10.11
        spec: python@3.10.11

    tar:
      externals:
      - spec: tar@1.34
        prefix: /usr
      buildable: false
      version: []

    autoconf:
      externals:
      - spec: autoconf@2.69
        prefix: /usr
      buildable: false
      version: []

    automake:
      externals:
      - spec: automake@1.15.1
        prefix: /usr
      buildable: false
      version: []

    openssl:
      externals:
      - spec: openssl@1.1.1l
        prefix: /usr
      buildable: false
      version: []

    pkg-config:
      externals:
      - spec: pkg-config@0.29.2
        prefix: /usr
      buildable: false
      version: []

    gmake:
      externals:
      - spec: gmake@4.2.1
        prefix: /usr
      buildable: false
      version: []

    libtool:
      externals:
      - spec: libtool@2.4.6
        prefix: /usr
      buildable: false
      version: []

    m4:
      externals:
      - spec: m4@1.4.18
        prefix: /usr
      buildable: false
      version: []

    boost:
      externals:
      - spec: boost@1.84.0
        prefix: /opt/aurora/24.086.0/spack/oneapi/0.7.0/install/2024.04.15.002/linux-sles15-x86_64/oneapi-2024.04.15.002/boost-1.84.0-kgmlpiv7w2oba3dge2zl7rtotjuorgkr
        modules:
         - boost/1.84.0
      buildable: false
      version: []

    cmake:
      externals:
      - spec: cmake@3.27.7
        prefix: /opt/aurora/24.086.0/spack/gcc/0.7.0/install/linux-sles15-x86_64/gcc-12.2.0/cmake-3.27.7-mbl7dvgbiblpavhu53h5cheyrmpaikdz
        modules:
         - cmake/3.27.7

      buildable: false
      version: []

  view: true

Please note that the adios2 binaries will fail to link due to the following issue:


  >> 1142    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Abort'
  >> 1143    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Bcast'
  >> 1144    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Barrier'

suggesting one of the dependent libraries on this system includes MPI symbols for some reason. A workaround is to hack the spack package (var/spack/repos/builtin/packages/adios2/package.py) to include the following
args.append("-DCMAKE_EXE_LINKER_FLAGS=-Wl,-rpath=/opt/aurora/24.086.0/CNDA/mpich/20231026/mpich-ofi-all-icc-default-pmix-gpu-drop20231026/lib/ -L/opt/aurora/24.086.0/CNDA/mpich/20231026/mpich-ofi-all-icc-default-pmix-gpu-drop20231026/lib/ -lmpi")
when putting together the cmake args.

Expected behavior
The output of "adios2-config --cxx-flags" should include "-I" or "-isystem" before the include path

Desktop:

  • OS/Platform: Sunspot
  • Build: compiler oneapi@2024.04.15.002 cmake cmake@3.27.7 built via spack with spec adios2@2.10.0%oneapi@2024.04.15.002 cflags='-fp-model=precise' cxxflags='-fp-model=precise' fflags='-fp-model=precise' ~aws+blosc2+bzip2~campaign~cuda~dataspaces+fortran~hdf5~ipo~kokkos+libcatalyst~libpressio+mgard+mpi~pic+png~python~rocm+sst~sycl+sz+zfp build_system=cmake build_type=Release generator=make arch=linux-sles15-x86_64
@giltirn
Copy link
Author

giltirn commented Sep 18, 2024

Update:
I tried compiling the spec adios2@2.9.2 % oneapi on Sunspot instead, using the otherwise same environment as above, and found that it compiled successfully without the MPI link errors and also produces an adios2-config that gives the expected output

ckelly@uan-0002:~/bld/chimbuko_spack_env_9_12_24/adios2_env_vers_test> /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/adios2-2.9.2-kiiwf3oo2qouajt4lpfrqevwuxb2xxsx/bin/adios2-config --cxx-flags
-DADIOS2_USE_MPI -isystem /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/adios2-2.9.2-kiiwf3oo2qouajt4lpfrqevwuxb2xxsx/include

This suggests that both issues (possibly related, although I can't see how they would be) stem from changes between 2.9.2 and 2.10.0.

For more information regarding the MPI link issue, the build of 2.10.0 fails with


     1139    [ 75%] Linking CXX executable ../../../../bin/adios2_remote_server
     1140    cd /tmp/ckelly/spack-stage/spack-stage-adios2-2.10.0-rkkwyaqfyxywyk55hmmr7ow625kcbfje/spack-build-rkkwyaq/source/adios2/toolkit/remote && /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack
             /linux-sles15-x86_64/oneapi-2024.04.15.002/cmake-3.27.9-34sy6yon6urhi5e6szgdlk7kcvjlakgr/bin/cmake -E cmake_link_script CMakeFiles/adios2_remote_server.dir/link.txt --verbose=1
     1141    /home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/lib/spack/env/oneapi/icpx -O3 -DNDEBUG CMakeFiles/adios2_remote_server.dir/remote_server.cpp.o CMakeFiles/adios2_remote_server.dir/remote_common.cp
             p.o -o ../../../../bin/adios2_remote_server  -Wl,-rpath,/tmp/ckelly/spack-stage/spack-stage-adios2-2.10.0-rkkwyaqfyxywyk55hmmr7ow625kcbfje/spack-build-rkkwyaq/lib64:/home/ckelly/bld/chimbuko_spack_
             env_9_12_24/spack/opt/spack/linux-sles15-x86_64/gcc-12.2.0/c-blosc2-2.11.1-r4lknhue5cmioynluyak4dupnoqjpfxn/lib64:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/one
             api-2024.04.15.002/zstd-1.5.6-rjqalogquhy65go25uuaolps5e4npyaj/lib:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/gcc-12.2.0/lz4-1.9.4-jeh3svnxgjqkufwqc6zosl7mo5trs
             5yb/lib:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/zlib-ng-2.1.6-2sjoqerebjgnkrjzjleemexeuha6mswe/lib:/home/ckelly/bld/chimbuko_spack_env_
             9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/zfp-0.5.5-5n2irrjgchglgzqjaoidvapjftko4fhz/lib64:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::: ../../../../lib64/libadios2_evpath.so.2.10.0 ../../../../lib64/libadios2_core.so.2.10.0 ../../../../lib64/libadios2_ffs.so.2.10.0 ../../../../li
             b64/libadios2_atl.so.2.10.0 -pthread -ldl -Wl,-rpath-link,/tmp/ckelly/spack-stage/spack-stage-adios2-2.10.0-rkkwyaqfyxywyk55hmmr7ow625kcbfje/spack-build-rkkwyaq/lib64:/home/ckelly/bld/chimbuko_spac
             k_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/gcc-12.2.0/c-blosc2-2.11.1-r4lknhue5cmioynluyak4dupnoqjpfxn/lib64:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/o
             neapi-2024.04.15.002/zstd-1.5.6-rjqalogquhy65go25uuaolps5e4npyaj/lib:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/gcc-12.2.0/lz4-1.9.4-jeh3svnxgjqkufwqc6zosl7mo5t
             rs5yb/lib:/home/ckelly/bld/chimbuko_spack_env_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/zlib-ng-2.1.6-2sjoqerebjgnkrjzjleemexeuha6mswe/lib:/home/ckelly/bld/chimbuko_spack_en
             v_9_12_24/spack/opt/spack/linux-sles15-x86_64/oneapi-2024.04.15.002/zfp-0.5.5-5n2irrjgchglgzqjaoidvapjftko4fhz/lib64
  >> 1142    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Abort'
  >> 1143    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Bcast'
  >> 1144    /usr/bin/ld: ../../../../lib64/libadios2_core.so.2.10.0: undefined reference to `MPI_Barrier'
  >> 1145    icpx: error: linker command failed with exit code 1 (use -v to see invocation)
  >> 1146    make[2]: *** [source/adios2/toolkit/remote/CMakeFiles/adios2_remote_server.dir/build.make:120: bin/adios2_remote_server] Error 1
     1147    make[2]: Leaving directory '/tmp/ckelly/spack-stage/spack-stage-adios2-2.10.0-rkkwyaqfyxywyk55hmmr7ow625kcbfje/spack-build-rkkwyaq'
  >> 1148    make[1]: *** [CMakeFiles/Makefile2:1257: source/adios2/toolkit/remote/CMakeFiles/adios2_remote_server.dir/all] Error 2
     1149    make[1]: Leaving directory '/tmp/ckelly/spack-stage/spack-stage-adios2-2.10.0-rkkwyaqfyxywyk55hmmr7ow625kcbfje/spack-build-rkkwyaq'
  >> 1150    make: *** [Makefile:139: all] Error 2

My first thought was that this issue would affect every binary but it may also be just adios2_remote_server as I notice that this application did not exist in 2.9.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@giltirn