Compiling on JUWELS with quda

The following instruction refers to configure quda and the quda_work branch in tmlqcd on JUWELS.

The following modules list has been loaded for compilation (dated 05.03.2019)

module load \
GCCcore/.7.3.0 \
Intel/2019.0.117-GCC-7.3.0 \
ParaStationMPI/5.2.1-1 \
imkl/2019.0.117 \
intel-para/2018b \
flex/2.6.4 \
CUDA/9.2.88

First we configure and compile the devel branch of lattice/quda.

We have used the 4ed1c90defd551574a7b7f0426532cc86b86e006 commit from the develop branch. The source code can be found: /p/project/chbn28/hbn28d/code/quda We have used the following configuration script: cmake \
-DCMAKE_INSTALL_PREFIX="${PROJECT_chbn28}/hbn28d/libs/quda_igcc7.3.0_parastation" \
-DQUDA_BUILD_ALL_TESTS=OFF \
-DQUDA_GPU_ARCH=sm_70 \
-DQUDA_INTERFACE_QDP=ON \
-DQUDA_INTERFACE_MILC=OFF \
-DQUDA_MPI=ON \
-DQUDA_DIRAC_WILSON=ON \
-DQUDA_DIRAC_TWISTED_MASS=ON \
-DQUDA_DIRAC_TWISTED_CLOVER=ON \
-DQUDA_DIRAC_NDEG_TWISTED_MASS=ON \
-DQUDA_DIRAC_CLOVER=ON \
-DQUDA_DYNAMIC_CLOVER=ON \
-DQUDA_DIRAC_DOMAIN_WALL=OFF \
-DQUDA_MULTIGRID=ON \
-DQUDA_DIRAC_STAGGERED=OFF /p/scratch/chbn28/hbn28d/code/quda

For the tmlqcd side we have used the master branch of etmc-tmlqcd. We have used the commit 8e5554578d2bd5ee3b02a185976b9e6656d04c4c. The code can be found in /p/project/chbn28/hbn28d/build/etmc_tmlqcd_master

We build the tmlqcd library using the following configure file

MKLROOT=/gpfs/software/juwels/stages/2018b/software/imkl/2019.0.117-ipsmpi-2018b \
CC=mpicc CXX=mpic++ LD=/gpfs/software/juwels/stages/2018b/software/psmpi/5.2.1-1-iccifort-2019.0.117-GCC-7.3.0/bin/mpic++ \
CFLAGS="-O3 -I${MKLROOT}/include " \
CXXFLAGS="-O3 -std=c++11" \
LDFLAGS="-L/gpfs/software/juwels/stages/2018b/software/psmpi/5.2.1-1-iccifort-2019.0.117-GCC-7.3.0/lib/" \
/p/home/jusers/pittler1/juwels/code/etmc_master/configure \
--with-lemondir=/p/project/chbn28/hbn28d/build/lemon \
--with-limedir=/p/home/jusers/pittler1/juwels/build/lime \
--enable-mpi --with-mpidimension=4 --enable-omp \
--disable-halfspinor --enable-gaugecopy \
--enable-alignment=64 \
--disable-sse2 --disable-sse3 \
--with-cudadir="/gpfs/software/juwels/stages/2018b/software/CUDA/9.2.88/" \
--with-qudadir="/p/project/chbn28/hbn28d/build/quda_igcc7.3.0_parastation" \
--with-lapack="-Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -liomp5 -lpthread -lm -ldl"