-
Notifications
You must be signed in to change notification settings - Fork 2
/
README.howto
112 lines (62 loc) · 4.1 KB
/
README.howto
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
================================
How to compile GENESIS on Fugaku
================================
1. Generatl compile procedure
To run GENESIS on Fugaku, plase go to the main directory and do the following procedures:
autoreconf
./configure --host=Fugaku --enable-single
make
make install
Then, you can use binary "spdyn" in src/bin directory to execute GENESIS on Fugaku.
2. Compile options
You can assign several options when compiling GENESIS.
1) Compile with single precision
./configure --enable-single --host=Fugaku
It should be noted that double precision is used in constraints and energy evaluations even with "--enable-single" to keep accuracy
2) Compile with mixed precision (integration with double precision and force/calculation with single precision)
./configure --enable-mixed --host=Fugaku
3) Compile with precise timers for each component (single precision case)
./configure --enable-single --enable-pktimer --host=Fugaku
When using timers, you can add the following options:
--with-fj_timer_detail (timer for every components)
--with-fapp (generate profiler for operation count)
** It should be noted that GENESIS works on almost CPUs as well as Fugaku. When you compile on the intel machine, we recommend to write Fortran and C compiler explicitly by defining FC and CC. Herebelow we wrote one example by assuming that MPI intel Fortran/C compilers are mpiifort and mpiicc:
./configure --enable-single FC=mpiifort CC=mpiicc
** In the case of cross compiler, please write "--host=XX" like Fugaku case (you can write anything in XX). Further compiler coptions can be understood by reading configure file.
============================
How to submit jobs on Fugaku
============================
1. Fugaku job script is in run/scripts directory. From the script files, you can identify the number of nodes/MPI/OpenMP threads.
2. If you prepare the script file (here we assumed that it is XX.sh), you can execute it on Fugaku by typing
pjsub XX.sh
3. You can understand your submiited job condition by typing
pjstat
======================
Contents of saved data
======================
1. 1.5Mx_systems: Conventional MD results for 1.5M system and its duplications
1) input: initial strucutre file and parameter/topology files
2) Except input directory, we generate the directory according to the Figure/Table numbers
2. gREST_105.1M: gREST benchmark results
1) input: initial structure and parameter/topology files
2) Operation_Number: Operation count calculation
3) Except input and Operation_Number, we generate the directory according to the Table numbers.
========================
How to generate profiler
========================
1. For accurate operation count, we first compile GENESIS without SIMD option:
./configure --enable-single --host=Fugaku --enable-debug=1 --enable-pktimer --with-fapp
2. In the job script XX.sh, we add "fapp -C -Icpupa,nompi -Hevent=pa1 -d ./rep1" before typing "mpiexec". Then, the program generates directory named "rep1" which has all the profiler data.
3. By typing "fapppx -A -Icpupa -tcsv -o pa1.csv -d ./rep1", we generate the profiler file named "pa1.csv" which can be imported by excel program.
4. Put "pa1.csv" and a template excel file prepared by Fujitsu in the same directory. By importing the data of "pa1.csv" from the template, we can check operation counts.
5. For understanding FLOPS/Operation wait/SIMD rate/Memory hit rate and so on, we compile GENESIS with full optimization condition:
./configure --enable-single --host=Fugaku --enable-pktimer --with-fapp
6. Do the same procedure as 2-4 and we can obtain profiler information.
=======================
How to generate Figures
=======================
1. Figures 1, 3, 4, 5, 7 are drawn by Microsoft Powerpoint. Drawn figures are saved as jpg files.
2. Figure 2 is imported from https://www.r-ccs.riken.jp/en/postk/project/outline
3. Crowd system in Figure 6 and 9 are drawn by VMD software.
4. Figure 8 is drawn by Origin 8.5J. After making a plot in Origin, the graph is exported as eps and character font is refined by Adobe Illustrator CS5.
In all cases, we drew without specfic usgae of scripts.