Skip to content

Commit

Permalink
Merge branch 'main' into microphysics
Browse files Browse the repository at this point in the history
* main: (65 commits)
  Minor
  Minor
  Add comments
  Add execute premission check
  Bugfix string length
  Apply suggestions from code review
  [Workflow] Update all parameters wiki page
  Update output format
  Update description
  Load Opt__Output_Text_Length_Int when restert
  Rename DUMP_DIR to OUTPUT_DIR
  [Workflow] Update all parameters wiki page
  Let OUTPUT_DIR shorter
  Add file premission check
  Fix file path
  Remove useless variable
  Fix Input__Note output
  Update missed filename
  Rename DUMP_DIR to OUTPUT_DIR
  Apply suggestions from code review
  ...
  • Loading branch information
jzuhone committed Jan 19, 2025
2 parents 97d881f + 5fd2415 commit 47cac99
Show file tree
Hide file tree
Showing 101 changed files with 838 additions and 102 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,8 @@ For variables with `Default/Min/Max` labeled as `Depend`, click the parameter na
| [[ OPT__OUTPUT_PRES \| Runtime-Parameters:-Outputs#OPT__OUTPUT_PRES ]] | 0 | None | None | output gas pressure [0] ##HYDRO ONLY## |
| [[ OPT__OUTPUT_RESTART \| Runtime-Parameters:-Outputs#OPT__OUTPUT_RESTART ]] | 0 | None | None | output data immediately after restart [0] |
| [[ OPT__OUTPUT_TEMP \| Runtime-Parameters:-Outputs#OPT__OUTPUT_TEMP ]] | OutTempDefault | None | None | output gas temperature [0 (HD) or 1 (SRHD)] ##HYDRO ONLY## |
| [[ OPT__OUTPUT_TEXT_FORMAT_FLT \| Runtime-Parameters:-Outputs#OPT__OUTPUT_TEXT_FORMAT_FLT ]] | "%24.16e" | None | None | string format of output text files [%24.16e] |
| [[ OPT__OUTPUT_TEXT_FORMAT_FLT \| Runtime-Parameters:-Outputs#OPT__OUTPUT_TEXT_FORMAT_FLT ]] | "%24.16e" | None | None | string format of floating-point variables in output text files [%24.16e] |
| [[ OPT__OUTPUT_TEXT_LENGTH_INT \| Runtime-Parameters:-Outputs#OPT__OUTPUT_TEXT_LENGTH_INT ]] | 12 | 0 | None | string length of integer variables in output text files [12] |
| [[ OPT__OUTPUT_TOTAL \| Runtime-Parameters:-Outputs#OPT__OUTPUT_TOTAL ]] | 1 | 0 | 2 | output the simulation snapshot: (0=off, 1=HDF5, 2=C-binary) [1] |
| [[ OPT__OUTPUT_USER \| Runtime-Parameters:-Outputs#OPT__OUTPUT_USER ]] | 0 | None | None | output the user-specified data -> edit "Output_User.cpp" [0] |
| [[ OPT__OUTPUT_USER_FIELD \| Runtime-Parameters:-Outputs#OPT__OUTPUT_USER_FIELD ]] | 0 | None | None | output user-defined derived fields [0] -> edit "Flu_DerivedField_User.cpp" |
Expand Down Expand Up @@ -341,6 +342,7 @@ For variables with `Default/Min/Max` labeled as `Depend`, click the parameter na
| [[ OPT__UM_IC_REFINE \| Runtime-Parameters:-Initial-Conditions#OPT__UM_IC_REFINE ]] | 1 | None | None | refine UM_IC from level OPT__UM_IC_LEVEL to MAX_LEVEL [1] |
| [[ OPT__UNIT \| Runtime-Parameters:-Units#OPT__UNIT ]] | 0 | None | None | specify code units -> must set exactly 3 basic units below [0] ##USELESS FOR COMOVING## |
| [[ OPT__VERBOSE \| Runtime-Parameters:-Miscellaneous#OPT__VERBOSE ]] | 0 | None | None | output the simulation progress in detail [0] |
| [[ OUTPUT_DIR \| Runtime-Parameters:-Outputs#OUTPUT_DIR ]] | "." | None | None | set the output directory [.] |
| [[ OUTPUT_DT \| Runtime-Parameters:-Outputs#OUTPUT_DT ]] | -1.0 | None | None | output data every OUTPUT_DT time interval ##OPT__OUTPUT_MODE==2 ONLY## |
| [[ OUTPUT_PART_X \| Runtime-Parameters:-Outputs#OUTPUT_PART_X ]] | -1.0 | None | None | x coordinate for OPT__OUTPUT_PART [-1.0] |
| [[ OUTPUT_PART_Y \| Runtime-Parameters:-Outputs#OUTPUT_PART_Y ]] | -1.0 | None | None | y coordinate for OPT__OUTPUT_PART [-1.0] |
Expand Down
16 changes: 16 additions & 0 deletions doc/wiki/Runtime-Parameters-related/Runtime-Parameters:-Outputs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ Parameters described on this page:
[OPT__OUTPUT_TOTAL](#OPT__OUTPUT_TOTAL),  
[OPT__OUTPUT_PART](#OPT__OUTPUT_PART),  
[OPT__OUTPUT_TEXT_FORMAT_FLT](#OPT__OUTPUT_TEXT_FORMAT_FLT),  
[OPT__OUTPUT_TEXT_LENGTH_INT](#OPT__OUTPUT_TEXT_LENGTH_INT),  
[OPT__OUTPUT_USER](#OPT__OUTPUT_USER),  
[OPT__OUTPUT_PAR_MODE](#OPT__OUTPUT_PAR_MODE),  
[OPT__OUTPUT_PAR_MESH](#OPT__OUTPUT_PAR_MESH),  
Expand Down Expand Up @@ -31,6 +32,7 @@ Parameters described on this page:
[OUTPUT_PART_Y](#OUTPUT_PART_Y),  
[OUTPUT_PART_Z](#OUTPUT_PART_Z),  
[INIT_DUMPID](#INIT_DUMPID),  
[OUTPUT_DIR](#OUTPUT_DIR)  


Parameters below are shown in the format:   **`Name`   (Valid Values)   [Default Value]**
Expand Down Expand Up @@ -75,6 +77,12 @@ This will affect files generated by, for example,
[OPT__OUTPUT_BASEPS](#OPT__OUTPUT_BASEPS).
* **Restriction:**

<a name="OPT__OUTPUT_TEXT_LENGTH_INT"></a>
* #### `OPT__OUTPUT_TEXT_LENGTH_INT` &ensp; (&#8805;0) &ensp; [12]
* **Description:**
String length of integer data in output text files.
* **Restriction:**

<a name="OPT__OUTPUT_USER"></a>
* #### `OPT__OUTPUT_USER` &ensp; (0=off, 1=on) &ensp; [0]
* **Description:**
Expand Down Expand Up @@ -283,6 +291,14 @@ in most cases, even when restarting simulations, unless you want to overwrite
the default value.
* **Restriction:**

<a name="OUTPUT_DIR"></a>
* #### `OUTPUT_DIR` &ensp; (string) &ensp; [.]
* **Description:**
Specify the output data directory. In most cases, it is unnecessary to set this value
unless you want to overwrite the default directory.
* **Restriction:**
The string length should be shorter than `MAX_STRING-100`.


## Remarks

Expand Down
97 changes: 97 additions & 0 deletions example/grackle/Make.mach.eureka-intel
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#=======================================================================
#
# FILE: Make.mach.eureka-intel
#
# DESCRIPTION: Makefile for the calab-ntu Eureka GPU cluster with Intel
# compiler and OpenMP support
#
# AUTHOR: Hsi-Yu Schive (modified by Guan-Ming Su, Barry T. Chiang,
# and Chun-Yen Chen)
#
# DATE: April 1, 2016 (August 28, 2024)
#=======================================================================

MACH_TEXT = Eureka Intel
MACH_VALID = 0
MACH_FILE = Make.mach.eureka-intel

#-----------------------------------------------------------------------
# Install paths (local variables)
#-----------------------------------------------------------------------

LOCAL_HDF5_INSTALL = /software/hdf5/default
LOCAL_FC_INSTALL = /software/intel/oneapi/compiler/default/linux/compiler/lib/intel64

#-----------------------------------------------------------------------
# Compiler settings
#-----------------------------------------------------------------------

MACH_CPP = icpc # C preprocessor command

# Compilers
MACH_CC_NOMPI = icc # C compiler
MACH_CXX_NOMPI = icpc # C++ compiler
MACH_FC_NOMPI = ifort # Fortran 77
MACH_F90_NOMPI = ifort # Fortran 90
MACH_LD_NOMPI = icpc # Linker
MACH_LIBTOOL = libtool

#-----------------------------------------------------------------------
# Machine-dependent defines
#-----------------------------------------------------------------------

# Note: When compiling against HDF5 version 1.8 or greater, you need to
# compile HDF5 with --with-default-api-version=v16, or specify with
# -DH5_USE_16_API.

MACH_DEFINES = -DLINUX -DH5_USE_16_API

#-----------------------------------------------------------------------
# Compiler flag settings
#-----------------------------------------------------------------------

MACH_OMPFLAGS = -fopenmp # OpenMP flags
MACH_CPPFLAGS = -P -traditional # C preprocessor flags
MACH_CFLAGS = -mp1 -prec_div -fp_port -align # C compiler flags
MACH_CXXFLAGS = -mp1 -prec_div -fp_port -align # C++ compiler flags
MACH_FFLAGS = # Fortran 77 compiler flags
MACH_F90FLAGS = -mp1 -prec_div -fp_port -align #-save -zero # Fortran 90 compiler flags
MACH_LDFLAGS = #-lifcore -lifport -lpthread -ldl # Linker flags

#-----------------------------------------------------------------------
# Optimization flags
#-----------------------------------------------------------------------

MACH_OPT_WARN = # Flags for verbose compiler warnings
MACH_OPT_DEBUG = -g -O0 # Flags for debugging
MACH_OPT_HIGH = -O2 # Flags for high conservative optimization
MACH_OPT_AGGRESSIVE = -O3 #-Mfptrap -Mflushz -fastsse -Mdaz -Mnontemporal -Mnofprelaxed -Mvect=altcode,assoc,prefetch -Kieee # Flags for aggressive optimization

#-----------------------------------------------------------------------
# Includes
#-----------------------------------------------------------------------

LOCAL_INCLUDES_HDF5 = -I$(LOCAL_HDF5_INSTALL)/include # HDF5 includes
MACH_INCLUDES = $(LOCAL_INCLUDES_HDF5)

#-----------------------------------------------------------------------
# Libraries
#-----------------------------------------------------------------------

LOCAL_LIBS_HDF5 = -L$(LOCAL_HDF5_INSTALL)/lib -lhdf5 # HDF5 libraries
LOCAL_LIBS_MACH = -L$(LOCAL_FC_INSTALL) -lm -lifcore -lifport # Machine-dependent libraries

MACH_LIBS = $(LOCAL_LIBS_HDF5) $(LOCAL_LIBS_MACH)

#-----------------------------------------------------------------------
# Installation
#-----------------------------------------------------------------------

# if $(HOME)/local does not exist, mkdir before `make install`

### Use this when make omp-on is set
MACH_INSTALL_PREFIX = <local_grackle_directory>/grackle_eureka_double_omp
### Use this when make omp-off is set
#MACH_INSTALL_PREFIX = <local_grackle_directory>/grackle_eureka_double_no_omp
MACH_INSTALL_LIB_DIR =
MACH_INSTALL_INCLUDE_DIR =
97 changes: 97 additions & 0 deletions example/grackle/Make.mach.spock-intel
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
#=======================================================================
#
# FILE: Make.mach.spock-intel
#
# DESCRIPTION: Makefile for the calab-ntu Spock GPU cluster with Intel
# compiler and OpenMP support
#
# AUTHOR: Hsi-Yu Schive (modified by Guan-Ming Su, Barry T. Chiang,
# and Chun-Yen Chen)
#
# DATE: April 1, 2016 (August 28, 2024)
#=======================================================================

MACH_TEXT = Spock Intel
MACH_VALID = 0
MACH_FILE = Make.mach.spock-intel

#-----------------------------------------------------------------------
# Install paths (local variables)
#-----------------------------------------------------------------------

LOCAL_HDF5_INSTALL = /software/hdf5/1.10.6-intel-2023.1.0
LOCAL_FC_INSTALL = /software/intel/oneapi/compiler/latest/linux/compiler/lib/intel64

#-----------------------------------------------------------------------
# Compiler settings
#-----------------------------------------------------------------------

MACH_CPP = icpc # C preprocessor command

# Compilers
MACH_CC_NOMPI = icc # C compiler
MACH_CXX_NOMPI = icpc # C++ compiler
MACH_FC_NOMPI = ifort # Fortran 77
MACH_F90_NOMPI = ifort # Fortran 90
MACH_LD_NOMPI = icpc # Linker
MACH_LIBTOOL = libtool

#-----------------------------------------------------------------------
# Machine-dependent defines
#-----------------------------------------------------------------------

# Note: When compiling against HDF5 version 1.8 or greater, you need to
# compile HDF5 with --with-default-api-version=v16, or specify with
# -DH5_USE_16_API.

MACH_DEFINES = -DLINUX -DH5_USE_16_API

#-----------------------------------------------------------------------
# Compiler flag settings
#-----------------------------------------------------------------------

MACH_OMPFLAGS = -fopenmp # OpenMP flags
MACH_CPPFLAGS = -P -traditional # C preprocessor flags
MACH_CFLAGS = -mp1 -prec_div -fp_port -align # C compiler flags
MACH_CXXFLAGS = -mp1 -prec_div -fp_port -align # C++ compiler flags
MACH_FFLAGS = # Fortran 77 compiler flags
MACH_F90FLAGS = -mp1 -prec_div -fp_port -align #-save -zero # Fortran 90 compiler flags
MACH_LDFLAGS = #-lifcore -lifport -lpthread -ldl # Linker flags

#-----------------------------------------------------------------------
# Optimization flags
#-----------------------------------------------------------------------

MACH_OPT_WARN = # Flags for verbose compiler warnings
MACH_OPT_DEBUG = -g -O0 # Flags for debugging
MACH_OPT_HIGH = -O2 # Flags for high conservative optimization
MACH_OPT_AGGRESSIVE = -O3 #-Mfptrap -Mflushz -fastsse -Mdaz -Mnontemporal -Mnofprelaxed -Mvect=altcode,assoc,prefetch -Kieee # Flags for aggressive optimization

#-----------------------------------------------------------------------
# Includes
#-----------------------------------------------------------------------

LOCAL_INCLUDES_HDF5 = -I$(LOCAL_HDF5_INSTALL)/include # HDF5 includes
MACH_INCLUDES = $(LOCAL_INCLUDES_HDF5)

#-----------------------------------------------------------------------
# Libraries
#-----------------------------------------------------------------------

LOCAL_LIBS_HDF5 = -L$(LOCAL_HDF5_INSTALL)/lib -lhdf5 # HDF5 libraries
LOCAL_LIBS_MACH = -L$(LOCAL_FC_INSTALL) -lm -lifcore -lifport # Machine-dependent libraries

MACH_LIBS = $(LOCAL_LIBS_HDF5) $(LOCAL_LIBS_MACH)

#-----------------------------------------------------------------------
# Installation
#-----------------------------------------------------------------------

# if $(HOME)/local does not exist, mkdir before `make install`

### Use this when make omp-on is set
MACH_INSTALL_PREFIX = <local_grackle_directory>/grackle_spock_double_omp
### Use this when make omp-off is set
#MACH_INSTALL_PREFIX = <local_grackle_directory>/grackle_spock_double_no_omp
MACH_INSTALL_LIB_DIR =
MACH_INSTALL_INCLUDE_DIR =
15 changes: 15 additions & 0 deletions example/grackle/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
First-time GRACKLE installation guide (on calab-ntu gpu-clusters as an example):
========================================
1. First download the source code by executing "git clone https://github.com/grackle-project/grackle"
2. Next navigate to "cd grackle". The required UVB interpolation tables can be downloaded via "git submodule update --init"
3. Next navigate to "./configure; cd src/clib"
4. Copy into the config files "Make.mach.eureka-intel" or "Make.mach.spock-intel" (from the directory "<your_gamer_directory>/example/grackle/") and change the prefix: "MACH_INSTALL_PREFIX = <your_grackle_directory>/grackle_spock_double_omp" in "Make.mach.spock-intel" to the path you wish to install the package
5. Next execute "make clean; make machine-spock-intel" or "make clean; make machine-eureka-intel"
6. Next execute "make precision-64" (double precision is recommended for GRACKLE) and "make omp-on" (GAMER does utilize OpenMP)
7. Next execute "make show-config" to verify "CONFIG_PRECISION: 64" and "CONFIG_OMP: on"
8. Next execute "make save-config-gamer" (save this GAMER-spock specific config setting) and "make load-config-gamer" (load this GAMER-spock specific config setting)
9. Lastly for code compilation, execute "make -j 4; make install"
10. To compile the corresponding gamer executable:
(1) Add the path "GRACKLE_PATH <your_grackle_directory>/grackle_spock_double_omp" to "<your_gamer_directory>/configs/spock_intel.config"
(2) Copy "generate_make.sh" from the AGORA test problem and change "--machine=spock_intel" in that file, and then execute "sh generate_make.sh"
(3) Generate a gamer executable via "make clean; make -j 4"
3 changes: 2 additions & 1 deletion example/test_problem/ELBDM/DiskHeating/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ SPEC_INT_GHOST_BOUNDARY 4 # ghost boundary size for spectral int
# data dump
OPT__OUTPUT_TOTAL 1 # output the simulation snapshot: (0=off, 1=HDF5, 2=C-binary) [1]
OPT__OUTPUT_PART 0 # output a single line or slice: (0=off, 1=xy, 2=yz, 3=xz, 4=x, 5=y, 6=z, 7=diag) [0]
OPT__OUTPUT_TEXT_FORMAT_FLT %24.16e # string format of output text files [%24.16e]
OPT__OUTPUT_TEXT_FORMAT_FLT %24.16e # string format of floating-point variables in output text files [%24.16e]
OPT__OUTPUT_USER 0 # output the user-specified data -> edit "Output_User.cpp" [0]
OPT__OUTPUT_PAR_MODE 0 # output the particle data: (0=off, 1=text-file, 2=C-binary) [0] ##PARTICLE ONLY##
OPT__OUTPUT_BASEPS 0 # output the base-level power spectrum [0]
Expand Down Expand Up @@ -276,6 +276,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/ExtPot/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ OUTPUT_PART_X 0.5 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y 0.5 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z 0.5 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ OUTPUT_PART_X 0.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y 0.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z 0.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/HaloMerger/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ OUTPUT_PART_X 0.12938422 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y 0.12938422 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z 0.12938422 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/IsolatedHalo/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# yt inline analysis (SUPPORT_LIBYT only)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/LSS/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/LSS_Hybrid/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]


# miscellaneous
Expand Down
1 change: 1 addition & 0 deletions example/test_problem/ELBDM/Perturbation/Input__Parameter
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,7 @@ OUTPUT_PART_X -1.0 # x coordinate for OPT__OUTPUT_PART [-
OUTPUT_PART_Y -1.0 # y coordinate for OPT__OUTPUT_PART [-1.0]
OUTPUT_PART_Z -1.0 # z coordinate for OPT__OUTPUT_PART [-1.0]
INIT_DUMPID -1 # set the first dump ID (<0=auto) [-1]
OUTPUT_DIR . # set the output directory [.]

# miscellaneous
OPT__VERBOSE 0 # output the simulation progress in detail [0]
Expand Down
Loading

0 comments on commit 47cac99

Please sign in to comment.