All molecular simulation software on ARCHER2 is compiled and looked after by the EPCC support, if you have problems with any of the compiled codes on ARCHER2 then please contact the ARCHER2 support.
AMBER 18/20
Do note that ARCHER2 does not have a central install of AMBER, if you own your own license then you can install this under your own directory. If you do not know how to do this then you can get in touch and ask for support This email address is being protected from spambots. You need JavaScript enabled to view it.
A single node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --ntasks=128#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:45:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module use /path/to/your/custom/modulesmodule load amber20
export OMP_NUM_THREADS=1
srun pmemd.MPI -O -i example.in -c example.rst -p example.top -o example.out
A multinode all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --ntasks=640#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:25:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module use /path/to/your/custom/modulesmodule load amber20
export OMP_NUM_THREADS=1
srun pmemd.MPI -O -i example.in -c example.rst -p example.top -o example.out
Gromacs 2020.3
During our benchmarking of ARCHER2, using our benchmark suite, we got superior performance with the all MPI cases. However you may be running non-typical loads and may see good performance from mixed MPI and OMP, for example if you are running coarse grain or martini simulations at smaller sizes.
A single node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --ntasks=128#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load gromacs/2020.3
export OMP_NUM_THREADS=1
srun gmx_mpi mdrun -deffnm benchmark
A multiple node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --ntasks=640#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load gromacs/2020.3
export OMP_NUM_THREADS=1
srun gmx_mpi mdrun -deffnm benchmark
A single node mixed MPI and OMP example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load gromacs/2020.3
export OMP_NUM_THREADS=8
srun gmx_mpi mdrun -deffnm benchmark -ntomp 8
A multiple node mixed MPI and OMP example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load gromacs/2020.3
export OMP_NUM_THREADS=8
srun gmx_mpi mdrun -deffnm benchmark -ntomp 8
Lammps 3.3.2020
At the time of early testing on ARCHER2. Lammps was not compiled with OMP support, so we compiled our own with the extra required modules needed. You will noticed that we are using a custom module in the mixed MPI + OMP examples. We obtained vastly superior benchmark results when using the mixed OMP and MPI configurations rather than just the pure MPI. This is likely because once we got to using 16 or 32 MPI and 8 or 4 OMP threads per node, that the job load matches the layout of the CPU better.
A single node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --ntasks=128#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:40:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load lammps/3_Mar_2020
#export OMP_NUM_THREADS=1
srun lmp -i benchmark.in -log benchmark.log
A multiple node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --ntasks=640#SBATCH --tasks-per-node=128#SBATCH --cpus-per-task=1#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load lammps/3_Mar_2020
#export OMP_NUM_THREADS=1
srun lmp -i benchmark.in -log benchmark.log
A single node mixed MPI and OMP example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module use /path/to/custom/modules/module load lammps-omp
export OMP_NUM_THREADS=8
srun --unbuffered --cpu-bind=cores lmp_archer2 -sf omp -pk omp 8 -i benchmark.in -log benchmark.log
A multiple node mixed MPI and OMP example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module use /path/to/your/custom/modules/module load lammps-omp
export OMP_NUM_THREADS=8
srun --unbuffered --cpu-bind=cores lmp_archer2 -sf omp -pk omp 8 -i benchmark.in -log benchmark.log
NAMD 2.14
The examples below are to be used with the SMP builds of NAMD that are centrally available on ARCHER2. We obtained vastly superior benchmark results when using the mixed OMP and MPI configurations rather than just the pure MPI. This is likely because once we got to using 16 or 32 MPI and 8 or 4 OMP threads per node, that the job load matches the layout of the CPU better.
A single node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --ntasks=1#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load namd/2.14
export OMP_NUM_THREADS=1
srun namd2 +ppn 127 +pemap 1-127 +commap 0 benchmark.in > benchmark.log
A multiple node all MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --ntasks=5#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load namd/2.14
export OMP_NUM_THREADS=1
srun namd2 +ppn 127 +pemap 1-127 +commap 0 benchmark.in > benchmark.log
A single node mixed OMP and MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=1#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load namd/2.14
export OMP_NUM_THREADS=8
srun --unbuffered --cpu-bind=cores namd2 +setcpuaffinity +ppn 7 benchmark.in > benchmark.log
A multiple node mixed OMP and MPI example
#!/bin/bash
#SBATCH --job-name=hecbiosim-benchmark#SBATCH --nodes=5#SBATCH --tasks-per-node=16#SBATCH --cpus-per-task=8#SBATCH --time=00:20:00
#SBATCH --account=<account-code>
#SBATCH --partition=standard#SBATCH --qos=standard
module load namd/2.14
export OMP_NUM_THREADS=8
srun --unbuffered --cpu-bind=cores namd2 +setcpuaffinity +ppn 7 benchmark.in > benchmark.log