Here we present the results of our benchmarking of the ARM cluster at Dirac. The benchmarks we are running are our HECBioSim benchmark suite here. Our benchmarks aren't too heavily tuned as these are designed to be used by our community to work out how much HPC time to ask for on this resource, so are set at a level that would be reasonable for any normal biomolecular MD simulation. More information on the benchmarks themselves can be found at the above link.
The ARM cluster at Dirac is hosted by Leicester University and is a Tier2 national machine in the UK. The cluster is made up of 4096 cores of ARMs Marvel ThunderX2 platform each node has 64 cores (2 x 32 core) and are linked by a Mellanox Bluefield interconnect.
Gromacs 2020.2 Single Precision
nodes | cores | 20k atoms | 61k atoms | 465k atoms | 1.4M atoms | 3M atoms |
ns/day | ns/day | ns/day | ns/day | ns/day | ||
1 | 64 | 74.0 | 25.0 | 3.1 | 1.35 | 0.633 |
2 | 128 | 109.3 | 42.8 | 6.2 | 2.64 | 1.255 |
3 | 192 | 178.8 | 58.6 | 9.5 | 3.77 | 1.867 |
4 | 256 | 211.4 | 74.9 | 11.9 | 5.11 | 2.453 |
5 | 320 | 242.3 | 89.4 | 14.3 | 6.24 | 2.928 |
6 | 384 | 250.2 | 100.0 | 17.4 | 6.88 | 3.458 |
7 | 448 | 145.8 | 130.4 | 20.5 | 7.76 | 3.844 |
8 | 512 | 216.7 | 100.5 | 21.3 | 9.35 | 4.275 |
10 | 640 | failed | 141.5 | 19.5 | 9.89 | 5.318 |
12 | 768 | failed | 153.5 | 23.0 | 11.27 | 6.285 |
14 | 896 | failed | failed | 25.1 | 12.49 | 7.24 |
16 | 1024 | failed | 26.5 | 15.19 | 7.945 | |
18 | 1152 | 30.3 | 16.17 | 7.244 | ||
20 | 1280 | 32.1 | 16.18 | 9.673 | ||
22 | 1408 | 27.3 | 18.35 | 8.333 | ||
24 | 1536 | 45.9 | 18.39 | 9.037 | ||
28 | 1792 | 37.8 | 21.55 | 10.054 | ||
32 | 2048 | 36.8 | 14.85 | 11.177 | ||
36 | 2304 | failed | ||||
40 | 2560 | failed | ||||
48 | 3072 | failed | ||||
64 | 4096 |
LAMMPS 07.8.2019
nodes | cores | 20k atoms | 61k atoms | 465k atoms | 1.4M atoms | 3M atoms |
ns/day | ns/day | ns/day | ns/day | ns/day | ||
1 | 64 | 19.4 | 6.3 | 0.80 | 0.26 | 0.13 |
2 | 128 | 31.6 | 11.0 | 1.52 | 0.55 | 0.23 |
3 | 192 | 37.3 | 15.2 | |||
4 | 256 | 39.1 | 17.9 | 2.79 | 1.08 | 0.51 |
5 | 320 | 37.7 | 18.6 | |||
6 | 384 | 34.3 | 19.1 | 3.85 | 1.47 | 0.73 |
7 | 448 | 37.3 | 18.7 | |||
8 | 512 | 43.4 | 21.3 | 5.01 | 1.86 | 0.91 |
10 | 640 | 47.1 | 25.8 | 6.10 | 2.24 | 1.15 |
12 | 768 | 51.6 | 26.3 | 7.04 | 2.65 | 1.32 |
14 | 896 | 43.7 | 27.3 | |||
16 | 960 | 7.71 | 3.15 | 1.64 | ||
18 | 1280 | 9.24 | 3.84 | 1.90 | ||
20 | 1600 | 10.64 | 4.46 | 2.34 | ||
22 | 1920 | 11.21 | 5.11 | 2.77 | ||
24 | 2240 | 11.85 | 5.67 | 3.16 | ||
28 | 2880 | 12.83 | 6.70 | 3.68 | ||
32 | 3840 | 13.14 | 7.54 | 4.22 |
NAMD 2.13
nodes | cores | 20k atoms | 61k atoms | 465k atoms | 1.4M atoms | 3M atoms |
ns/day | ns/day | ns/day | ns/day | ns/day | ||
1 | 64 | 24.0 | 9.1 | 1.28 | 0.45 | 0.17 |
2 | 128 | 37.8 | 16.2 | 2.34 | 0.79 | 0.35 |
3 | 192 | 53.4 | 18.4 | |||
4 | 256 | 68.3 | 15.7 | 3.10 | 1.34 | 0.72 |
5 | 320 | 77.1 | 17.8 | |||
6 | 384 | 88.3 | 26.2 | 5.23 | 1.61 | 1.01 |
7 | 448 | 76.2 | 29.3 | |||
8 | 512 | 81.0 | 28.4 | 6.25 | 2.52 | 1.21 |
10 | 640 | 77.4 | 28.1 | 7.71 | 2.86 | 1.41 |
12 | 768 | 76.1 | 33.9 | 10.12 | 3.51 | 2.06 |
14 | 896 | 59.6 | 40.5 | |||
16 | 960 | 66.8 | 30.2 | 12.32 | 4.23 | 2.51 |
18 | 1280 | 13.48 | 5.51 | 3.10 | ||
20 | 1600 | 6.69 | 6.45 | 3.53 | ||
22 | 1920 | 13.38 | 7.59 | 3.74 | ||
24 | 2240 | 10.83 | 9.14 | 4.51 | ||
28 | 2880 | 19.78 | 11.01 | 5.50 | ||
32 | 3840 | 20.44 | 12.52 | 6.99 |