VSC-5¶
Note
To compute on VSC-5, see VSC-5 partitions and QOS
VSC-5 is the latest addition to the Vienna Scientific Cluster infrastructure, supplied by Opens external link in new windowMegware. The system comprises 1,540 Opens external link in new windowAMD EPYC Milan processors and 98,560 cores (more than double of the VSC-4), and features a hardware partition of CPU and GPU nodes connected via Mellanox HDR infiniband (200 Gbit/s bandwidth).
Hardware¶
Partition | Nodes | Architecture | CPU | Cores per CPU (physical/with HT) | GPU | RAM | Use |
---|---|---|---|---|---|---|---|
zen3_0512* | 564 | AMD | 2x AMD 7713 | 64/128 | No | 512 GB | The default partition |
zen3_1024 | 120 | AMD | 2x AMD 7713 | 64/128 | No | 1 TB | High Memory partition |
zen3_2048 | 20 | AMD | 2x AMD 7713 | 64/128 | No | 2 TB | Higher Memory partition |
cascadelake_0384 | 48 | Intel | 2x Intel Cascadelake | 48/96 | No | 384 GB | Directly use programs compiled for VSC-4 |
zen2_0256_a40x2 | 45 | AMD | 2x AMD 7252 | 8/16 | 2x NVIDIA A40 | 256 GB | Best for single precision GPU code |
zen3_0512_a100x2 | 60 | AMD | 2x AMD 7713 | 64/128 | 2x NVIDIA A100 | 512 GB | Best for double precision GPU code |
CPU partition features¶
710 compute nodes equipped with two AMD EPYC 7713 (Milan), 64 cores (in total 90,880 cores) with a base frequency of 2GHz. Most of the nodes are equipped with 512GB of memory, but there are also large memory nodes (120 nodes with 1TB and 20 nodes 2TB). Each node has a 2TB NVME disk. The system is complemented with 10 login nodes with identical processors as the compute nodes.
GPU partition features¶
60 compute nodes equipped with two AMD EPYC 7713 (Milan) processors, 64 cores (in total 7,680 cores) operating at 2GHz and 512 GB memory, while each node is additionally equipped with two NVIDIA A100 cards (40GB memory). Each node has a 2TB NVME disk. The system is complemented with 3 master nodes for node deployment.
Cooling¶
The VSC-5 mostly uses a direct hot water cooling system Opens external link in new windowColdCon. It operates at temperatures up to 50°C and allows energy-efficient free cooling throughout the year. Direct water cooling permits direct extraction of the heat produced in the compute nodes, in particular by the CPUs, the memory DIMMS, the SSDs and the Infiniband adapters. The waste heat can be reused to produce hot water or support the existing heating infrastructure.
For the cooling of other components like power supplies, PDUs, infiniband and ethernet switches air-cooling via in-row coolers is used.
The GPU partition as well as login and service nodes are also air-cooled.
Performance¶
The current configuration of the CPU partition achieves a benchmark value (Rmax) of 2.31 PFlop/s with a theoretical peak performance (Rpeak) of 3.13 PFlop/s. In addition, the GPU partition of VSC-5 delivers 1.17 PFlop/s. Thus, VSC-5 has an aggregated peak performance of 4.30 PFlop/s.