FE&R Proposal Statement

Facilities, Equipment and other Resources Proposal Statement

ARC resources that Researchers can include in Research proposals to describe Facilities, Equipment and other Resources at Virginia Tech:

Computing resources will be provided through the Advanced Research Computing (ARC) Center within the Division Information Technology at Virginia Tech.  ARC provides cutting-edge high-performance computing and visualization resources. Currently available high performance computing (HPC) systems include:

  1. Huckleberry: “Deep learning” cluster with 14 compute nodes equipped with two IBM Power8 CPU, four NVIDIA P100 GPU and NVLINK; Infiniband interconnect
  2. Cascades: General purpose cluster with 190 compute nodes equipped with two 16-core Intel Xeon “Broadwell” CPU and 128 GB of memory; 39 compute nodes equipped with two 12-core Intel Xeon "Skylake" CPU, 376 GB of memory, and two NVIDIA V100 GPU; 4 compute nodes with two NVIDIA K80 GPU, 512 GB of memory and one 2 TB NVMe flash card; 2 four-socket compute nodes with four 18-core Intel Xeon “Broadwell” CPU and 3 TB of memory; Mellanox EDR Infiniband interconnect
  3. NewRiver: General purpose cluster with 100 compute nodes equipped with two 12-core Intel Xeon “Haswell” CPU, 128 GB memory and 18 TB SAS drives; 39 compute nodes with 512 GB memory and two NVIDIA P100 GPU; 16 compute nodes with 512 GB of memory and two 200 GB SSD; 8 compute nodes with 512 GB of memory and NVIDIA K80 GPU; 2 four-socket compute nodes equipped with four 15-core Intel Xeon “IvyBridge” CPU and 3 TB of memory; Infiniband interconnect
  4. Dragonstooth: High-throughput cluster with 48 two-socket compute nodes equipped with two 12-core Intel Xeon “Haswell” CPU, 256 GB of memory and four 480GB SSD drives

Parallel filesystems provide over 4 Petabytes of high performance storage, and a tape archive is provided to support long term data storage. ARC’s Visionarium Lab also provides an array of visualization resources, including the VisCube, an immersive 10′ x 10′ three-dimensional visualization environment. In all, the VT Visionarium provides nearly 86 million pixels, 4 billion triangles-per-second and 22 TB/s of GPU memory bandwidth. ARC resources are able to leverage Virginia Tech’s excellent network connectivity, and network. Virginia offers access to advanced national networks, including ESnet, Internet2, and Mid Atlantic Crossroads.

Upcoming Resources
In the winter of 2019-2020, ARC purchased a new cluster, TinkerCliffs, which is scheduled to be released in the summer of 2020. This large-scale CPU resource will replace BlueRidge, which was retired at the end of 2019. The following table summarizes the hardware that makes up TinkerCliffs:

Base Compute Nodes High Memory Nodes Intel Nodes Total
Vendor Cray Cray HPE -
Chip AMD EPYC 7702 AMD EPYC 7702 Intel Xeon Platinum 9242 -
Nodes 308 8 16 332
Cores/Node 128 128 96 352
Memory (GB)/Node 256 1,024 384 1,664
Total Cores 39,424 1,024 1,536 41,984
Total Memory (GB) 78,848 8,192 6,144 93,184
Interconnect HDR IB HDR IB HDR IB -

Summer 2020 will also see release of a new 7PB BeeGFS parallel file system. This will ultimately replace the existing GPFS system that serves the Work and Groups file systems.