New ARC Cluster: DragonsTooth

ARC is happy to announce the release of a new cluster, named DragonsTooth, available at DragonsTooth is made up of 48 nodes, each equipped with:

  • 2 x Intel Xeon E5-2680v3 (Haswell) 2.5 GHz 12-core CPU (same CPU as NewRiver)
  • 256 GB 2133 MHz DDR4 memory for large-memory problems
  • 4 x 480 GB SSD Hard Drives for fast local I/O ($TMPDIR)
  • 806 GFlops/s theoretical double-precision peak

Nodes are connected to each other and to storage via 10 gigabit ethernet (10GbE), a communication channel with high bandwidth but higher latency than InfiniBand (IB). As a result, DragonsTooth is better suited to jobs that require less communication between nodes and to/from storage than NewRiver, which has similar nodes but a low-latency IB interconnect. For example,

  • If you are running multinode (MPI) jobs, you may want to use NewRiver
  • If you are running single node jobs, you may want to use DragonsTooth (especially jobs with low I/O requirements or I/O that can be moved to local disk $TMPDIR)

For that reason, DragonsTooth's queue structure is designed to allow more jobs and longer-running jobs than on other ARC clusters: jobs can run for up to 30 days and users can run up to 288 jobs at a time (though fewer long jobs can run at a time).

To facilitate users getting started, we have:

  • Created DragonsTooth accounts for NewRiver users who have logged in in the last 6 months
  • Created DragonsTooth allocations for NewRiver projects with nonzero balance. These allocations are 100,000 SUs good for 6 months.

So many NewRiver users will be able to log in and start running immediately. Other questions can request accounts or allocationsusing our usual forms.