Introduction
This guide is designed to be a reference for users of the high-performance computing (HPC) facility: foseres. It provides all the information needed to access the system, transfer data, manage your resources (disk and compute time), submit jobs, compile programs and manage your environment. It has been been adapted from the documentation of the tesseract service:
https://tesseract-dirac.readthedocs.io/en/latest/user-guide/introduction.html
Acknowledging foseres
You should use the following phrase to acknowledge foseres in all research outputs that have used the facility:
This work was carried out using the computational facilities of the High Performance Computing Centre, University of Plymouth- https://www.plymouth.ac.uk/about-us/university-structure/faculties/science-engineering/hpc.
Getting help
If you need any assistance in using foseres, please send a support requests to HPC Support at hpcsupport@plymouth.ac.uk, and our team will try to help you as soon as possible. In your Email, please share as much detail as possible about your query.
Training
Some ressources are available here:
Hardware
The current foseres compute provision consists of 52 nodes, connected together by a single Intel OPA fabric.
There are 2 login nodes that share a common software environment and file system with the compute nodes.
Standard Compute Nodes
Foseres standard compute nodes each contain two 2.1 GHz, 16-core Intel Xeon E5-2683v4 (Broadwell) series processors. Each of the cores in these processors support 2 hardware threads (Intel hyperthreads technology), which are disabled by default.
There are 52 standard compute nodes on foseres giving a total of 1,664 cores.
The compute nodes on foseres have 128 GB of memory shared between the two processors. The memory is arranged in a non-uniform access (NUMA) form: each 16-core processor is a single NUMA region with local memory of 64 GB. Access to the local memory by cores within a NUMA region has a lower latency than accessing memory on the other NUMA region.
There are three levels of cache, configured as follows:
L1 Cache 32 KB Instr., 32 KB Data (per core)
L2 Cache 256 KB (per core)
L3 Cache 40 MB (shared)
OPA Interconnect
The system has a single Intel OPA fabric and every compute node and login node has a single OPA interface. The Lustre file system servers have two connections to the OPA fabric and all Lustre file system IO traverses the OPA fabric.
Topology …
File systems and Data Infrastructure
There is currently a single Lustre parallel file system available on foseres. It is a Lustre parallel file system desgined to give high read/write bandwidth for parallel I/O operations.
The Lustre file system has a total of 0.5 PB available. The login and compute nodes mount the storage as / and all home and work directories are available on all nodes.
#iThe compute nodes are diskless. Each node boots from a cluster management noded called the Rack Leader and NFS mounts the root file system from #this management node.
Note
Data on the Lustre file system is not automatically backed up.
Parallel I/O
For a description of the terms associated with Lustre parallel file systems please see the description on Wikipedia:
The default striping on the Lustre filesystem is 1 stripe, and the default stripe size is 1 MiB.