Requesting resources using select and place
From Yale HPC Wiki
Contents |
PBS has switched from the "nodes" notation to the "select" notation for Bulldogj cluster and newer. Sometimes it doesn't translate "nodes" statements properly, so we recommend converting to select.
Using Select
Select requests are made via the -l flag as the old "nodes" requests. Here's an example:
-l select=4:ncpus=8:mpiprocs=8:mem=30gb
This means give me four nodes, 8 cpus per node, 8 repetitions of the hostname in the nodefile per node (OpenMPI needs this to launch jobs properly), using 30GB per node. The neat thing about using mem is that it's a consumable resource, so
-l select=32:ib=true:ncpus=1:mpiprocs=1:mem=5gb -l place=free
This means that the job needs 32 processes anywhere that's free and 5GB of memory/process (up to three processes can land on a 16GB node or six on a 32GB node). This allows other memory-light processes to share the node, using the available cores. Memory usage is enforced, meaning jobs will be killed if they use more memory than they request, so this is a safe option and provides more throughput on the cluster.
Further Info
Consult PBS_Professional page for more information.
For a complete list of resources and options see the PBS manpages, including
man qsub man pbs_resources man pbs_job_attributes
