Example scripts for multiple serial simulations, OpenMP jobs, and hybrid jobs
Here are examples of job submission files for various cases. You can copy one of those files in your home folder and modify it according to your needs. You will need to adapt it to the server that you are using since not all servers have the same number of cores or features.
Multiple serial jobs
When you need to run the same program for multiple datasets, it is best to group those jobs in a single job. This is even required on some our servers which allow only a single job per node (Colosse and Mp2) to avoid wasting resources.
To efficiently group your simulations, you ideally want compute times that are roughly the same for each job. You also need to make sure that enough memory is available on a single compute node. You need take this into consideration when choosing the number of jobs to run on a node.
The principle is simple : you launch tasks in background using the & character. Then, the command wait ensures the script waits until all background tasks are done before terminating the job. In the example below, we assume that the program prog is within the folder $HOME/program_dir et that each subdirectory contains input files that are different, such that each job writes its result in different output files. We also assume that all necessary module have already been loaded.
Sequential task array
You may submit a batch of serial tasks with bqTools on the servers that have it installed. For other servers, here is how you can run serial tasks on a larger number of dataset, which are for example in directories $SCRATCH/dirX, where X goes from 0 to 255 and where a maximum of 10 tasks are run simultaneously.
First a Moab example, to be submitted with the msub command.
Same example, with Torque, to be submitted with qsub.
Because OpenMP jobs only works on a shared-memory architecture, they can only be run on a single node at a time.
Note : The KMP_* parameters below are only valid for the Intel compiler. For more information, see the OpenMP page.
Here is an example of job using MPI with one process per node and OpenMP on all cores of a single node. The example uses the Intel compiler.
Additional server-specific documentation