Your code on the compute servers
Calcul Québec's compute servers are able to run all programs that run on Linux. In the simplest case, the software you need is already installed on one of the compute servers. It will then be accessible in module form. If this is not the case, you can ask our support team to install it for you, or do it yourself.
Modules are software blocks that can be added to your environments to modify its behaviour. This modular architecture allows having multiple versions of the same application installed without there being a conflict. Modules are generally managed by the Environment Modules tool.
A module file contains all informations needed to configure the environment to be able to use an application. Typically, a module file contains instructions that modify or initialize environment variables, like PATH and LD_LIBRARY_PATH, to use the various programs that are installed. Multiple module files are available on each system.
Main module commands
The commande module has multiple sub-commands. The normal syntax is
[name@server $] module <command> [other options]
To obtain the list of available subcommands, use
[name@server $] module help
To list the available modules on a given system, use
[name@server $] module avail
You can add the -l option to obtain the list with the last modification date of the modules
[name@server $] module avail -l
You can obtain a list of available modules for a particular library or tool
[name@server $] module avail openmpi
The list sub-command shows modules that are presently loaded in your environment.
[nom@serveur $] module list
Sub-Command add or load
The add sub-command (load is a synonym) loads a given module. For example
[name@server $] module add compilers/gcc/4.7.2
[name@server $] module load compilers/gcc/4.7.2
could load the module for the GCC compiler, version 4.7.2. To get the name of all modules available or a given server use the avail sub-command.
You can load more than one module with the same command. For example
[name@server $] module add compilers/gcc/4.7.2 mpi/openmpi/1.6.3_gcc
would load GCC 4.7.2 followed by the OpenMPI library, version 1.6.3, compiled for GCC.
Sub-Command rm or unload
The counterpart for the add (load) sub-command is the rm (unload) sub-command, which removes a module from your environment. For example
[name@server $] module rm compilers/gcc/4.7.2
[name@server $] module unload compilers/gcc/4.7.2
removes GCC 4.7.2 from your environment.
With the purge sub-command you can delete all modules you loaded in one fell swoop.
[name@server $] module purge
Sub-Commands show, help and whatis
The show, help and whatis sub-commands give you more information on a given module. The show sub-command shows the whole module file, the help command shows a help message, and the whatis command shows a description for the module.
[name@server $] module help python64/2.7.3
Sub-Command apropos or keyword
With the apropos or keyword sub-commands you can look for a keyword in all modules. If you do not know which module is appropriate for your computation, you can search the descriptions like this. For example,
[name@server $] module keyword Fourier
gives the list of modules that will probably be able to do Fourier transforms.
Automatically loading modules
If you always load the same modules, you can add those to a list that is automatically added to your environment by default. To manage this list, use the initadd, initrm and initlist sub-commands. Note that these commands do not have any effect on your running session, as they only apply to future sessions.
If you obtain an error message when you run one of these commands, it is possible that your $HOME directory does not contain the .modules configuration file. You can create this file using the following command:
[name@server $] echo "module add null" > $HOME/.modules
To get an application or module installed
If an application is not available on the server you use, you can choose between the following options.
- You can check if the application is available on another compute server. To see the list of compute servers, please refer to Calcul Québec's Web site.
- If your application is not available on any server, or if you would like to use a particular server and the application is not available there, you can contact Calcul Québec's support team and ask if the application could be installed. In the majority of cases that will be possible. Depending on the use case, the application will be globally installed, only for you, or for your research group. If it is impossible to get your program to work on one of our compute servers, our team could study possible alternatives.
- You can also try to install the application yourself (see the following section).
Do it yourself
If the application is easy to install, or if it is an in-house code, you can install it yourself. To do that you need the source code and a compiler. All compute servers offer multiple compilers for multiple languages. For the C, C++ and Fortran programming languages, different versions of the GNU and Intel compilers are generally available. Other compilers, such as Portland Group (PGI) and CLang are also sometimes offered.
Retrieving the source code
You should first retrieve the application's source code. The method used depends on the application. The simplest situation is to download an archive containing the source code from a Web site. You should then copy it to a compute server, generally using Secure Copy. Please refer to the page Connecting and transferring files for instructions how to transfer. Lastly, you should unpack the source code, typically using one of the following commands:
[name@server $] tar xfvz mon_application.tar.gz
[name@server $] unzip mon_application.zip
In the most simple cases, you can compile a given application in a few commands. The exact commands depend on the application's packaging, but are often similar. You should give close attention to the installation directory, that should be put inside your personal or group file space. Once you entered the directory containing the source code, you often find a file named README or INSTALL that contain installation instructions. Read those instructions. If the following subsections, we categorize various ways of compiling an application or library. Note that you must load a compiler in advance.
The Make utility is frequently used to compile and application or library. If the source code directory contains a file named Makefile, then it is possible that the following command is sufficient to compile the source code :
[name@server $] make
The Autoconf package comprises the most used set of utilities to deploy applications in a portable fashion. If the directory contains a file named configure, it is most likely that Autoconf has been used. In that case, you should be able to compile and install your application simply using the command
[name@server $] ./configure --prefix="$HOME/myapplication/" && make && make install
Depending on the application's dependencies, you may need to load more libraries or even install (or ask to install) more yourself.
This command configures, compiles and installs the application in the directory $HOME/myapplication, so it is within your home folder.
Cmake is another tool that is frequently used to deploy applications. If the source code contains a file name CMakeLists.txt, that is probably the tool that is needed to compile the application. If this is the case, you should probably use the following command:
[nom@serveur $] cmake -DCMAKE_INSTALL_PREFIX="$HOME/myapplication" . && make && make install
to configure, compile and install your application into the $HOME/myapplication/ directory.
Note : it is possible that you should load a module to make the command cmake available.
Creating a module
To know how to create your own modules, please refer to the page Creating a module.
Particularities for each server