Your code on the compute servers

De Wiki de Calcul Québec
Aller à : Navigation, rechercher
Cette page est une traduction de la page Votre code sur les serveurs de calcul et la traduction est complétée à 100 % et à jour.

Autres langues :anglais 100% • ‎français 100%

Calcul Québec's compute servers are able to run all programs that run on Linux. In the simplest case, the software you need is already installed on one of the compute servers. It will then be accessible in module form. If this is not the case, you can ask our support team to install it for you, or do it yourself.

Sommaire

Modules

Modules are software blocks that can be added to your environments to modify its behaviour. This modular architecture allows having multiple versions of the same application installed without there being a conflict. Modules are generally managed by the Environment Modules tool.

A module file contains all informations needed to configure the environment to be able to use an application. Typically, a module file contains instructions that modify or initialize environment variables, like PATH and LD_LIBRARY_PATH, to use the various programs that are installed. Multiple module files are available on each system.

Main module commands

The commande module has multiple sub-commands. The normal syntax is

[name@server $] module <command> [other options]


To obtain the list of available subcommands, use

[name@server $] module help


Sub-Command avail

To list the available modules on a given system, use

[name@server $] module avail


You can add the -l option to obtain the list with the last modification date of the modules

[name@server $] module avail -l


You can obtain a list of available modules for a particular library or tool

[name@server $] module avail openmpi


Sub-Command list

The list sub-command shows modules that are presently loaded in your environment.

[nom@serveur $] module list


Sub-Command add or load

The add sub-command (load is a synonym) loads a given module. For example

[name@server $] module add compilers/gcc/4.7.2


ou

[name@server $] module load compilers/gcc/4.7.2


could load the module for the GCC compiler, version 4.7.2. To get the name of all modules available or a given server use the avail sub-command.

You can load more than one module with the same command. For example

[name@server $] module add compilers/gcc/4.7.2 mpi/openmpi/1.6.3_gcc


would load GCC 4.7.2 followed by the OpenMPI library, version 1.6.3, compiled for GCC.

Sub-Command rm or unload

The counterpart for the add (load) sub-command is the rm (unload) sub-command, which removes a module from your environment. For example

[name@server $] module rm compilers/gcc/4.7.2


or

[name@server $] module unload compilers/gcc/4.7.2


removes GCC 4.7.2 from your environment.

Sub-Command purge

With the purge sub-command you can delete all modules you loaded in one fell swoop.

[name@server $] module purge


Sub-Commands show, help and whatis

The show, help and whatis sub-commands give you more information on a given module. The show sub-command shows the whole module file, the help command shows a help message, and the whatis command shows a description for the module.

[name@server $] module help python64/2.7.3


Sub-Command apropos or keyword

With the apropos or keyword sub-commands you can look for a keyword in all modules. If you do not know which module is appropriate for your computation, you can search the descriptions like this. For example,

[name@server $] module keyword Fourier


gives the list of modules that will probably be able to do Fourier transforms.

Automatically loading modules

If you always load the same modules, you can add those to a list that is automatically added to your environment by default. To manage this list, use the initadd, initrm and initlist sub-commands. Note that these commands do not have any effect on your running session, as they only apply to future sessions.

If you obtain an error message when you run one of these commands, it is possible that your $HOME directory does not contain the .modules configuration file. You can create this file using the following command:

[name@server $] echo "module add null" > $HOME/.modules


To get an application or module installed

If an application is not available on the server you use, you can choose between the following options.

  • You can check if the application is available on another compute server. To see the list of compute servers, please refer to Calcul Québec's Web site.
  • If your application is not available on any server, or if you would like to use a particular server and the application is not available there, you can contact Calcul Québec's support team and ask if the application could be installed. In the majority of cases that will be possible. Depending on the use case, the application will be globally installed, only for you, or for your research group. If it is impossible to get your program to work on one of our compute servers, our team could study possible alternatives.
  • You can also try to install the application yourself (see the following section).

Do it yourself

If the application is easy to install, or if it is an in-house code, you can install it yourself. To do that you need the source code and a compiler. All compute servers offer multiple compilers for multiple languages. For the C, C++ and Fortran programming languages, different versions of the GNU and Intel compilers are generally available. Other compilers, such as Portland Group (PGI) and CLang are also sometimes offered.

Retrieving the source code

You should first retrieve the application's source code. The method used depends on the application. The simplest situation is to download an archive containing the source code from a Web site. You should then copy it to a compute server, generally using Secure Copy. Please refer to the page Connecting and transferring files for instructions how to transfer. Lastly, you should unpack the source code, typically using one of the following commands:

[name@server $] tar xfvz mon_application.tar.gz


or

[name@server $] unzip mon_application.zip


In the most simple cases, you can compile a given application in a few commands. The exact commands depend on the application's packaging, but are often similar. You should give close attention to the installation directory, that should be put inside your personal or group file space. Once you entered the directory containing the source code, you often find a file named README or INSTALL that contain installation instructions. Read those instructions. If the following subsections, we categorize various ways of compiling an application or library. Note that you must load a compiler in advance.

Make

The Make utility is frequently used to compile and application or library. If the source code directory contains a file named Makefile, then it is possible that the following command is sufficient to compile the source code :

[name@server $] make


Autoconf

The Autoconf package comprises the most used set of utilities to deploy applications in a portable fashion. If the directory contains a file named configure, it is most likely that Autoconf has been used. In that case, you should be able to compile and install your application simply using the command

[name@server $] ./configure --prefix="$HOME/myapplication/" && make && make install


Depending on the application's dependencies, you may need to load more libraries or even install (or ask to install) more yourself.

This command configures, compiles and installs the application in the directory $HOME/myapplication, so it is within your home folder.

Cmake

Cmake is another tool that is frequently used to deploy applications. If the source code contains a file name CMakeLists.txt, that is probably the tool that is needed to compile the application. If this is the case, you should probably use the following command:

[nom@serveur $] cmake -DCMAKE_INSTALL_PREFIX="$HOME/myapplication" . && make && make install


to configure, compile and install your application into the $HOME/myapplication/ directory.

Note : it is possible that you should load a module to make the command cmake available.

Creating a module

To know how to create your own modules, please refer to the page Creating a module.

Particularities for each server

The modules for Briarée and Hadès are shared. Please consult the list of modules available on Briarée and Hadès.
On Colosse, the directory lmodulefiles in your home directory ($HOME) as well as those in all directories under /rap/ to which you have access are automatically added to your $MODULEPATH, if they exist. Please consult the list of modules available on Colosse.
On Cottos, a file named $HOME/.modules.cottos determines your modules by default. Your default command shell, that could be tcsh, Bash, or another shell, will execute this script when you connect. The initlist, initadd and initrm command use this file.
The modules for Briarée and Hadès are shared. Please consult the list of modules available on Briarée and Hadès.
On Helios, the directory lmodulefiles in your home directory ($HOME) as well as those in all directories under /rap/ to which you have access are automatically added to your $MODULEPATH, if they exist. Please consult the list of modules available on Helios.
On Mammouth parallèle II, the automatic loading of modules has been preconfigured. In your account, the file $HOME/.modules_mp2 determines your default modules. The commands initlist, initadd and initrm will modify this file by changing the list of default modules. Note that these modifications will not alter the current session. You can consult list of modules available on Mp2.

Mammouth série II

Sur Mammouth série II, le chargement automatique de modules est préconfiguré. Dans votre compte, le fichier $HOME/.modules détermine vos modules par défaut. Les commandes initlist, initadd et initrm modifieront ce fichier de façon à modifier la liste de modules par défaut. Notez que ces modifications ne s'appliquent pas à la session courante.

Vous pouvez consulter la liste des modules disponibles sur Ms2.
The command module command is not available on Psi.


Outils personnels
Espaces de noms

Variantes
Actions
Navigation
Ressources de Calcul Québec
Outils
Partager