TensorFlow

De Wiki de Calcul Québec
Aller à : Navigation, rechercher
Cette page est une traduction de la page Tensorflow et la traduction est complétée à 90 % et à jour.
Les traductions désuètes sont marquées comme ceci.

Autres langues :anglais 90% • ‎français 100%

TensorFlow is an open source machine-learning tool developed by Google

This page contains the information necessary for the compilation and use of TensorFlow on the GPU clusters of Calcul Québec

Sommaire

How to Use on Guillimin

Listing the Available Versions

To see the available versions, enter the following command:

 
 [name@lg-1r14-n04 ~]$ module spider Tensorflow
 


Loading the Required Modules

These modules should be loaded before any use of TensorFlow:

 
 [name@lg-1r14-n04 ~]$ module load foss/2015b
 


Python 2

 
 [name@lg-1r14-n04 ~]$ module load Tensorflow/1.0.0-Python-2.7.12
 


Python 3

 
 [name@lg-1r14-n04 ~]$ module load Tensorflow/1.0.0-Python-3.5.2
 


How to Use on Helios

Listing the Available Versions

The most recent version of TensorFlow is made available in the weeks following its introduction. To see the available versions, enter the following command

 
 [name@helios ]$ ls /software-gpu/apps/python/wheelhouse/tensorflow-0.*
 


Loading the Required Modules

These modules should be loaded before any use of TensorFlow.

 
 [name@helios ]$ module load compilers/gcc/4.8.5 cuda/7.5 libs/cuDNN/5
 


Python 2

 
 [name@helios ]$ module load apps/python/2.7.10
 


Python 3

 
 [name@helios ]$ module load apps/python/3.5.0
 


Installing with pip

To install the latest available version, create and activate a virtual environment. Then enter the following command:

 
 [name@helios venv]$ pip install tensorflow
 


Testing the Tensorflow installation

 
 [name@helios venv]$ python -m tensorflow.models.image.mnist.convolutional
 


Compiling TensorFlow (Helios)

Detailed compilation instructions for Helios (v0.12.1)

Load the Necessary Modules

 
 [name@helios ]$ module purge -f
 [name@helios ]$ module load compilers/gcc/4.8.5 compilers/java/1.8 apps/buildtools \
                           cuda/7.5 libs/cuDNN/5 compilers/swig apps/git apps/bazel/0.4.3
 


Obtain the TensorFlow source and move to the source directory.

 
 [name@helios ]$ git clone https://github.com/tensorflow/tensorflow.git
 [name@helios ]$ cd tensorflow
 


3. Select the version of the code (checkout)

 
 [name@helios tensorflow]$ git checkout 0.12.1
 


Configuring the Compiler Options

 
 [nom@helios tensorflow]$ cp third_party/gpus/crosstool/CROSSTOOL.tpl{,.orig}
 [nom@helios tensorflow]$ GCC_PREFIX=$(dirname $(dirname $(which gcc)))
 [nom@helios tensorflow]$ sed -i "\;linker_flag: \"-B/usr/bin/\";a \
 > \ \ linker_flag: \"-Wl,-rpath=$GCC_PREFIX/lib64\"" third_party/gpus/crosstool/CROSSTOOL.tpl
 [nom@helios tensorflow]$ sed -i "\;linker_flag: \"-B/usr/bin/\";a \
 > \ \ linker_flag: \"-Wl,-rpath=$GCC_PREFIX/lib\"" third_party/gpus/crosstool/CROSSTOOL.tpl
 [nom@helios tensorflow]$ sed -i "s;-B/usr/bin;-B$(dirname $(which ar));g" third_party/gpus/crosstool/CROSSTOOL.tpl
 


Create a Python virtual environment for the compilation.

Python 2
 
 [name@helios tensorflow]$ module load apps/python/2
 [name@helios tensorflow]$ virtualenv buildenv
 


Python 3
 
 [name@helios tensorflow]$ module load apps/python/3
 [name@helios tensorflow]$ pyvenv buildenv
 


Activate the Virtual Environment

 
 [name@helios tensorflow]$ source buildenv/bin/activate
 


Install the Necessary Python Packages

 
 (buildenv)[name@helios tensorflow]$ pip install numpy wheel
 Collecting numpy
 Collecting wheel
 Installing collected packages: numpy, wheel
 


Adapt the Crosstool Wrapper Script and correct bug for this version

 
 (buildenv)[name@helios tensorflow]$ sed -i "/cmd = 'PATH=' + PREFIX_DIR + ' ' + cmd/s/^/#/" third_party/gpus/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc.tpl
 (buildenv)[name@helios tensorflow]$ sed -i 's;zlib-1.2.8.tar.gz;fossils/zlib-1.2.8.tar.gz;g' tensorflow/workspace.bzl # Bug in Zlib for Bazel 0.12.1
 


Configure the CUDA and cuDNN Paths

 
 (buildenv)[name@helios tensorflow]$ echo -e "\nN\nN\n\nN\nY\n\n7.5\n/software-gpu/cuda/7.5.18\n5\n/software-gpu/libs/cuDNN/5_cuda7.5\n3.5,3.7\n" | MALLOC_ARENA_MAX=4 ./configure
 # Python - default
 # with Google Cloud Platform: N
 # with HDFS: N
 # Python library path - default
 # OpenCL support: N
 # with GPU: Y
 # GCC - default
 # CUDA version: 7.5
 # CUDA path : /software-gpu/cuda/7.5.18
 # cuDNN : 5
 # cuDNN path : /software-gpu/libs/cuDNN/5_cuda7.5
 # Compute capability : 3.5,3.7
 


Compiling TensorFlow and the build_pip_package Application

 
 (buildenv)[name@helios tensorflow]$ MALLOC_ARENA_MAX=4 bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
 


Creating the wheel package for TensorFlow

 
 (buildenv)[name@helios tensorflow]$ mkdir dist
 (buildenv)[name@helios tensorflow]$ bazel-bin/tensorflow/tools/pip_package/build_pip_package $PWD/dist
 


The compiled TensorFlow wheel package will be in the dist sub-directory.

Quitting the Virtual Environment

 
 (buildenv)[name@helios tensorflow]$ deactivate
 


Stop the Bazel compilation server

 
 [name@helios tensorflow]$ bazel shutdown
 


Compilation Scripts

Guillimin-specific instructions

Using the Helios and [1] compilation notes.

Adjustments for v0.10.0

See /software/CentOS-6/tools/python/tensorflow-0.10.diff

Paths to /sb/software and /software are both necessary. For example:

  • cxx_builtin_include_directory: "/sb/software/CentOS-6/eb/software/Core/GCCcore/4.9.3/include"
  • cxx_builtin_include_directory: "/software/CentOS-6/eb/software/Core/GCCcore/4.9.3/include"
Outils personnels
Espaces de noms

Variantes
Actions
Navigation
Ressources de Calcul Québec
Outils
Partager