• Contenu
• Install the platform without MAGMA
  • Hardware requirements
  • Environment requirements
  • Installation process
• Install the platform with MAGMA
  • Why MAGMA ?
  • Configure MAGMA with ATLAS
    • Dependencies : blas, lapack, atlas
    • extraction
    • configuration
  • Configure MAGMA with MKL
    • extraction
    • configuration
  • compilation and installation
    • compilation
    • installation
    • tune (not tested)
  • Platform installation

Install the platform without MAGMA¶

The COMPASS platform is distributed as a single bundle of CArMA and SuTrA libraries and YoGA and its AO extension for Yorick.

Hardware requirements¶

The system must contain at least an x86 CPU and a CUDA capable GPU. list of compatible GPUs can be found here http://www.nvidia.com/object/cuda_gpus.html. Specific requirements apply to clusters (to be updated).

Environment requirements¶

The system must be running a 64 bit distribution of Linux or Mac OS with the latest NVIDIA drivers and CUDA toolkit. The installation of the corresponding version of the CULA tools is also required. The following installation instructions are valid if the default installation paths have been selected for these components.

Additionally, to benefit from the user-oriented features of the platform, Yorick should be installed as well as the latest version of Python and the associated pygtk module.

To install Yorick, download the latest version from the github repository:

git clone https://github.com/dhmunro/yorick.git yorick.git

then cd onto the created directory and install:

./configure && make && make install

once Yorick is locally installed, you will have to add this directory : yorick.git/relocate/bin to your PATH to have an easy access to the yorick executable. You may want to add support for command history by using rlwrap and alias the yorick executable as :

alias yorick='rlwrap path_to_yorick_executable/yorick'

Installation process¶

First check out the latest version from the svn repository :

svn co https://version-lesia.obspm.fr/repos/compass compass

then go in the newly created directory and then trunk:

cd compass/trunk

once there, you need to modify system variables in the define_var.sh executable :

emacs define_var.sh

in this file define properly CUDA_ROOT, CULA_ROOT and YoGA path. Note that for the latter, as YoGA is distributed with SUTrA you should just point to the newly created trunk directory. On a Linux system you should normally have:

export CUDA_ROOT=/usr/local/cuda
export CULA_ROOT=/usr/local/cula
export YOGA_DIR=/home/MyUserName/path2compass/trunk

in this file, you also have to indicate the proper architecture of your GPU so as the compiler will generate the appropriate code. Modify the following line:

export GENCODE="arch=compute_12,code=sm_12" 

and change both 12 to your architecture : for instance a Tesla Fermi will have 3.0 computing capabilities so change 12 to 30, a Kepler GPU will have 3.5 computing capabilities, change 12 to 35

Once this is done, you're ready to compile the whole library. First run define_var.sh to define the system variables that will be used during the compilation process:

./define_var.sh

then identify the absolute path to your Yorick executable using:

which yorick

and run the compilation script:

./reinstall absolute_path_to_yorick

If you did not get any error, CArMA, SuTrA and YoGA are now installed on your machine. You can check that everything is working by launching a GUI to test a simulation:

yorick -i yoga_ao/ywidgets/widget_ao.i

Install the platform with MAGMA¶

Why MAGMA ?¶

The MAGMA project aims to develop a dense linear algebra library similar to LAPACK but for heterogeneous/hybrid architectures, starting with current "Multicore+GPU" systems.

Unlike CULA, MAGMA propose a dense linear algebra library handling double for free.

But MAGMA needs a LAPACK and a BLAS implementation. Actually, we try two options : ATLAS BLAS (free, easy to install) and MKL (free, need a registration but more powerful)

Configure MAGMA with ATLAS¶

Dependencies : blas, lapack, atlas¶

Use your package manager to install dependencies:

• on scientific linux : yum install blas-devel lapack-devel atlas-devel
• on debian : apt-get install libblas-dev liblapack-dev libatlas-base-dev libatlas-dev

extraction¶

MAGMA is available here : http://icl.cs.utk.edu/magma/software/index.html

extract the tgz file and go into the new directory

_{$ tar xf magma-1.4.1-beta.tar.gz}
$ cd magma-1.4.1

configuration¶

You have to create your own make.inc :

• example on a scientific linux : please verify GPU_TARGET, LAPACKDIR, ATLASDIR, CUDADIR

#//////////////////////////////////////////////////////////////////////////////
#   -- MAGMA (version 1.4.1) --
#      Univ. of Tennessee, Knoxville
#      Univ. of California, Berkeley
#      Univ. of Colorado, Denver
#      November 2013
#//////////////////////////////////////////////////////////////////////////////

# GPU_TARGET specifies for which GPU you want to compile MAGMA:
#     "Tesla"  (NVIDIA compute capability 1.x cards)
#     "Fermi"  (NVIDIA compute capability 2.x cards)
#     "Kepler" (NVIDIA compute capability 3.x cards)
# See http://developer.nvidia.com/cuda-gpus

GPU_TARGET ?= Fermi

CC        = gcc
NVCC      = nvcc
FORT      = gfortran

ARCH      = ar
ARCHFLAGS = cr
RANLIB    = ranlib

OPTS      = -fPIC -O3 -DADD_ -fopenmp -DMAGMA_SETAFFINITY
F77OPTS   = -fPIC -O3 -DADD_
FOPTS     = -fPIC -O3 -DADD_ -x f95-cpp-input
NVOPTS    =       -O3 -DADD_ -Xcompiler "-fno-strict-aliasing -fPIC" 
LDOPTS    = -fPIC -fopenmp

# Depending on how ATLAS and LAPACK were compiled, you may need one or more of:
LIB       = -llapack -lf77blas -latlas -lcblas -lcublas -lcudart -lstdc++ -lm -lgfortran

# define library directories here or in your environment
LAPACKDIR = /usr/lib64
ATLASDIR  = /usr/lib64/atlas
CUDADIR   = /usr/local/cuda

LIBDIR    = -L$(LAPACKDIR) \
            -L$(ATLASDIR) \
            -L$(CUDADIR)/lib64

INC       = -I$(CUDADIR)/include

• on debian : please verify GPU_TARGET, LAPACKDIR, ATLASDIR, CUDADIR (TBD)

Configure MAGMA with MKL¶

extraction¶

To download MKL, you have to create a account here : https://registrationcenter.intel.com/RegCenter/NComForm.aspx?ProductID=1517

extract l_ccompxe_2013_sp1.1.106.tgz and go into l_ccompxe_2013_sp1.1.106

install it with ./install_GUI.sh and add IPP stuff to default choices

configuration¶

Just copy make.inc.mkl-gcc to make.inc please verify GPU_TARGET, MKLDIR, CUDADIR

compilation and installation¶

compilation¶

just compile the shared target (and test if you want)

~$ make -j 8 shared

installation¶

To install libraries and include files in a given prefix, run:

~$ make install prefix=/usr/local/magma

The default prefix is /usr/local/magma. You can also set prefix in make.inc.

tune (not tested)¶

For multi-GPU functions, set $MAGMA_NUM_GPUS to set the number of GPUs to use.
For multi-core BLAS libraries, set $OMP_NUM_THREADS or $MKL_NUM_THREADS or $VECLIB_MAXIMUM_THREADS to set the number of CPU threads, depending on your BLAS library.

Platform installation¶

Just just define $MAGMA_PATH and use the standard procedure