Communication and Information Centre (kiz)

Unit 1: Environment Modules System

 

This page: https://www.uni-ulm.de/?bwhpc-ex1

Here are the presentations/Info preceding this exercise:

shell commands cheat sheet

Introduction-talk

Module-talk

 

Documentation on module system: 

https://www.bwhpc-c5.de/wiki/index.php/Software_Modules

Overview on installed programms on the web:

https://cis-hpc.uni-konstanz.de/prod.cis/

Documentation on specific software modules e.g.:

https://www.bwhpc-c5.de/wiki/index.php/Mathematica

Login to bwunicluster or Justus with your username:

ssh -X  »username«@bwunicluster.kit.edu or 

ssh -X »username«@justus.uni-ulm.de

Commands overview:

  • module whatis [<software-module>]
  • module help <software-module>
  • module load <software-module>
  • module list
  • module unload  <software-module> // module purge
  • module show <software-module>

Note: if you do "module load system/easymodule", you can use the command "mod"  instead of "module" with the same syntax.  "mod" allows you to search for substrings in module names and displays results in a pager in which you can search with "/".  

Exercises/Questions:

Find out using the modules system:

1) Which versions of the intel compiler are available?
2) Which compiler software is available?
3) get the short description of the program chem/vmd
4) get the help for the module of the program chem/vmd

module avail compiler/intel
module avail compiler
module whatis chem/vmd
module help chem/vmd

Loading a module:

Run the following
matlab -nosplash -nodisplay -nodesktop
module list
module load math/matlab
module list
matlab -nosplash -nodisplay -nodesktop
quit
module unload math/matlab
module list

Program with man-pages

man icc
icc --version
echo $PATH
oldpath=$PATH
echo $MANPATH
module load compiler/intel
man icc
icc --version
echo $PATH
echo $MANPATH
echo $PATH | sed sX"$oldpath"XX # only shows the difference
Now compare to what the module does with the path (more below):
module show compiler/intel 2>&1 | grep ' PATH'
module unload compiler/intel
echo $PATH
echo $MANPATH

Compare with

module load chem/lammps
module list
module load compiler/intel/14.0
module list
module purge
module list

What is the problem loading compiler/intel/14.0?

You cannot load two different versions of the same program. Administrators installing the modules set a conflict to the same module in different versions and sometimes to other conflicting packages. Obviously, not all possible conflicts can be foreseen. Try unloading all unrelated modules if you encounter problems with a module/software.

Same problem:

module load compiler/intel/14.0
module list
module load chem/lammps
module list
module purge

Interaction with system tools

gcc --version
module load compiler/gnu
module list
gcc --version
module remove compiler/gnu
module list
gcc --version
 Seeing what a module does:
module show chem/vmd
 The PATH variable contains the places that are searched when you type a command on the shell. The module adds to (pre-pends) PATH. Pre-pending means the programs in the module will be found before other programs of the same name on the system
Complexity of the changes made by a module varies.
module show compiler/intel

Directory with HPC examples

module load math/matlab
cd $MATLAB_EXA_DIR
ls -la
head -5 README
cd
module purge

The module system/easymodule. Compare new "mod" command with regular behavior
module load system/easymodule
module avail # compare
mod avail #compare
module avail | grep matl
mod avail | grep matl
mod avail matlab

Get the file bashcourse.tgz with wget https://kaos.rz.uni-ulm.de/bashcourse.tgz and unpack it. The login node justus-login04 has the RedHat system numpy installed for this workshop. Connect to the node and run numpy_test.py. Now load the numpy module compiled with intel compiler and mkl and compare the execution times

Why might your result be inaccurate?