Module
From Yale HPC Wiki
Module is a package that lets users add or remove changes to your environment to use various applications or libraries. It also allows a user to lookup what applications are available on a cluster without doing an "ls" exploration. Currently module is only available on clusters bulldogj and newer. If you want it installed on another cluster please email us.
Contents |
Module Documentation
Module man pages are available using
man module
or
man modulefile
Basic commands are available by running just "module".
# module Modules Release 3.2.6 2007-02-14 (Copyright GNU GPL v2 1991): Usage: module [ switches ] [ subcommand ] [subcommand-args ] Switches: -H|--help this usage info -V|--version modules version & configuration options -f|--force force active dependency resolution -t|--terse terse format avail and list format -l|--long long format avail and list format -h|--human readable format avail and list format -v|--verbose enable verbose messages -s|--silent disable verbose messages -c|--create create caches for avail and apropos -i|--icase case insensitive -u|--userlvl <lvl> set user level to (nov[ice],exp[ert],adv[anced]) Available SubCommands and Args: + add|load modulefile [modulefile ...] + rm|unload modulefile [modulefile ...] + switch|swap [modulefile1] modulefile2 + display|show modulefile [modulefile ...] + avail [modulefile [modulefile ...]] + use [-a|--append] dir [dir ...] + unuse dir [dir ...] + update + refresh + purge + list + clear + help [modulefile [modulefile ...]] + whatis [modulefile [modulefile ...]] + apropos|keyword string + initadd modulefile [modulefile ...] + initprepend modulefile [modulefile ...] + initrm modulefile [modulefile ...] + initswitch modulefile1 modulefile2 + initlist + initclear
Listing available modules
Run the command "module avail". This list is constantly being updated.
[ch427@bulldogj ~]$ module avail ------------- /usr/local/cluster/software/module_3.2.6/Modules/3.2.6/modulefiles/Base -------------- 3.2.6 modules use_custom_modules module-info null yale_hpc --------- /usr/local/cluster/software/module_3.2.6/Modules/3.2.6/modulefiles/Applications ---------- NAMD mpis/openmpi/1.2.6_gcc_4 intel_compilers/10.1 mpis/openmpi_intel/1.2.6_Intel_10.1.015 mpis/mvapich2/1.0.3_gcc_4 octave/3.0.2 mpis/mvapich2_intel/1.0.3_Intel_10.1.015 ------------- /usr/local/cluster/software/module_3.2.6/Modules/3.2.6/modulefiles/Libs -------------- fftw/3.1.2-intel glpk-4.33 gsl/1.11 imsl/clibs/6.0 imsl/flibs/6.0 intel_integrated_performance_primitives/5.3.3.075 intel_mkl/10.0.3.020 intel_threading_building_blocks/2.0 nag_c/mark8 nag_fortran_gcc/mark21 nag_fortran_intel/mark21 nag_parallel_library/1.0 nag_pgi/mark21
Information on a module
For discovering what a module is run
module whatis <modulefile>
For discovering what a module does run
module help <modulefile>
For discovering exactly what a module does to your environment run
module display <modulefile>
Loading and removing modules
Modules can be added using the command
module load <modulefile>
Modules can be removed using the command
module unload <modulefile>
Swapping one module with another
If you want to unload one module and load another (to switch MPIs for example), you can run
module swap <loaded modulefile> <new modulefile>
Making the modules persistent
The load and unloading of modules is not persistent across logins. This means once you submit a PBS script or logout your environment goes back to when you first logged in. To make your changes "stick", you can add this to your PBS script (for just one application or run) or your .bashrc file(to make it permanent on all logins). Alternatively you can run the commands:
To add a modulefile permanently.
module initprepend <modulefile>
To remove a modulefile permanently.
module initrm <modulefile>
If you get an error like
ModuleCmd_Init.c(507):WARN:164: Cannot find a 'module load' command in any of the 'bash' startup files
Try running before the initadd
echo "module load null" >> ~/.bashrc
Module Errors, Conflicts and Dependencies
In some cases, modulefiles are written with dependencies or conflicts. For example, only one mpi application can be loaded at one time. This is intentional to reduce confusion about which mpi application is being used. In this case, the error will look like:
$ module load mpis/mvapich2/1.0.3_gcc_4 mpis/mvapich2/1.0.3_gcc_4(17):ERROR:150: Module 'mpis/mvapich2/1.0.3_gcc_4' conflicts with the currently loaded module(s) 'mpis/mvapich2_intel/1.0.3_Intel_10.1.015' mpis/mvapich2/1.0.3_gcc_4(17):ERROR:102: Tcl command execution failed: conflict mpis
Dependencies are also sometimes written into modulefiles. For example, the NAG intel fortran libraries require one type of intel compilers to be loaded. In this case, the error will look something like:
$ module load nag_fortran_intel/mark21 nag_fortran_intel/mark21(11):ERROR:151: Module 'nag_fortran_intel/mark21' depends on one of the module(s) 'intel_compilers/10.1' nag_fortran_intel/mark21(11):ERROR:102: Tcl command execution failed: prereq intel_compilers
Creating your own modules
A module called use_custom_modules will allow one to write custom modules in their home directory. Begin by adding the module.
module load use_custom_modules module initadd use_custom_modules
Now when you run "module avail" you will see <home>/privatemodules listed. There will also be a directory at ~/privatemodules where you can add your own modules. Start with the system modules as a model.
Using modules in qsub scripts
The module command is not passed to custom qsub scripts. To use module commands in your qsub script, you can either pass your current environment variables to qsub or source the module script in your qsub script.
You can pass your current shell variable to qsub using the "-V" flag:
qsub -V my_qsub_script
This will pass your current PATH, library path, include files, etc. to jobs submitted with qsub.
To make module commands available to qsub add these lines to your qsub script:
if [ -f /usr/local/cluster/configs/bashrc ];then source /usr/local/cluster/configs/bashrc fi
