Skip to content

Extending the software stack

Tip

The EESSI-extend module makes it possible to build and install whole software stacks that are available to the user, group.

In a nutshell, EESSI-extend is a module that loads EasyBuild in an environment that is configured to install software to one specific location at a time. This location may be configured by the user, by setting one of the variables mentioned below.

Using the EESSI-extend module is a short and informative introduction to building software with EasyBuild on EESSI. It is recommended that new users first read that page before proceeding.

Note that a specific version of EESSI's compatibility must be selected when building software. This will default to the most recent release available on the system if no version number is supplied. For a quick overview of EESSI-extend, see the module's help.

ml help EESSI-extend/2023.06-easybuild

EESSI-extend for user installations

By default, the EESSI-extend module sets the EESSI_USER_INSTALL variable. This can be configured to point to any location on the filesystem, but by default it points to $HOME/eessi. The user will have full write permissions to this location, and can build out any software that they require.

Tip

You can define multiple locations. For instance, if you run out of space on the $HOME filesystem, just build additional software at EESSI_USER_INSTALL=$DATA/eessi.

EESSI-extend for project/group installations

Often you will want to install software in a way that it is usable for everyone within a research group. This can be done by setting up some shared storage, setting the variable EESSI_PROJECT_INSTALL to point there, loading EESSI-extend and building the software with easybuild.

Tip

If both EESSI_USER_INSTALL and EESSI_PROJECT_INSTALL are defined, both sets of installations are exposed, but new installations are created as user installations.

See our examples page for an example of how to install a software package for a group.

Advanced

It can sometimes be a challenge to find an easyconfig file that build right away. Part of the reason for this is that some toolchains are better supported than others in EESSI. Software built with a well supported toolchain will be quicker to build with EasyBuild, because it will be easier to resolve the dependencies from already installed software. Note that passing the -r or --robot flag to EasyBuild will cause it to automatically build and install any missing dependencies. Even though this is possible, the recommended approach is to try to align the dependencies with what is already available through the existing software repositories as much as is practical. This is not only because the software will be quicker to build, but also because there will likely be better support from the community if any extra hooks or patches are required to build the software.

Manually building software on top of EESSI

Warning

Even with the help of the buildenv module, this documentation is intended for "experts" in the runtime linker and it's behaviour, and most cases are untested. Any feedback on this topic is highly appreciated.

If one is not familiar with Easybuild or prefers building software in a more traditional way, this can be done via EESSI's buildenv module. This will also build the software using EESSI's libraries, and has the advantage that all the bundled modules are compatible with each other plus some useful settings.

The following two principles are best practices for extending/building on top of EESSI modules:

  • Use the least amount of modules possible (some are pulled as dependencies and might clash).
  • Declare precisely which module to use with the name and version module load HDF5/1.14.3-gompi-2023b instead of module load HDF5).
myname@n3001-007:~$ module load buildenv/default-foss-2023b
myname@n3001-007:~$ module list
Currently Loaded Modules:
  1) EESSI/2023.06
  2) GCCcore/13.2.0
  3) GCC/13.2.0
  4) numactl/2.0.16-GCCcore-13.2.0
  5) libxml2/2.11.5-GCCcore-13.2.0
  6) libpciaccess/0.17-GCCcore-13.2.0
  7) hwloc/2.9.2-GCCcore-13.2.0
  8) OpenSSL/1.1
  9) libevent/2.1.12-GCCcore-13.2.0
 10) UCX/1.15.0-GCCcore-13.2.0
 11) libfabric/1.19.0-GCCcore-13.2.0
 12) PMIx/4.2.6-GCCcore-13.2.0
 13) UCC/1.2.0-GCCcore-13.2.0
 14) OpenMPI/4.1.6-GCC-13.2.0
 15) OpenBLAS/0.3.24-GCC-13.2.0
 16) FlexiBLAS/3.3.1-GCC-13.2.0
 17) FFTW/3.3.10-GCC-13.2.0
 18) gompi/2023b
 19) FFTW.MPI/3.3.10-gompi-2023b
 20) ScaLAPACK/2.2.0-gompi-2023b-fb
 21) foss/2023b
 22) buildenv/default-foss-2023b

Info

All software built with that will link to the EESSI compilers and libraries.

Example: HBT-HERONS

myname@n3001-007:~$ git clone https://github.com/SWIFTSIM/HBT-HERONS.git && cd HBT-HERONS

myname@n3001-007:~$ mkdir build && cd build

myname@n3001-007:~$ module load buildenv/default-foss-2023b
myname@n3001-007:~$ module load CMake/3.27.6-GCCcore-13.2.0
myname@n3001-007:~$ module load HDF5/1.14.3-gompi-2023b

myname@n3001-007:~$ cmake ../ && make -j 128