Skip to content

Spack

Jump to bottom
Harshula Jayasuriya edited this page Jan 30, 2024 · 26 revisions

Architecture Specifiers

https://spack.readthedocs.io/en/latest/basic_usage.html#architecture-specifiers

ACCESS-OM2 Docker Container (x86)

Download the Spack development Dockerfile and then build the dev image by running:

docker build -f Dockerfile.base-spack -t spack-dev:20230915 --target=dev  .

It contains the relevant Intel Compilers used to build ACCESS-OM2.

Environment Variables

Run the following command in the Spack source directory to find the environment variables read by Spack:

grep -r getenv lib/*
grep -r 'in os.environ' lib/*

Mirror

A Spack mirror contains the source code for Spack packages. On Gadi, a compute node does not allow Spack to download the source code from the Internet. A Spack mirror allows building Spack packages on a Gadi compute node. More details are available at https://spack.readthedocs.io/en/latest/mirrors.html . Below are instructions on how to create a Spack mirror containing all the source code required to build a particular package.

How to create a Spack mirror:

spack spec PACKAGE_NAME > ./mirror.list
spack mirror create --file ./mirror.list

How to add a Spack mirror to a Spack repository

spack mirror add local_filesystem file://FULLPATH_TO_MIRROR_DIRECTORY

OpenMPI

The OpenMPI SPD contains the mapping of MPI standard to OpenMPI implementation:

    provides("mpi")
    provides("mpi@:2.2", when="@1.6.5")
    provides("mpi@:3.0", when="@1.7.5:")
    provides("mpi@:3.1", when="@2.0.0:")

Reverse Dependencies

$ spack dependents -t oasis3-mct
access-om2  cice5  libaccessom2  mom5

Schemas

The schemas for the configuration files are located at lib/spack/spack/schema/ in the Spack source directory.

Spack Environments

Spack environments are a way to package up relevant spack packages, their configurations and compilers into a file, allowing a quick install of all the relevant packages into an isolated environment, in any spack installation. This isolated environment can take packages and compilers that are already installed in the global spack environment, allowing some caching to take place.

The Environment Files

The files that generate or represent an environment are broken down further: into abstract spack.yaml files, and concretized spack.lock files.

These can be found in spack/var/spack/environments/ENV_NAME/.

The spack.yaml file (spec file)

The spack.yaml file is an abstract list of packages, configurations of said packages, compilers that packages will use, among many other things. Specific versions of dependencies do not have to be specified, and in fact the dependencies do not need to be specified at all, unless further customization is required.

It looks like this:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - intel-oneapi-compilers@2021.2.0 arch=linux-rocky8-x86_64
  - mom5@master%intel@2021.2.0 arch=linux-rocky8-x86_64
  view: true
  concretizer:
    unify: true
  compilers:
  - compiler:
      spec: intel@=2021.2.0
      paths:
        cc: /opt/spack/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/intel-oneapi-compilers-2021.2.0-lkzqdmlrys3wqeb3yakfwi3slmyvccf6/compiler/2021.2.0/linux/bin/intel64/icc
        cxx: /opt/spack/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/intel-oneapi-compilers-2021.2.0-lkzqdmlrys3wqeb3yakfwi3slmyvccf6/compiler/2021.2.0/linux/bin/intel64/icpc
        f77: /opt/spack/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/intel-oneapi-compilers-2021.2.0-lkzqdmlrys3wqeb3yakfwi3slmyvccf6/compiler/2021.2.0/linux/bin/intel64/ifort
        fc: /opt/spack/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/intel-oneapi-compilers-2021.2.0-lkzqdmlrys3wqeb3yakfwi3slmyvccf6/compiler/2021.2.0/linux/bin/intel64/ifort
      flags: {}
      operating_system: rocky8
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
...

It contains what packages need to be installed (mom5), their versions (master) as well as further configuration options (arch=linux-rocky8-x86_64). Furthermore, it notes what compilers are in this environment (intel@=2021.2.0). Compared to a lock file, it is fairly human readable, and can be edited either manually or (as noted later) built up through spack install calls as you would in the global environment.

It might look familiar if you've seen or edited files like ~/.spack/linux/compilers.yaml - it is the same, in fact, but is not tied to the global environment, but it's own isolated one.

If you activate an environment that was based on a spack.yaml file, it will need to be concretized with specific versions and dependencies at install time. If you want to see the specifics of the concretized spack.yaml file, you can see the newly-generated spack.lock file. Read on below...

The spack.lock file (lock file)

The spack.lock file is a concretized version of the spack.yaml file, essentially a reusable, replicable, moment-in-time snapshot of the decisions made by spack in determining what versions of dependencies to use. If you create an environment based on this file, it will nessecarily be exactly the same as where ever you took it from - this is not the case for spack.yaml files!

This file is not very human readable, but we can glean some information from it - mostly exact commit hashes of packages, their dependencies, and so on, as well as any configuration details.

Generating your own environment file

Generating your own environment files isn't complicated. It's exactly as it is done outside of an environment, but with some extra command-line flags.

spack env create my-env  # create an empty environment
spack env activate my-env  # activate this environment (can also use 'spacktivate my-env')
# if you do 'spack env status' you will note that you are in the environment. This info is also present when doing 'spack find'

Now you can install compilers, packages and other things as normal, with said flags...

spack install --add cool-compiler@v1.2.3  # this '--add's the install of the compiler to the spack.yaml file 
spack compiler find --scope env:my-env  # this adds any newly installed compilers to the environment (and the spack.yaml file)
spack install --add cool-package@master%cool-compiler@v1.2.3  # packages work similarly

If you cat spack/var/spack/environments/my-env/spack.yaml you can verify that these packages and compilers have been added as expected. Likewise for the spack.lock file.

Generating an environment based on a environment file

If you have an environment file, it's even simpler:

spack env create my-cool-env ./some-cool.spack.yaml  # this could also be a path to a lock file if you didn't want to concretize at install time
spack env activate my-cool-env  # verify with 'spack env status', if you want
spack install  # what to install is taken from the environment file definition!

Read more on environments in spacks own documentation: https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html and https://spack.readthedocs.io/en/latest/environments.html

Clone this wiki locally