From f27983a48dbe506dc75d19602cc1eee336ac5574 Mon Sep 17 00:00:00 2001 From: Alan O'Cais Date: Thu, 26 Sep 2024 14:44:48 +0200 Subject: [PATCH] Fix all broken links --- docs/appendix/terminology.md | 117 +------------------------------- docs/eessi/high-level-design.md | 8 +-- docs/eessi/inspiration.md | 4 +- docs/eessi/using-eessi.md | 10 +-- docs/eessi/what-is-eessi.md | 2 +- docs/index.md | 49 ++++--------- 6 files changed, 28 insertions(+), 162 deletions(-) diff --git a/docs/appendix/terminology.md b/docs/appendix/terminology.md index e0212a2..cbd0253 100644 --- a/docs/appendix/terminology.md +++ b/docs/appendix/terminology.md @@ -1,93 +1,12 @@ -# CernVM-FS Terminology +# Terminology -An overview of terms used in the context of CernVM-FS, in alphabetical order. - ---- - -## Catalog { #catalog } - -A **catalog** of a CernVM-FS [repository](#repository) is a table that lists files and directories along with the corresponding -metadata (sizes, timestamps, etc.). - -Catalogs can be *nested*: subtrees of the [repository](#repository) may have their own catalog. - -For more information on the *catalog* concept, see [the CernVM-FS -documentation](https://cvmfs.readthedocs.io/en/stable/cpt-details.html#file-catalog). - ---- - -## CernVM { #cernvm } - -**CernVM** is a virtual machine image based on CentOS combined with a custom, virtualization-friendly Linux kernel, -and which includes the [CernVM-FS](../cvmfs/what-is-cvmfs.md) client software. - -It is used for the CERN [Large Hadron Collider (LHC)](https://home.cern/science/accelerators/large-hadron-collider) -experiment, and was created to remove a need for the installation of the experiment software and to minimize the -number of platforms (compiler-OS combinations) on which experiment software needs to be supported and tested. - -While originally developed in conjunction, the [CernVM File System](#cvmfs) today is a product -that is completely independent from the CernVM virtual appliance. - -For more information on CernVM, see the [website](https://cernvm.cern.ch/appliance/) -and [documentation](https://cernvm.readthedocs.io). +An overview of terms used in the context of EESSI, in alphabetical order. --- ## CernVM-FS { #cvmfs } -*(see [What is CernVM-FS?](../cvmfs/what-is-cvmfs.md))* - ---- - -## Client { #client } - -A **client** in the context of CernVM-FS is a computer system on which a CernVM-FS [repository](#repository) -is being accessed, on which it will be presented as a [POSIX](https://en.wikipedia.org/wiki/POSIX) -read-only file system in a subdirectory of `/cvmfs`. - ---- - -## Proxy { #proxy } - -A **proxy**, also referred to as *squid proxy*, is a forward caching -[proxy server](https://en.wikipedia.org/wiki/Proxy_server) which acts as an intermediary between a CernVM-FS -[client](#client) and the [Stratum-1 replica servers](#stratum1). - -It is used to improve the latency observed when accessing the contents of a [repository](#repository), -and to reduce the load on the [Stratum-1 replica servers](#stratum1). - -A commonly used proxy is [Squid](http://www.squid-cache.org). - -For more information on proxies, see the -[CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-squid.html). - ---- - -## Publishing { #publishing } - -**Publishing** is the process of adding more files to a CernVM-FS [repository](#repository), -which is done via a *transaction* mechanism, and is on possible on the [Stratum-0 server](#stratum0), -via a [publisher](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#publisher-configuration), -or via a [repository gateway](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#repository-gateway-configuration). - -The workflow of publishing content is covered in detail in the -[CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-repository-gateway.html#publication-workflow). - ---- - -## Repository { #repository } - -A CernVM-FS **repository** is where the files and directories that you want to distribute via CernVM-FS are -stored, which usually correspond to a collection of software installations. - -It is a form of [*content-addressable storage (CAS)*](https://en.wikipedia.org/wiki/Content-addressable_storage), -and is the single source of (new) data for the file system being presented as a subdirectory of `/cvmfs` -on [client](#client) systems that [mount](https://en.wikipedia.org/wiki/Mount_(computing)) the repository. - -!!! note - - A CernVM-FS repository includes [**software installations**](#software-installations), - not [*software packages*](https://en.wikipedia.org/wiki/Package_format) like RPMs. +*(see [What is CernVM-FS?](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/))* --- @@ -107,33 +26,3 @@ in a single *package* to distribute as a whole. files that are part of that software installation. --- - -## Stratum 0 server { #stratum0 } - -A **Stratum 0 server**, often simply referred to a *Stratum 0* (Stratum Zero), is the central server -for one or more CernVM-FS [repositories](#repository). - -It is the single source of (new) data, since it hosts the *master copy* of the repository contents. - -Adding or updating files in a CernVM-FS repository ([publishing](#publishing)) can only be done on the Stratum 0 server, -either directly via the `cvmfs_server publish` command, or indirectory via a publisher server. - -For more information, see the [CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-repo.html). - ---- - -## Stratum 1 replica server { #stratum1 } - -A **Stratum 1 replica server**, often simply referred to a *Stratum 1* (Stratum One), is a standard web server -that acts as a mirror server for one or more CernVM-FS [repositories](#repository). - -It holds a complete copy of the data for each CernVM-FS repository it serves, -and automatically synchronises with the [Stratum 0](#stratum0). - -There is typically a *network* of several Stratum 1 servers for a CernVM-FS repository, -which are geographically distributed. - -[Clients](#client) can be configured to automatically connect to the closest Stratum 1 server by using -the CernVM-FS [GeoAPI](https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#ordering-of-servers-according-to-geographic-proximity). - -For more information, see the [CernVM-FS documentation](https://cvmfs.readthedocs.io/en/stable/cpt-replica.html). diff --git a/docs/eessi/high-level-design.md b/docs/eessi/high-level-design.md index 61e9a6b..6a9963f 100644 --- a/docs/eessi/high-level-design.md +++ b/docs/eessi/high-level-design.md @@ -6,8 +6,7 @@ and is aligned with the [motivation and goals](motivation-goals.md) of the proje In the remainder of this section of the tutorial, we will **explore the layered structure of the EESSI software stack**, and [how to use it](using-eessi.md). -In the [next section](../access/index.md) will cover in detail how you can get access to EESSI -(and other publicly available CernVM-FS repositories). +In the [next section](../access/index.md) will cover in detail how you can get access to EESSI. ## Layered structure @@ -26,8 +25,9 @@ the EESSI project consists of 3 layers, which are constructed by leveraging vari ### Filesystem layer {: #filesystem_layer } -The *filesystem layer* uses [**CernVM-FS**](../cvmfs/what-is-cvmfs.md) to distribute the EESSI software stack to -client systems. +The *filesystem layer* uses +CernVM-FS**](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/) to distribute the +EESSI software stack to client systems. As presented in the previous section, CernVM-FS is a mature open source software project that was created exactly for this purpose: diff --git a/docs/eessi/inspiration.md b/docs/eessi/inspiration.md index 63d9b2a..24b223c 100644 --- a/docs/eessi/inspiration.md +++ b/docs/eessi/inspiration.md @@ -4,8 +4,8 @@ The EESSI concept is heavily inspired by software stack provided by the [Digital Research Alliance of Canada](https://alliancecan.ca/en/about/alliance) (a.k.a. The Alliance, formerly known as *Compute Canada*), which is a shared software stack used on all [national host sites for Advanced Research Computing in Canada]( https://alliancecan.ca/en/services/advanced-research-computing/federation/national-host-sites) -that is distributed across Canada (and beyond) using [CernVM-FS](../cvmfs/what-is-cvmfs.md); -see also [here](../cvmfs/flagship-repositories.md#the-alliance). +that is distributed across Canada (and beyond) using +[CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/). EESSI is significantly more ambitious in its goals however, in various ways. diff --git a/docs/eessi/using-eessi.md b/docs/eessi/using-eessi.md index 557d14e..889a6b2 100644 --- a/docs/eessi/using-eessi.md +++ b/docs/eessi/using-eessi.md @@ -7,7 +7,9 @@ Let's break it down step by step. ## 0) Is EESSI available? -First, check whether the EESSI [CernVM-FS](../cvmfs/what-is-cvmfs.md) repository is available on your system. +First, check whether the EESSI +[CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/) repository is +available on your system. Try checking the contents of the `/cvmfs/software.eessi.io` directory with the `ls` command: @@ -126,12 +128,10 @@ $ command -v python You may notice a bit of "lag" initially when starting to use software provided by the EESSI software layer. This is expected, since CernVM-FS may need to first download the files that are required - to run the software you are using; - see also [On-demand downloading of files and metadata](../cvmfs/what-is-cvmfs.md#features-ondemand). + to run the software you are using. You should not observe any significant start-up delays anymore when running the same software shortly after, - since then CernVM-FS will be able to serve the necessary files from the local client cache; - see also [Multi-level caching](../cvmfs/what-is-cvmfs.md#features-caching). + since then CernVM-FS will be able to serve the necessary files from the local client cache. --- diff --git a/docs/eessi/what-is-eessi.md b/docs/eessi/what-is-eessi.md index 5135796..d3ef62a 100644 --- a/docs/eessi/what-is-eessi.md +++ b/docs/eessi/what-is-eessi.md @@ -9,7 +9,7 @@ is a collaboration between different European partners in the HPC (High Performa EESSI provides a common stack of *optimized* scientific software installations that work on any Linux distribution, and currently supports both `x86_64` (AMD/Intel) and `aarch64` (Arm 64-bit) systems, which is distributed -via [CernVM-FS](../cvmfs/what-is-cvmfs.md). +via [CernVM-FS](https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/cvmfs/what-is-cvmfs/). --- diff --git a/docs/index.md b/docs/index.md index 44a528e..394a185 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,47 +5,39 @@

-This is an introductory tutorial to [CernVM-FS](https://cernvm.cern.ch/fs/), the CernVM File System, +This is an introductory tutorial to [EESSI](https://www.eessi.io/docs/), the European Environment +for Scientific Software Installations, with a focus on employing it in the context of High-Performance Computing (HPC). -In this tutorial you will learn what CernVM-FS is, how to get access to existing CernVM-FS repositories, -how to configure CernVM-FS, and how to use CernVM-FS [repositories](appendix/terminology.md#repository) +In this tutorial you will learn what EESSI is, how to get access to EESSI, +how to customise EESSI, and how to use EESSI [repositories](appendix/terminology.md#repository) on HPC infrastructure.
-**[Ready to go? Click here to start the tutorial!](cvmfs/what-is-cvmfs.md)** +**[Ready to go? Click here to start the tutorial!](eessi/what-is-eessi.md)**
## Recording -A first virtual edition of this tutorial was held on 4 December 2023, -the recording is available here: - -

- - -
-slides (PDF) available for download here -

+Once we have a recording of this tutorial available it will appear here. ### Slides -[Available for download here](https://raw.githubusercontent.com/multixscale/cvmfs-tutorial-hpc-best-practices/main/files/Best-Practices-for-CernVM-FS-in-HPC-20231204.pdf) +Once we have slides for this tutorial available they will appear here. ## Intended audience -This tutorial is intended for people with a background in HPC (system administrators, support team members, -end users, etc.) and who are new to CernVM-FS; no specific prior knowledge or experience with it is required. +This tutorial is intended for a general audience who are familiar with running sofware from the command line; +no specific prior knowledge or experience is required. -We expect it to be most valuable to people who are interested in using or providing access to one or more existing -CernVM-FS repositories on HPC infrastructure. +We expect it to be most valuable to people who are interested in running scientific software on variety of +compute infrastructures. ## Prerequisites - Basic knowledge of Linux shell environment -- Basic knowledge of networking (IP address, port, latency) - Basic knowledge of Linux file systems - Familiarity with High-Performance Computing environments is a plus - Hands-on experience with running scientific software workloads is a plus @@ -53,15 +45,9 @@ CernVM-FS repositories on HPC infrastructure. ## Practical information -### Registration - -Attendance is free, but **registration is required**: . - -Registration for online tutorial on Mon 4 Dec 2023 is **closed** *(since Sun 3 Dec 2023)* - ### Slack channel -Dedicated channel in EESSI Slack: [`#cvmfs-best-practices-hpc`](https://eessi-hpc.slack.com/archives/C068DV7GY3V) +Dedicated channel in EESSI Slack: [`#eessi-tutorial`](https://eessi-hpc.slack.com/archives/C068ACS7XK8) [Click here to join the EESSI Slack](https://join.slack.com/t/eessi-hpc/shared_invite/zt-1wqy0t8g6-PZJTg3Hjjm5Fm3XEOkzECg) @@ -81,22 +67,13 @@ Undertaking (JU)](https://eurohpc-ju.europa.eu) and countries participating in t ## Contributors -* Jakob Blomer (CERN, Switzerland) * Bob Dröge (University of Groningen, The Netherlands) * Kenneth Hoste (Ghent University, Belgium) * Alan O'Cais (University of Barcelona, Spain; CECAM) * Lara Peeters (Ghent University, Belgium) -* Laura Promberger (CERN, Switzerland) * Thomas Röblitz (University of Bergen, Norway) * Caspar van Leeuwen (SURF, The Netherlands) -* Valentin Völkl (CERN, Switzerland) - ## Additional resources -* [CernVM-FS website](https://cernvm.cern.ch/fs) -* [CernVM-FS documentation](https://cvmfs.readthedocs.io) -* [CernVM-FS @ GitHub](https://github.com/cvmfs) -* [CernVM-FS forum](https://cernvm-forum.cern.ch) -* [Introduction to CernVM-FS by *Jakob Blomer (CERN)* (2021)](https://easybuild.io/eum21/#cvmfs-talk) -* [Introductory tutorial on CernVM-FS (2021)](https://cvmfs-contrib.github.io/cvmfs-tutorial-2021) +* TBD