Enable ufs-weather-model on Gaea-C6

[RatkoVasic-NOAA]

Description

Gaea C6 nodes are now available. We should enable ufs-weather-model

Solution

Enable ufs-weather-model for Gaea-C6

Alternatives

None

Related to

Depends on spack-stack installation, which is ready on C6.

[GeorgeVandenberghe-NOAA]

I have built and run it with my own software stack on Gaea C6. Kicked off, can no longer run by policy but this is a datapoint

[jkbk2004]

@BrianCurtis-NOAA Sounds a version is available at /ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0

[BrianCurtis-NOAA]

@BrianCurtis-NOAA Sounds a version is available at /ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0

Thanks! I'll add that path, do you know if there will be a modulefiles associated with that c6 stack?

[RatkoVasic-NOAA]

@BrianCurtis-NOAA the only current problem is that we don't have disk space on C6 machine (f6 filesystem) for staged data (regression tests...). So, yes, you can test /ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0 installation, but don't know where to stage RT data.

[RatkoVasic-NOAA]

We have ready spack-stack 1.6.0 on Gaea-C6:

/ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0/envs/unified-env/install/modulefiles/Core
/ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0/envs/gsi-addon/install/modulefiles/Core
/ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0/envs/upp-addon-env/install/modulefiles/Core
/ncrc/proj/epic/spack-stack/c6/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core

Note changes comparing to Gaea-C5:

C5:
stack-intel/2023.1.0
stack-cray-mpich/8.1.25
C6:
stack-intel/2023.2.0
stack-cray-mpich/8.1.29

[AnnaSmoot-NOAA]

I'm trying to build SRW app on Gaea C6. I've made the changes to my build_gaea_intel.lua file as shown in the previous post for spack, intel, and cray-mpich, but I still get an error for intel. I see there is also:

/ncrc/proj/epic/spack-stack/latest-ue-intel/stack-intel/2023.1.0.lua

Should I be using this path? Or do you know of some other way to fix stack-intel/2023.2.0 being unknown?

I also have some other modules that are unknown:
"jasper/2.0.32" "netcdf-c/4.9.2" "wgrib2/2.0.8" "libpng/1.6.37" "cmake/3.23.1"

Does anyone have recommendations for these?

[RatkoVasic-NOAA]

@AnnaSmoot-NOAA can you please share with me your moduolefile directory, and which modulefile you are using? I can take a look.

[AnnaSmoot-NOAA]

@RatkoVasic-NOAA Thank you for your help! I'm using this .lua at this path:

/gpfs/f6/drsa-fire2/proj-shared/Anna.Smoot/srw_app_develop/ufs-srweather-app/modulefiles/build_gaea_intel.lua

[RatkoVasic-NOAA]

I have no access to /gpfs/f6/drsa-fire2/proj-shared/
could you please copy your modulefile/ directory to /gpfs/f6/drsa-fire2/world-shared/
Thanks.

[AnnaSmoot-NOAA]

@RatkoVasic-NOAA That makes sense. It's here now:

/gpfs/f6/drsa-fire2/world-shared/Anna.Smoot/build_gaea_intel.lua

[RatkoVasic-NOAA]

@AnnaSmoot-NOAA I needed whole modulefile/ directory (or just srw_common.lua file).

[RatkoVasic-NOAA]

@AnnaSmoot-NOAA , there are two changes you need to do in build_gaea_intel.lua:

< stack_intel_ver=os.getenv("stack_intel_ver") or "2023.2.0"
---
> stack_intel_ver=os.getenv("stack_intel_ver") or "2023.1.0"
12c12
< stack_mpich_ver=os.getenv("stack_mpich_ver") or "8.1.29"
---
> stack_mpich_ver=os.getenv("stack_mpich_ver") or "8.1.25"

Try it now, and if it doesn't work, send me your srw_common.lua file.

[AnnaSmoot-NOAA]

@RatkoVasic-NOAA, I updated the build_gaea_intel.lua file and the errors changed a little:
The following module(s) are unknown: "PrgEnv-intel/8.3.3" "intel-classic/2023.1.0" "craype/2.7.20"

The updated build_gaea_intel and srw_common luas are at: /gpfs/f6/drsa-fire2/world-shared/Anna.Smoot/

[RatkoVasic-NOAA]

@AnnaSmoot-NOAA
you can take corrected build_gaea_intel.lua form here:
/ncrc/home2/Ratko.Vasic/Anna/modulefiles/build_gaea_intel.lua
That one is using old unified-env environment (should work).

If you want to use new environment (upp-addon-env), then take both files:
/ncrc/home2/Ratko.Vasic/Anna/test/modulefiles/build_gaea_intel.lua /ncrc/home2/Ratko.Vasic/Anna/test/modulefiles/srw_common.lua

[AnnaSmoot-NOAA]

@RatkoVasic-NOAA Thanks so much! I used the new environment and successfully built the SRW app.

[junwang-noaa]

@jkbk2004 May I ask if you have a time line when the UFS weather model will be ported to C6?

[jkbk2004]

@jkbk2004 May I ask if you have a time line when the UFS weather model will be ported to C6?

@junwang-noaa Internal C6 on-boarding session continues toward end of September. I think it's a reasonable to set a target migration around second week of October.

[GeorgeVandenberghe-NOAA]

I can build it with my offline stack but have encountered evidence of subtle library issues that prevent running it. I would need a testcase to validate the built result. THe issue I am running into is on Hercules with PIO in MOM6 causing file write failure aborts, reported in a separate thread and since it's my offline stack, not relevant to create a GFS issue about it. It runs fine on hercules when built with the admin stack.

…

On Thu, Sep 19, 2024 at 2:19 PM JONG KIM ***@***.***> wrote: @jkbk2004 <https://github.com/jkbk2004> May I ask if you have a time line when the UFS weather model will be ported to C6? @junwang-noaa <https://github.com/junwang-noaa> Internal C6 on-boarding session continues toward end of September. I think it's a reasonable to set a target migration around second week of October. — Reply to this email directly, view it on GitHub <#2407 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANDS4FU53DOD2QZ4SWAA36LZXLMQHAVCNFSM6AAAAABNAJMWOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRRGEZDMMJUGE> . You are receiving this because you commented.Message ID: ***@***.***>

-- George W Vandenberghe *Lynker Technologies at * NOAA/NWS/NCEP/EMC 5830 University Research Ct., Rm. 2141 College Park, MD 20740 ***@***.*** 301-683-3769(work) 3017751547(cell)

[RatkoVasic-NOAA]

@junwang-noaa , what is missing is common disk space for C6 (for ICs and baselines RT data). We don't have yet space on /f6 file system (like we had on /f5). @ulmononian is working with admins to provide us with that space. As soon as we get it, we can enable ufs-WM on that machine. Libraries are up-to-date here.