From 46f652dda9164f9ccdf05633c6f260ccb2fa77dc Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA Date: Tue, 24 Dec 2024 15:40:20 +0000 Subject: [PATCH 1/7] updated container name and locations --- .../ContainerQuickstart.rst | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index 0607a232b..a4fcba8cc 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -66,7 +66,7 @@ Build the Container Level 1 Systems ^^^^^^^^^^^^^^^^^^ -On most Level 1 systems, a container named ``ubuntu20.04-intel-ue-1.4.1-srw-dev.img`` has already been built at the following locations: +On most Level 1 systems, a container named ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` has already been built at the following locations: .. list-table:: Locations of pre-built containers :widths: 20 50 @@ -77,7 +77,7 @@ On most Level 1 systems, a container named ``ubuntu20.04-intel-ue-1.4.1-srw-dev. * - Derecho [#fn]_ - /glade/work/epicufsrt/contrib/containers * - Gaea [#fn]_ - - /lustre/f2/dev/role.epic/containers + - /gpfs/f5/epic/world-shared/containers * - Hera - /scratch1/NCEPDEV/nems/role.epic/containers * - Jet @@ -97,20 +97,20 @@ Users can simply set an environment variable to point to the container: .. code-block:: console - export img=/path/to/ubuntu20.04-intel-ue-1.4.1-srw-dev.img + export img=/path/to/ubuntu22.04-intel-ue-1.6.0-srw-dev.img Users may convert the container ``.img`` file to a writable sandbox: .. code-block:: console - singularity build --sandbox ubuntu20.04-intel-srwapp $img + singularity build --sandbox ubuntu22.04-intel-srwapp $img When making a writable sandbox on Level 1 systems, the following warnings commonly appear and can be ignored: .. code-block:: console INFO: Starting build... - INFO: Verifying bootstrap image ubuntu20.04-intel-ue-1.4.1-srw-dev.img + INFO: Verifying bootstrap image ubuntu22.04-intel-ue-1.6.0-srw-dev.img WARNING: integrity: signature not found for object group 1 WARNING: Bootstrap image could not be verified, but build will continue. @@ -123,7 +123,7 @@ On non-Level 1 systems, users should build the container in a writable sandbox: .. code-block:: console - sudo singularity build --sandbox ubuntu20.04-intel-srwapp docker://noaaepic/ubuntu20.04-intel-srwapp:develop + sudo singularity build --sandbox ubuntu22.04-intel-srwapp docker://noaaepic/ubuntu22.04-intel-srwapp:develop Some users may prefer to issue the command without the ``sudo`` prefix. Whether ``sudo`` is required is system-dependent. @@ -132,13 +132,13 @@ Some users may prefer to issue the command without the ``sudo`` prefix. Whether .. code-block:: console - sudo singularity build --sandbox ubuntu20.04-intel-srwapp docker://noaaepic/ubuntu20.04-intel-srwapp:release-public-v2.2.0 + sudo singularity build --sandbox ubuntu22.04-intel-srwapp docker://noaaepic/ubuntu22.04-intel-srwapp:release-public-v2.2.0 For easier reference, users can set an environment variable to point to the container: .. code-block:: console - export img=/path/to/ubuntu20.04-intel-srwapp + export img=/path/to/ubuntu22.04-intel-srwapp .. _RunContainer: @@ -257,13 +257,13 @@ where: * ``-c`` indicates the compiler on the user's local machine (e.g., ``intel/2022.1.2``) * ``-m`` indicates the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``) * ```` refers to the local machine (e.g., ``hera``, ``jet``, ``noaacloud``, ``macos``, ``linux``). See ``MACHINE`` in :numref:`Section %s ` for a full list of options. - * ``-i`` indicates the container image that was built in :numref:`Step %s ` (``ubuntu20.04-intel-srwapp`` or ``ubuntu20.04-intel-ue-1.4.1-srw-dev.img`` by default). + * ``-i`` indicates the container image that was built in :numref:`Step %s ` (``ubuntu22.04-intel-srwapp`` or ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` by default). For example, on Hera, the command would be: .. code-block:: console - ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=ubuntu20.04-intel-ue-1.4.1-srw-dev.img + ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=ubuntu22.04-intel-ue-1.6.0-srw-dev.img .. attention:: From 0f3c10dee857661b9d8413749ad03be390a844e4 Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA Date: Tue, 24 Dec 2024 15:52:24 +0000 Subject: [PATCH 2/7] update container paths and names --- .../BuildingRunningTesting/ContainerQuickstart.rst | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index a4fcba8cc..b9d0217a7 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -90,7 +90,6 @@ On most Level 1 systems, a container named ``ubuntu22.04-intel-ue-1.6.0-srw-dev. .. [#fn] On these systems, container testing shows inconsistent results. .. note:: - * On Gaea, Singularity/Apptainer is only available on the C5 partition, and therefore container use is only supported on Gaea C5. * The NOAA Cloud containers are accessible only to those with EPIC resources. Users can simply set an environment variable to point to the container: @@ -103,7 +102,7 @@ Users may convert the container ``.img`` file to a writable sandbox: .. code-block:: console - singularity build --sandbox ubuntu22.04-intel-srwapp $img + singularity build --sandbox ubuntu22.04-intel-ue-1.6.0-srw-dev $img When making a writable sandbox on Level 1 systems, the following warnings commonly appear and can be ignored: @@ -123,7 +122,7 @@ On non-Level 1 systems, users should build the container in a writable sandbox: .. code-block:: console - sudo singularity build --sandbox ubuntu22.04-intel-srwapp docker://noaaepic/ubuntu22.04-intel-srwapp:develop + sudo singularity build --sandbox ubuntu22.04-intel-ue-1.6.0-srw-dev docker://noaaepic/ubuntu22.04-intel21.10-srw:ue160-fms202401-dev Some users may prefer to issue the command without the ``sudo`` prefix. Whether ``sudo`` is required is system-dependent. @@ -132,13 +131,13 @@ Some users may prefer to issue the command without the ``sudo`` prefix. Whether .. code-block:: console - sudo singularity build --sandbox ubuntu22.04-intel-srwapp docker://noaaepic/ubuntu22.04-intel-srwapp:release-public-v2.2.0 + sudo singularity build --sandbox ubuntu22.04-intel-ue-1.6.0-srw-dev docker://noaaepic/ubuntu20.04-intel-srwapp:release-public-v2.2.0 For easier reference, users can set an environment variable to point to the container: .. code-block:: console - export img=/path/to/ubuntu22.04-intel-srwapp + export img=/path/to/ubuntu22.04-intel-ue-1.6.0-srw-dev .. _RunContainer: @@ -257,13 +256,13 @@ where: * ``-c`` indicates the compiler on the user's local machine (e.g., ``intel/2022.1.2``) * ``-m`` indicates the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``) * ```` refers to the local machine (e.g., ``hera``, ``jet``, ``noaacloud``, ``macos``, ``linux``). See ``MACHINE`` in :numref:`Section %s ` for a full list of options. - * ``-i`` indicates the container image that was built in :numref:`Step %s ` (``ubuntu22.04-intel-srwapp`` or ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` by default). + * ``-i`` indicates the full path to the container image that was built in :numref:`Step %s ` (``ubuntu22.04-intel-ue-1.6.0-srw-dev`` or ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` by default). For example, on Hera, the command would be: .. code-block:: console - ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=ubuntu22.04-intel-ue-1.6.0-srw-dev.img + ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=$img .. attention:: From a2d708b680622fcecd389bd5879d5bcda9c087b4 Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA Date: Tue, 24 Dec 2024 15:56:46 +0000 Subject: [PATCH 3/7] update container dirs --- .../BuildingRunningTesting/ContainerQuickstart.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index b9d0217a7..c596e49e4 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -172,11 +172,11 @@ The list of directories printed will be similar to this: .. code-block:: console - bin discover lfs lib media run singularity usr - boot environment lfs1 lib32 mnt sbin srv var - contrib etc lfs2 lib64 opt scratch sys work - data glade lfs3 libx32 proc scratch1 tmp - dev home lfs4 lustre root scratch2 u + bin discover home lfs4 lustre root scratch2 tmp + boot environment lfs lib media run singularity u + contrib etc lfs1 lib32 mnt sbin srv usr + data glade lfs2 lib64 opt scratch sys var + dev gpfs lfs3 libx32 proc scratch1 third-party-programs.txt work Users can run ``exit`` to exit the shell. From 6e3f3018dff85e7d92b09c2a9528db924c3f4d16 Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA Date: Tue, 24 Dec 2024 20:16:53 +0000 Subject: [PATCH 4/7] reorganized workflow steps --- .../ContainerQuickstart.rst | 61 +++++++------------ 1 file changed, 21 insertions(+), 40 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index c596e49e4..2ebd56e63 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -200,77 +200,58 @@ Generate the Forecast Experiment ================================= To generate the forecast experiment, users must: -#. :ref:`Activate the workflow ` +#. :ref:`Stage the container ` #. :ref:`Set experiment parameters to configure the workflow ` #. :ref:`Run a script to generate the experiment workflow ` The first two steps depend on the platform being used and are described here for Level 1 platforms. Users will need to adjust the instructions to match their machine configuration if their local machine is a Level 2-4 platform. -.. _SetUpPythonEnvC: +.. _SetUpCont: -Activate the Workflow +Stage the container ------------------------ -Copy the container's modulefiles to the local working directory so that the files can be accessed outside of the container: +To set up the container with your host system, run ``stage-srw.sh`` script: .. code-block:: console - singularity exec -B /:/ $img cp -r /opt/ufs-srweather-app/modulefiles . - -After this command runs, the local working directory should contain the ``modulefiles`` directory. + ./stage-srw.sh -c= -m= -p= -i=$img -To activate the workflow, run the following commands: +where: -.. code-block:: console - - module use /path/to/modulefiles - module load wflow_ + * ``-c`` indicates the compiler on the user's local machine (e.g., ``intel/2022.1.2``) + * ``-m`` indicates the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``) + * ```` refers to the local machine (e.g., ``hera``, ``jet``, ``noaacloud``, ``macos``, ``linux``). See ``MACHINE`` in :numref:`Section %s ` for a full list of options. + * ``-i`` indicates the full path to the container image that was built in :numref:`Step %s ` (``ubuntu22.04-intel-ue-1.6.0-srw-dev`` or ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` by default). -where: +For example, on Hera, the command would be: - * ``/path/to/modulefiles`` is replaced with the actual path to the modulefiles on the user's local system (often ``$PWD/modulefiles``), and - * ```` is a valid, lowercased machine/platform name (see the ``MACHINE`` variable in :numref:`Section %s `). +.. code-block:: console -The ``wflow_`` modulefile will then output instructions to activate the workflow. The user should run the commands specified in the modulefile output. For example, if the output says: + ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=$img -.. code-block:: console +.. attention:: - Please do the following to activate conda: - > conda activate workflow_tools + The user must have an Intel compiler and MPI on their system because the container uses an Intel compiler and MPI. Intel compilers are now available for free as part of the `Intel oneAPI Toolkit `__. -then the user should run |activate|. This will activate the |wflow_env| conda environment. The command(s) will vary from system to system, but the user should see |prompt| in front of the Terminal prompt at this point. +After this command runs, the working directory should contain the ``srw.sh`` script and a ``ufs-srweather-app`` directory. -.. _SetUpConfigFileC: +.. _SetUpConfigFileC: Configure the Workflow --------------------------- -Run ``stage-srw.sh``: +Configuring the workflow for the container is similar to configuring the workflow without a container. The only exception is that there is no need to activate the ``srw_app`` conda environment. That is because there is a conflict between the container's conda and the host’s conda. To get around this, the container’s conda environment bin directory is appended to the system’s ``PATH`` variable in the ``python_srw.lua`` and ``build__intel.lua`` modulefiles with the ``stage-srw.sh`` script. Activate the workflow by running the following commands: .. code-block:: console - ./stage-srw.sh -c= -m= -p= -i=$img + module use ufs-srweather-app/modulefiles + module load wflow_ where: - * ``-c`` indicates the compiler on the user's local machine (e.g., ``intel/2022.1.2``) - * ``-m`` indicates the :term:`MPI` on the user's local machine (e.g., ``impi/2022.1.2``) - * ```` refers to the local machine (e.g., ``hera``, ``jet``, ``noaacloud``, ``macos``, ``linux``). See ``MACHINE`` in :numref:`Section %s ` for a full list of options. - * ``-i`` indicates the full path to the container image that was built in :numref:`Step %s ` (``ubuntu22.04-intel-ue-1.6.0-srw-dev`` or ``ubuntu22.04-intel-ue-1.6.0-srw-dev.img`` by default). - -For example, on Hera, the command would be: - -.. code-block:: console - - ./stage-srw.sh -c=intel/2022.1.2 -m=impi/2022.1.2 -p=hera -i=$img - -.. attention:: - - The user must have an Intel compiler and MPI on their system because the container uses an Intel compiler and MPI. Intel compilers are now available for free as part of the `Intel oneAPI Toolkit `__. - -After this command runs, the working directory should contain ``srw.sh``, a ``ufs-srweather-app`` directory, and an ``ush`` directory. + * ```` is a valid, lowercased machine/platform name (see the ``MACHINE`` variable in :numref:`Section %s `). -.. COMMENT: Check that the above is true for the dev containers... From here, users can follow the steps below to configure the out-of-the-box SRW App case with an automated Rocoto workflow. For more detailed instructions on experiment configuration, users can refer to :numref:`Section %s `. From 85c08fd39806683acf8d91f5bb1b72117649951a Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA Date: Tue, 24 Dec 2024 21:18:53 +0000 Subject: [PATCH 5/7] fix typos --- .../BuildingRunningTesting/ContainerQuickstart.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index 2ebd56e63..1077e5a16 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -82,9 +82,9 @@ On most Level 1 systems, a container named ``ubuntu22.04-intel-ue-1.6.0-srw-dev. - /scratch1/NCEPDEV/nems/role.epic/containers * - Jet - /mnt/lfs5/HFIP/hfv3gfs/role.epic/containers - * - NOAA Cloud + * - NOAA Cloud [#fn]_ - /contrib/EPIC/containers - * - Orion/Hercules [#fn]_ + * - Orion/Hercules - /work/noaa/epic/role-epic/contrib/containers .. [#fn] On these systems, container testing shows inconsistent results. @@ -131,7 +131,7 @@ Some users may prefer to issue the command without the ``sudo`` prefix. Whether .. code-block:: console - sudo singularity build --sandbox ubuntu22.04-intel-ue-1.6.0-srw-dev docker://noaaepic/ubuntu20.04-intel-srwapp:release-public-v2.2.0 + sudo singularity build --sandbox ubuntu20.04-intel-srwapp-release-public-v2.2.0 docker://noaaepic/ubuntu20.04-intel-srwapp:release-public-v2.2.0 For easier reference, users can set an environment variable to point to the container: @@ -208,10 +208,10 @@ The first two steps depend on the platform being used and are described here for .. _SetUpCont: -Stage the container +Stage the Container ------------------------ -To set up the container with your host system, run ``stage-srw.sh`` script: +To set up the container with your host system, run the ``stage-srw.sh`` script: .. code-block:: console From 4b5be461d639de1c31959e2fadcf055b66afd50a Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA <96196752+EdwardSnyder-NOAA@users.noreply.github.com> Date: Wed, 8 Jan 2025 11:52:34 -0600 Subject: [PATCH 6/7] Update ContainerQuickstart.rst update reference name --- doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index 1077e5a16..dee4773f6 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -366,7 +366,7 @@ where ``/path/to`` is replaced by the actual path to the user's experiment direc New Experiment =============== -To run a new experiment in the container at a later time, users will need to rerun the commands in :numref:`Section %s ` to reactivate the workflow. Then, users can configure a new experiment by updating the experiment variables in ``config.yaml`` to reflect the desired experiment configuration. Basic instructions appear in :numref:`Section %s ` above, and detailed instructions can be viewed in :numref:`Section %s `. After adjusting the configuration file, regenerate the experiment by running ``./generate_FV3LAM_wflow.py``. +To run a new experiment in the container at a later time, users will need to rerun the commands in :numref:`Section %s ` to reactivate the workflow. Then, users can configure a new experiment by updating the experiment variables in ``config.yaml`` to reflect the desired experiment configuration. Basic instructions appear in :numref:`Section %s ` above, and detailed instructions can be viewed in :numref:`Section %s `. After adjusting the configuration file, regenerate the experiment by running ``./generate_FV3LAM_wflow.py``. .. _appendix: From 560f4523f2db421646fa1547360a0f850184bf01 Mon Sep 17 00:00:00 2001 From: EdwardSnyder-NOAA <96196752+EdwardSnyder-NOAA@users.noreply.github.com> Date: Wed, 8 Jan 2025 12:11:10 -0600 Subject: [PATCH 7/7] Update ContainerQuickstart.rst fix names --- doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst index dee4773f6..c7762dec6 100644 --- a/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/ContainerQuickstart.rst @@ -366,7 +366,7 @@ where ``/path/to`` is replaced by the actual path to the user's experiment direc New Experiment =============== -To run a new experiment in the container at a later time, users will need to rerun the commands in :numref:`Section %s ` to reactivate the workflow. Then, users can configure a new experiment by updating the experiment variables in ``config.yaml`` to reflect the desired experiment configuration. Basic instructions appear in :numref:`Section %s ` above, and detailed instructions can be viewed in :numref:`Section %s `. After adjusting the configuration file, regenerate the experiment by running ``./generate_FV3LAM_wflow.py``. +To run a new experiment in the container at a later time, users will need to rerun the commands in :numref:`Section %s ` to reactivate the workflow. Then, users can configure a new experiment by updating the experiment variables in ``config.yaml`` to reflect the desired experiment configuration. Basic instructions appear in :numref:`Section %s ` above, and detailed instructions can be viewed in :numref:`Section %s `. After adjusting the configuration file, regenerate the experiment by running ``./generate_FV3LAM_wflow.py``. .. _appendix: