Getting Started

Installing SWELL¶

The SWELL documentation describes these steps in greater detail. This section provides a high-level summary of the steps needed to install and run SWELL.

Install uv first if it is not already available. On Discover, consider moving ~/.cache to $NOBACKUP because the uv cache can grow quickly. See the SWELL docs for offline or C shell installation alternatives.

Start by loading the modules required by SWELL and JEDI:

source /discover/nobackup/projects/gmao/advda/swell/jedi_modules/spackstack_1.9_intel

Next, clone the SWELL repository. On Discover, we recommend working in $NOBACKUP:

cd $NOBACKUP
mkdir swell-project
cd swell-project
git clone https://github.com/GEOS-ESM/swell.git swell-develop

Create a Python virtual environment and install SWELL in it:

uv venv
source .venv/bin/activate
cd swell-develop
# Make sure the environment is active before installation.
uv pip install -e .

Configuring Cylc on Discover¶

Run mkdir -p "$HOME/.cylc/flow", then create $HOME/.cylc/flow/global.cylc with:

[scheduler]
  UTC mode = True
  process pool timeout = PT10M
  process pool size = 4

[platforms]
  [[nccs_discover_cascade]]
    job runner = slurm
    install target = localhost
    hosts = localhost
  [[nccs_discover_sles15]]
    job runner = slurm
    install target = localhost
    hosts = localhost

Current SWELL configures the Cylc executable automatically on Discover; an old ~/bin/cylc wrapper should be removed or renamed. See Configuring Cylc on Discover.

Setting up R2D2 credentials¶

GEOS-CF experiments rely on R2D2 to fetch and store input and output files. Each user has their own R2D2 credentials. Run mkdir -p ~/.swell, then create ~/.swell/r2d2_credentials.yaml:

user: your_r2d2_username
api_key: your_r2d2_apikey

chmod 600 ~/.swell/r2d2_credentials.yaml

SWELL determines the Discover host and compiler automatically. Environment variables override the file; for the GMAO-supported GEOS-CF server (this is currently in testing phase), use:

export R2D2_USER="your_r2d2_username"
export R2D2_API_KEY="your_r2d2_apikey"
export R2D2_HOST=discover
export R2D2_COMPILER=intel
export R2D2_SERVER_HOST="http://13.217.72.149"
export R2D2_SERVER_PORT=8080

NASA observation downloads may also require an Earthdata account and a ~/.netrc file with permissions set to 600.

Creating a reusable activation script¶

To avoid repeating the module and virtual-environment setup in every new shell, create $NOBACKUP/swell-project/activate_swell-project.sh with the following contents:

#!/bin/bash

source /discover/nobackup/projects/gmao/advda/swell/jedi_modules/spackstack_1.9_intel
source "$NOBACKUP/swell-project/.venv/bin/activate"

To restore the SWELL environment in a new shell, source the script:

source "$NOBACKUP/swell-project/activate_swell-project.sh"

Verify the installation before continuing:

swell --help
cylc --version

Running a simple case¶

After installing SWELL and setting up your R2D2 credentials, you can run a simple hofx_cf experiment. In your activated Python environment, run:

swell create hofx_cf

This command creates the experiment under /discover/nobackup/${USER}/SwellExperiments/swell-hofx_cf. Explore this directory and locate the generated configuration at swell-hofx_cf-suite/experiment.yaml. This file contains the resolved configurations and settings for the experiment.

The create command prints information about R2D2 and the command needed to launch the experiment. The output will look similar to this abbreviated example:

                  _ _
 _____      _____| | |  Swell workflow deployment manager
/ __\ \ /\ / / _ \ | |  NASA Global Modeling and Assimilation Office
\__ \\ V  V /  __/ | |  Version 20260728
|___/ \_/\_/ \___|_|_|  Documentation located at: https://geos-esm.github.io/swell

Swell fetches and stores files using the R2D2 database framework
...
SwellCreateExperiment: Creating experiment: 'swell-hofx_cf' in
                       '/discover/nobackup/mabdiosk/SwellExperiments'
SwellCreateExperiment:
SwellCreateExperiment: Experiment successfully installed. To launch experiment use:
SwellCreateExperiment:
SwellCreateExperiment:   swell launch /discover/nobackup/mabdiosk/SwellExperiments/swell-hofx_cf/swell-hofx_cf-suite

Run the command printed in your output. For this example, it is:

swell launch /discover/nobackup/mabdiosk/SwellExperiments/swell-hofx_cf/swell-hofx_cf-suite

Follow the instructions in the launch output to open the Cylc TUI and monitor the geos_cf workflow.

After launch, Cylc stores task logs under ~/cylc-run/ by default. To place them in $NOBACKUP, launch with swell launch -l "$NOBACKUP/cylc-run" <suite-directory>.

Note: We did not specify which observations to use or the start and end dates of the experiment. Running swell create hofx_cf without an override.yaml uses the suite’s default configuration. The next section explains how to customize those values. See the SWELL documentation for additional configuration options.

Configuring a SWELL suite¶

SWELL builds an experiment by resolving a series of configuration “questions” defined by the suite. Each question receives a value through a hierarchy of defaults and overrides. The generated experiment.yaml shows the final values resolved for your hofx_cf experiment.

In this training, we focus on using an override.yaml file.

Using an override file (-o override.yaml)¶

For a repeatable configuration change, run swell create <suite> -o my_override.yaml. The override file follows the structure of the final experiment dictionary. Model-independent settings appear at the top level, while model-dependent settings belong under a models block keyed by model component. For example:

start_cycle_point: '2025-10-10T12:00:00Z'
final_cycle_point: '2025-10-20T00:00:00Z'

model_components:
  - geos_cf

skip_r2d2: false
jedi_build_method: use_existing
check_for_obs: false

models:
  geos_cf:
    window_length: PT6H
    window_type: 3D

    # Must match the background experiment stored in R2D2.
    background_experiment: geos_cf_oper
    background_time_offset: PT9H
    horizontal_resolution: c360
    vertical_resolution: 72
    npx: 361
    npy: 361

    # Must match the observations stored in R2D2.
    observations:
      - tempo_no2_tropo

    observation_providers:
      tempo_no2_tropo: nasa_v4

We store reusable override files in the swell-config repository so that configuration changes can be tracked and shared.

Editing suite_config.py¶

To change the defaults for a suite, edit the relevant QuestionList in src/swell/suites/<suite>/suite_config.py. Each entry, such as ingest_obs_cf, defines model-independent questions and per-model blocks such as geos_cf=[...] using the QuestionDefaults (qd) and SuiteQuestions (sq) helpers. This approach changes the SWELL source code and causes your configuration to diverge from the tagged version of the repository.

Editing the generated experiment directly¶

After swell create produces the experiment directory, experiment.yaml and flow.cylc are plain files that you can edit before launching. This approach is useful for one-time debugging. For any change that you need to reproduce, record it in an override file or, when changing suite defaults, in suite_config.py.

About R2D2¶

R2D2 (Research Repository for Data and Diagnostics) is a data catalogue. Swell asks R2D2 “give me the TEMPO NO₂ file for this window” and R2D2 answers with a file.

R2D2 is a metadata database plus a storage location. The server holds a database of what exists and where. When you request to fetch a file/item , the client asks the server where, then transfers the file from storage. With R2D2, users don’t need to know about the file’s name or location. They need to use R2D2 keys to describe the file so R2D2 can fetch the file.

Swell’s own R2D2 introduction has the full architecture diagram and the store/fetch reference. What follows is a high-level summary tailored for CF application.

Items¶

Everything in R2D2 is an item, and the item type determines which keys identify it. If the keys are incorrect and you get a 400 from the API. Here is a list of R2D2 items:

Notice the asymmetry as each item requires different identification keys:

Observations have no experiment. They are shared input data and independent of the experiment. Once tempo_no2_tropo for a given window is in R2D2, every experiment fetches the same file. This is the whole point of the ingest suites — ingest once, use forever...

Forecasts and feedback do have an experiment. The reason being, the results may vary based on the experiment configuration so we need to distinguish between the output files.

Next steps¶

Next, we examine the hofx_cf, ingest_obs_cf, and ingest_background_cf experiments in more detail. Together, these suites provide the main components of an end-to-end monitoring system.