set_namelist.py python script fails if first namelist variable has a null value

[mkavulich]

Original issue opened by @gsketefian migrated from regional_workflow repository: ufs-community/regional_workflow#386

I confirmed that this issue is still occurring, and updated the examples for easy replication.

Description

The set_namelist.py script fails if the configuration it is passed (via the -u flag) is such that the first variable has a null value.

For example, the following bash will fail:

fn_atm_nemsio=
FIXgsm=some_string
FIXLAM=another_string
settings="
'config': {
 'atm_files_input_grid': ${fn_atm_nemsio},
 'fix_dir_input_grid': ${FIXgsm},
 'fix_dir_target_grid': ${FIXLAM},
}
"
${USHDIR}/set_namelist.py -q -u "$settings" -o test.nml
Traceback (most recent call last):
  File "ush/set_namelist.py", line 355, in <module>
    set_namelist(sys.argv[1:])
  File "ush/set_namelist.py", line 332, in set_namelist
    update_dict(nml, cla.user_config.items(), quiet=cla.quiet)
  File "ush/set_namelist.py", line 299, in update_dict
    _ = dest[sect].pop(key, None)
  File "/contrib/miniconda3/4.5.12/envs/regional_workflow/lib/python3.8/site-packages/f90nml-1.1.2-py3.8.egg/f90nml/namelist.py", line 103, in __getitem__
KeyError: 'config'

Note that the second, third, etc variables (e.g. fix_dir_input_grid, which would happen if ${FIXgsm} was null) can be null; in such cases, set_namelist.py will still succeed. It only fails if the first variable is null.

fn_atm_nemsio=some_string
FIXgsm=
FIXLAM=another_string
settings="
'config': {
 'atm_files_input_grid': ${fn_atm_nemsio},
 'fix_dir_input_grid': ${FIXgsm},
 'fix_dir_target_grid': ${FIXLAM},
}
"
${USHDIR}/set_namelist.py -q -u "$settings" -o test.nml
cat test.nml
&config
    atm_files_input_grid = 'some_string'
    fix_dir_target_grid = 'another_string'
/

Solution

A temporary fix is to place the empty value that ${fn_atm_nemsio} represents in escaped double quotes. So the following will work:

settings="
'config': {
 'atm_files_input_grid': \"${fn_atm_nemsio}\",
 'fix_dir_input_grid': ${FIXgsm},
 'fix_dir_target_grid': ${FIXLAM},
}
"
${USHDIR}/set_namelist.py -q -u "$settings" -o ${nml_fn}

For consistency, set_namelist.py should be modified so that the first namelist variable can also be null, or so that all empty variables need to be placed in escaped double quotes.

[MichaelLueken]

@mkavulich and @gsketefian The set_namelist.py python script was replaced with uwtools in PR #1054.

A test was run using the config.community.yaml configuration, where LON_CTR= was added to scripts/exregional_make_grid.sh. In exregional_make_grid.sh, the regional_grid_nml namelist contains the following:

'regional_grid_nml':
  'plon': ${LON_CTR}
  'plat': ${LAT_CTR}
  'delx': ${DEL_ANGLE_X_SG}
  'dely': ${DEL_ANGLE_Y_SG}
  'lx': ${NEG_NX_OF_DOM_WITH_WIDE_HALO}
  'ly': ${NEG_NY_OF_DOM_WITH_WIDE_HALO}
  'pazi': ${PAZI}

As noted, plon is set to LON_CTR in the first namelist variable.

Adding LON_CTR= to the script, the yaml file correctly sets plon to null:

[2024-08-26T13:25:14]    DEBUG   regional_grid_nml:
[2024-08-26T13:25:14]    DEBUG     delx: 0.11241167188497128
[2024-08-26T13:25:14]    DEBUG     dely: 0.11241167188497128
[2024-08-26T13:25:14]    DEBUG     lx: -231
[2024-08-26T13:25:14]    DEBUG     ly: -143
[2024-08-26T13:25:14]    DEBUG     pazi: 0.0
[2024-08-26T13:25:14]    DEBUG     plat: 38.5
[2024-08-26T13:25:14]    DEBUG     plon: null

and the regional_grid_nml properly sets plon:

&regional_grid_nml
    plon = ,
    plat = 38.5
    delx = 0.11241167188497128
    dely = 0.11241167188497128
    lx = -231
    ly = -143
    pazi = 0.0
/

It looks like the update from set_namelist.py to uwtools has corrected this issue. Is it okay to close this issue?

[gsketefian]

@MichaelLueken Yes, it's ok with me to close this issue.

[MichaelLueken]

Thanks, @gsketefian! Closing this issue now.