Couldn't run METplus on multiple files using LOOP_BY = init

[pouwereou]

Hello everyone,
I have just configured METplus and I was able to run the use case successfully.
I was trying to run GridStat on multiple files (wrf output) according to the initial conditions using METplus wrapper. I did the first run of wrf with initial time as 2024090200, then the second run with initialisation time at 2024090206, the third run initialised at 2024090212, then the fourth run at 2024090218. The four runs were for the next two days.
So I created four directories called 2024090200, 2024090206, 2024090212 and 2024090218 which acted as input directories. Each directory contains two wrfoutput files obtained using pcp_subtract. For example, the 2024090200 directory contains "wrfprs_d01_20240902.nc" and "wrfprs_d01_20240902.nc".
I run the METplus with "run_metplus.py" and it was successful, but it only runs on the first file (wrfprs_d01_20240902.nc).
I need guidance on how to configure the configuration files so that it would run for all the files, especially the FCST_GRID_STAT_INPUT_TEMPLATE, LEAD_SEQ. I have read the documentation but it is still not clear to me how to do the correct configuration. I will upload the configuration files and the data I used. Thank you very much for your help.

This is the command I used:
/home/pouwereou.nimon/METplus/METplus-5.1.0/ush/run_metplus.py -c /home/pouwereou.nimon/METplus/METplus-5.1.0/parm/use_cases/met_tool_wrapper/GridStat/GridStat.conf -c /home/pouwereou.nimon/METplus/ohpc.pouwereou.nimon.test.conf

Pouwereou

[CPKalb]

Hi:

I don't see your observations as part of the data on our FTP server, so I wasn't able to fully test your configuration. However, based on the files and information I do see, I think the issue here is that the LEAD_SEQ variable is not defined. Since it's not defined, METplus is just processing the first file for each model initialization, but no other files. To get both forecasts processed, you would want to set the following in your configuration:

LEAD_SEQ = 0, 24

This will run both the files (wrfprs_d01_20240902.nc and wrfprs_d01_20240903.nc) in the 2024090200 directory. Does this give the expected output on your side?

Christina

[pouwereou]

Hi,
Thanks for your reply. I just wanted to say that I would like to aggregate all the statistics using Stat Analysis and also stratify by forecast lead time. I am currently working on it following the instructions you gave me, but it is not easy to get it right.

[pouwereou]

Hi Christina,
I have tried to run METplus with PROCESS_LIST = GridStat, StatAnalysis but it did not work. Instead I just ran PROCESS_LIST = StatAnalysis as I already have the result from GridStat. I got the message METplus has successfully finished running as user pouwereou.nimon(1039) and I only got ALL.stat and job.out as output which contain no results. I know something is missing, but I don't know how to find out. Maybe I could share my files with you, please.
I also have a few questions:

• If I want to run PROCESS_LIST = GridStat, StatAnalysis, in which config file do I have to specify this?
• If I want to run the METplus wrapper for stat_analysis, do I also need to edit the STATAnalysisConfig which is in /MET/share/met/config? Because there is also STATAnalysisConfig_wrapped which is in /METplus/METplus-5.1.0/parm/met_config and is set in the StatAnalysis.conf file.
  Thanks a lot,

Pouwereou

[CPKalb]

Hi:

You would want to set that PROCESS_LIST in the METplus configuration file. Could you share your METplus configuration so I can take a look? You should be able to post the METplus configuration file to this discussion, but you will likely have to change the filename to have a .txt extension to post it here. I may also need a small subset of the output from Grid-Stat to our FTP site for me to be able to test.

Christina

[pouwereou]

Hi,
Please find attached the METplus configuration.
StatAnalysis.conf.txt
I am about to upload a subset of the output from Grid-Stat to the FTP site.

Pouwereou

[pouwereou]

Hi,
I am also attaching this METplus configuration file.
defaults.conf.txt

[CPKalb]

Hi:

Since Stat Analysis aggregates over time, the time templates work differently here than they do for other METplus tools like Grid Stat. Specifically, you have set

MODEL1_STAT_ANALYSIS_OUT_STAT_TEMPLATE = {init?fmt=%Y%m%d%H}/{valid?fmt=%Y%m%d%H}.stat

Given that the run is set to aggregate over multiple model initializations, there isn't a single init or valid time for the run, so METplus fills in the word ALL for these templates. To get a time output in the filename, you will want to use one of the many filename template tags that are unique to Stat Analysis. For example, if change the following:

MODEL1_STAT_ANALYSIS_OUT_STAT_TEMPLATE = {init_beg?fmt=%Y%m%d%H}/{init_beg?fmt=%Y%m%d%H}_{fcst_lead?fmt=%HH}.stat

I get output files labeled with the model initialization and lead time. As for the job.out file, you are getting this because you have set:

STAT_ANALYSIS_OUTPUT_TEMPLATE = job.out

Stat Analysis can produce 2 types of output. The first is a .stat file that is the same format as the output from tools such as Grid Stat. This is specified in the variable MODEL1_STAT_ANALYSIS_OUT_STAT_TEMPLATE. The second type is the standard output that lacks the same number of columns as exist in MET's traditional .stat format. This is specified in STAT_ANALYSIS_OUTPUT_TEMPLATE. You have both specified so you will get both types of output. However, STAT_ANALYSIS_OUTPUT_TEMPLATE does not have any time templates set, so data for each run will be saved to the file job.out. In your configuration, Stat Analysis is run twice (once for the 0 hour lead time and once for the 24 hour lead time). So, the file job.out contains output from the second of these (the 24 hour lead time). Data from the 0 hour lead time is overwritten. You probably want to add time template information to this variable so you get an output file for each run of Stat Analysis

As for why the output is empty, I see the following information inside the METplus log file:

DEBUG 2: Processing 7 STAT files.
DEBUG 2: STAT Lines read     = 136
DEBUG 2: STAT Lines retained = 0
DEBUG 2:
DEBUG 2: Processing Job 1: -job aggregate_stat -model WRF -fcst_lead 000000 -fcst_valid_hour 120000 -fcst_init_hour 000000 -fcst_init_hour 120000 -fcst_var TMP -obtype ANALYS -line_type CTC -out_stat /glade/derecho/scratch/kalb/MET_help/nimon_data/out/2024090200/2024090200_000000.stat -out_line_type CTS -out_thresh >=12.7 -out_alpha 0.05000
DEBUG 1: Creating STAT output file "/glade/derecho/scratch/kalb/MET_help/nimon_data/out/2024090200/2024090200_000000.stat"
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=562124647
DEBUG 2: Computing output for 0 case(s).
WARNING:
WARNING: do_job_aggr_stat() -> no matching STAT lines found for job: -job aggregate_stat -model WRF -fcst_lead 000000 -fcst_valid_hour 120000 -fcst_init_hour 000000 -fcst_init_hour 120000 -fcst_var TMP -obtype ANALYS -line_type CTC -out_stat /glade/derecho/scratch/kalb/MET_help/nimon_data/out/2024090200/2024090200_000000.stat -out_line_type CTS -out_thresh >=12.7 -out_alpha 0.05000
WARNING:
DEBUG 2: Job 1 used 0 out of 0 STAT lines.

What this means is that all input data read into Stat Analysis is filtered out from consideration so Stat Analysis has no data left to perform an aggregation. There are two things that are causing all of the data to be filtered out. The first is FCST_VAR_LIST = TMP. This variable needs to be set to a value that's in the FCST_VAR column in the output from Grid Stat In the files you put on the FTP, I only see one forecast variable, and that is APCP. So, you will want to change this to FCST_VAR_LIST = APCP. The second problem is FCST_VALID_HOUR_LIST = 12. All of the forecasts are actually valid at 0. So, you need to change this to FCST_VALID_HOUR_LIST = 0.

With those two changes, I get output statistics in each file. Do these changes work on your end?

I'll add information on aggregating all statistics in another post.

Christina

[CPKalb]

One other important thing to note is that you have more than one masking region set in your output files. If you want to keep these masking regions separate for the Stat Analysis aggregation, you will want to set the following in the job:

STAT_ANALYSIS_JOB1 = -job aggregate_stat -line_type CTC -out_line_type CTS -out_thresh ge12.7 -by VX_MASK -out_stat [out_stat_file]

This will stratify your output results by the different masking regions. Also, I don't see the threshold of 12.7 in your output data. I only see 1 and 5. So, you probably want to also make the following change to your job:

STAT_ANALYSIS_JOB1 = -job aggregate_stat -line_type CTC -out_line_type CTS -by VX_MASK,FCST_THRESH -out_stat [out_stat_file]

This will give you output for each threshold separately.

Christina

[CPKalb]

Hi:

To aggregate statistics over all lead times will require another setup and configuration for Stat Analysis. Specifically, it will require another call to Stat Analysis in the PROCESS_LIST. This complicates things. The way to call a process more than once in METplus requires the use of configuration sections and instance names in the PROCESS_LIST, which can be tricky. I went ahead and took your StatAnalysis.conf and made the necessary additions for a second run of Stat Analysis that will aggregate over all data. Since this is a more complicated aspect of METplus, hopefully this file can serve as an example of how to set up this type of processing.

The settings in this file will run Stat Analysis another time to aggregate over all lead times, and will output the results to an aggregate_all directory.
StatAnalysis.conf.txt

Christina

[pouwereou]

Hi,
Thank you very much for your explanations which are very valuable for me. I am able to complete the job successfully if I take into account all your suggestions. For the aggregation statistics over all lead times (I used the config file you shared with me), I was expecting to get four directories (2024090200, 2024090206, 2024090212 and 2024090218) in the aggregate_all directory. I only got the 2024090200 directory. I thought that maybe the 2024090200_all_leads.stat file could contain the results of the individual lead times. For me the lead times are: 2024090200, 2024090206, 2024090212 and 2024090218. I don't know if I understand the concept? Please explain it to me.

Pouwereou

[CPKalb]

Hi:

For the aggregation statistics over all lead times, that configuration is set up to aggregate over all lead times and all model initialization times. This is why only a single directory is output. However, it sounds like what you want to do is aggregate over all lead times for each initialization time. In this case, the configuration will need to be changed to get this output.

You will want to change to LOOP_LIST_ITEMS = at the bottom of the file to

LOOP_LIST_ITEMS = FCST_INIT_HOUR_LIST

Make sure this change is made to the LOOP_LIST_ITEMS that is at the bottom of the file in the [all _leads] section (below the line that reads [all_leads]). If you change the other LOOP_LIST_ITEMS, it will impact the run of Stat-Analysis that is aggregated over each lead time.

When you do this, I expect this will give you output in 2 directories, (2024090200 and 2024090212). To get all 4 directories, you will need to add 6 and 18 to the FCST_INIT_HOUR_LIST.

Do you want the 6 and 18 hour runs to be included in the first run of Stat-Analysis (the one that aggregates over separate lead times)? If so, then set FCST_INIT_HOUR_LIST = 00, 06, 12, 18. However, if you want the 6 and 18 hour initialization hours to be excluded from the aggregation over different lead times but included when aggregating over all lead times, then you will need to create another variable in the [all_leads] section that says: FCST_INIT_HOUR_LIST = 00, 06, 12, 18. Does that make sense? I don't know if I explained it well.

Christina

[pouwereou]

Hi there,
I ran the stat analysis using LOOP_LIST_ITEMS = FCST_INIT_HOUR_LIST as you said, then I got the output in 2 and 4 directories respectively. I also applied it to 2m temperature and it worked perfectly. As I will be working with METplus I think I will understand more. Otherwise your explanations were clear to me, thank you very much.

Pouwereou,

[CPKalb]

Great! I'm going to go ahead and mark this as answered and close the discussion. If you have future questions, feel free to start up a new post.

Christina