METviewer Taylor Diagram graphical values do not match the R data values #2549

jwraby · 2024-04-10T22:53:39Z

jwraby
Apr 10, 2024

I will provide the plots and plot_spec files to Julie Prestopnik by email for further distribution within DTC as needed. The R data I have is as follows:
model valid_hour stat_name fcst_var stat_value stat_btcl stat_btcu nstats
URMA2pt5_d01 0 FBAR TMP 288.899386241959 NA NA 3
URMA2pt5_d01 0 FSTDEV TMP 6.70492916311306 NA NA 3
URMA2pt5_d01 0 OBAR TMP 288.918529804854 NA NA 3
URMA2pt5_d01 0 OSTDEV TMP 6.74934581722694 NA NA 3
URMA2pt5_d01 0 PR_CORR TMP 0.986896213222442 NA NA 3
URMA2pt5_d01 0 RMSE TMP 1.09010235655628 NA NA 3

Maybe I'm reading the diagram incorrectly, but for example the FSTDEV value of 6.7 doesn't match the scale values of the standard deviation along the x and y axes. 6.7 would be plotted well outside of the diagram as scaled. Also the RMSE value of 1.09 doesn't make sense when the red dot data point is well inside the radius of 0.5 from the "observation" location on the x-axis. The value of the Pearson Correlation Coef from the diagram looks reasonable compared to the value from the R data. Can you explain what I'm doing wrong interpreting the data from the diagram or diagnose the error I'm making when plotting the diagram? Thanks.

Answered by bikegeek

Apr 16, 2024

The R script tab no longer has any useful information, the R implementation is no longer used. The Python implementation of the Taylor Diagram is the only available means of plotting.

The RMS value isn't directly plotted. The pr_corr and stdev (normalized) are the only two values used to plot the point and the grid lines for the RMS represent the centered RMS difference. The RMS difference is related to the correlation and stdev by this relationship (from Taylor's paper):

 E^2 = sigmaf^2 _ sigmar^2 - (2sigmaf*sigmar *R)

    sigmaf^2 and sigmar^2 are variaces of the fcst and reference fields
    R is correlation coeff
    E is the centered RMS difference

So in a nutshell, the RMS value o…

View full answer

bikegeek · 2024-04-10T22:57:45Z

bikegeek
Apr 10, 2024
Collaborator

I will need to take a look at your input data and see if I can reproduce your results and compare those to the output points from the Python implementation in METplotpy. METviewer is now using the Python implementation of the Taylor diagram.

2 replies

jwraby Apr 10, 2024
Author

Thanks for your quick response! I just sent the plot_spec and PNG files by email to Julie. I can't upload the files into this discussion. Do you need any other data such as the STAT files.

jprestop Apr 10, 2024
Maintainer

@bikegeek I have put the following files at seneca:/d1/projects/METplus/discussions/2549:

plot_20240410_121652.png  
plot_20240410_121652.xml  
point_stat_URMA_MSA_REGION_d01_000000L_20161112_000000V.stat
point_stat_URMA_MSA_REGION_d01_000000L_20161113_000000V.stat
point_stat_URMA_MSA_REGION_d01_000000L_20161114_000000V.stat

bikegeek · 2024-04-11T22:03:58Z

bikegeek
Apr 11, 2024
Collaborator

Thanks for providing the data and other results. I have a question:

The output from METviewer indicates that you have the correct statistics needed to plot the Taylor Diagram (FSTDEV, OSTDEV, and PR_CORR), but the .stat files you provided (via Julie) are entirely MPR linetype.

Are you doing some processing to the MPR output before loading it into the METviewer database?
How are you obtaining the FSTDEV, OSTDEV, and PR_CORR statistics from your data?

The FSTDEV, OSTDEV, and PR_CORR statistics are in the CNT linetype, which is what I was anticipating for input data.

Regards,
Minna

0 replies

jwraby · 2024-04-11T22:27:30Z

jwraby
Apr 11, 2024
Author

Minna - Thanks for your response. I opened up one of the STAT files resident on the local linux workstation which is where METviewer runs and when I search for strings "CNT", "SL1L2", and "MPR" I find all 3 present. I checked the Point-Stat config file I used to produce them and I specified the output of all 3 line types using "BOTH" for each one. I'm not doing any processing of the STAT files prior to loading them into the database. After the diagram is plotted, checking the R data tab shows that all the needed statistics you list are present. See the dialog and data I input when I first created the discussion. I can try uploading and resending the 3 STAT files, but I won't be able to do this now as my system is having network issues. I'm not working tomorrow but will be back on Mon morning. R/ John

…

________________________________ From: bikegeek ***@***.***> Sent: Thursday, April 11, 2024 4:04 PM To: dtcenter/METplus ***@***.***> Cc: Raby, John W CIV USARMY DEVCOM ARL (USA) ***@***.***>; Author ***@***.***> Subject: Re: [dtcenter/METplus] METviewer Taylor Diagram graphical values do not match the R data values (Discussion #2549) Thanks for providing the data and other results. I have a question: The output from METviewer indicates that you have the correct statistics needed to plot the Taylor Diagram (FSTDEV, OSTDEV, and PR_CORR), but the .stat files you provided (via Julie) are entirely MPR linetype. Are you doing some processing to the MPR output before loading it into the METviewer database? How are you obtaining the FSTDEV, OSTDEV, and PR_CORR statistics from your data? The FSTDEV, OSTDEV, and PR_CORR statistics are in the CNT linetype, which is what I was anticipating for input data. Regards, Minna — Reply to this email directly, view it on GitHub<#2549 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUX5FH4ROHTKJYIWMQD4BQDY44CGJAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGI3DA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

0 replies

bikegeek · 2024-04-11T22:34:14Z

bikegeek
Apr 11, 2024
Collaborator

Thanks John, I don't think you need to re-run anything. Apologies, I searched your data and found the CNT. I'm multi-tasking with another project and was looking for ECNT instead of CNT in your input. I'll attempt to generate the Taylor diagram via command line (METplotpy) and see what data and plot it produces. Regards, Minna

…

On Thu, Apr 11, 2024 at 4:27 PM jwraby ***@***.***> wrote: Minna - Thanks for your response. I opened up one of the STAT files resident on the local linux workstation which is where METviewer runs and when I search for strings "CNT", "SL1L2", and "MPR" I find all 3 present. I checked the Point-Stat config file I used to produce them and I specified the output of all 3 line types using "BOTH" for each one. I'm not doing any processing of the STAT files prior to loading them into the database. After the diagram is plotted, checking the R data tab shows that all the needed statistics you list are present. See the dialog and data I input when I first created the discussion. I can try uploading and resending the 3 STAT files, but I won't be able to do this now as my system is having network issues. I'm not working tomorrow but will be back on Mon morning. R/ John ________________________________ From: bikegeek ***@***.***> Sent: Thursday, April 11, 2024 4:04 PM To: dtcenter/METplus ***@***.***> Cc: Raby, John W CIV USARMY DEVCOM ARL (USA) ***@***.***>; Author ***@***.***> Subject: Re: [dtcenter/METplus] METviewer Taylor Diagram graphical values do not match the R data values (Discussion #2549) Thanks for providing the data and other results. I have a question: The output from METviewer indicates that you have the correct statistics needed to plot the Taylor Diagram (FSTDEV, OSTDEV, and PR_CORR), but the .stat files you provided (via Julie) are entirely MPR linetype. Are you doing some processing to the MPR output before loading it into the METviewer database? How are you obtaining the FSTDEV, OSTDEV, and PR_CORR statistics from your data? The FSTDEV, OSTDEV, and PR_CORR statistics are in the CNT linetype, which is what I was anticipating for input data. Regards, Minna — Reply to this email directly, view it on GitHub< #2549 (comment)>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AUX5FH4ROHTKJYIWMQD4BQDY44CGJAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGI3DA>. You are receiving this because you authored the thread.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#2549 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4UJHWVMIWTV3372Y56Q43Y44E6RAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGM4DI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

0 replies

jwraby · 2024-04-11T22:38:16Z

jwraby
Apr 11, 2024
Author

Understand completely, Minna. No apology needed. R/ John

________________________________ From: bikegeek ***@***.***> Sent: Thursday, April 11, 2024 4:34 PM To: dtcenter/METplus ***@***.***> Cc: Raby, John W CIV USARMY DEVCOM ARL (USA) ***@***.***>; Author ***@***.***> Subject: Re: [dtcenter/METplus] METviewer Taylor Diagram graphical values do not match the R data values (Discussion #2549) Thanks John, I don't think you need to re-run anything. Apologies, I searched your data and found the CNT. I'm multi-tasking with another project and was looking for ECNT instead of CNT in your input. I'll attempt to generate the Taylor diagram via command line (METplotpy) and see what data and plot it produces. Regards, Minna

On Thu, Apr 11, 2024 at 4:27 PM jwraby ***@***.***> wrote: Minna - Thanks for your response. I opened up one of the STAT files resident on the local linux workstation which is where METviewer runs and when I search for strings "CNT", "SL1L2", and "MPR" I find all 3 present. I checked the Point-Stat config file I used to produce them and I specified the output of all 3 line types using "BOTH" for each one. I'm not doing any processing of the STAT files prior to loading them into the database. After the diagram is plotted, checking the R data tab shows that all the needed statistics you list are present. See the dialog and data I input when I first created the discussion. I can try uploading and resending the 3 STAT files, but I won't be able to do this now as my system is having network issues. I'm not working tomorrow but will be back on Mon morning. R/ John ________________________________ From: bikegeek ***@***.***> Sent: Thursday, April 11, 2024 4:04 PM To: dtcenter/METplus ***@***.***> Cc: Raby, John W CIV USARMY DEVCOM ARL (USA) ***@***.***>; Author ***@***.***> Subject: Re: [dtcenter/METplus] METviewer Taylor Diagram graphical values do not match the R data values (Discussion #2549) Thanks for providing the data and other results. I have a question: The output from METviewer indicates that you have the correct statistics needed to plot the Taylor Diagram (FSTDEV, OSTDEV, and PR_CORR), but the .stat files you provided (via Julie) are entirely MPR linetype. Are you doing some processing to the MPR output before loading it into the METviewer database? How are you obtaining the FSTDEV, OSTDEV, and PR_CORR statistics from your data? The FSTDEV, OSTDEV, and PR_CORR statistics are in the CNT linetype, which is what I was anticipating for input data. Regards, Minna — Reply to this email directly, view it on GitHub< #2549 (comment)>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AUX5FH4ROHTKJYIWMQD4BQDY44CGJAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGI3DA>. You are receiving this because you authored the thread.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#2549 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4UJHWVMIWTV3372Y56Q43Y44E6RAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGM4DI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<#2549 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUX5FH5HRCPVIE6L5CZUGGTY44FXXAVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAOBZGQZDI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

0 replies

bikegeek · 2024-04-15T22:38:55Z

bikegeek
Apr 15, 2024
Collaborator

I am reproducing a plot that is similar to the one generated by METviewer, with similar values that you produced. I'll need to check the formula used by the plot to see why the points don't appear where they should.

1 reply

jwraby Apr 16, 2024
Author

Thanks for the update.

bikegeek · 2024-04-16T18:32:55Z

bikegeek
Apr 16, 2024
Collaborator

I looked at the equations, and the stdev is being normalized, which is why we aren't observing the 6.x values we see in the R data output:

stdev = FSTDEV/OSTDEV

stdev = 6.704/6.749
= 0.993

and the Pearson's correlation is plotted without modification, which explains why the correlation on the plot is consistent with the R data value

1 reply

jwraby Apr 16, 2024
Author

This is good information. I wasn't aware of the normalization of the stdev. I looked in the Rscript tab to see if the script has that computation, but I didn't see it. Must be there because the location of the plotted red data point corresponds to the value of 0.993 on the Standard Deviation scale of the plot. Were you able to find out why the RMSE value doesn't appear to be plotted correctly? Looking at the plot it looks like the value should be about 0.2, but the R data value is 1.09.

bikegeek · 2024-04-16T20:32:03Z

bikegeek
Apr 16, 2024
Collaborator

The R script tab no longer has any useful information, the R implementation is no longer used. The Python implementation of the Taylor Diagram is the only available means of plotting.

The RMS value isn't directly plotted. The pr_corr and stdev (normalized) are the only two values used to plot the point and the grid lines for the RMS represent the centered RMS difference. The RMS difference is related to the correlation and stdev by this relationship (from Taylor's paper):

 E^2 = sigmaf^2 _ sigmar^2 - (2sigmaf*sigmar *R)

    sigmaf^2 and sigmar^2 are variaces of the fcst and reference fields
    R is correlation coeff
    E is the centered RMS difference

So in a nutshell, the RMS value on the Taylor diagram is supposed to reflect this relationship.

1 reply

jwraby Apr 16, 2024
Author

This is very informative. I really need to read the Taylor paper to better understand the diagram. I think the description in the METviewer User's Guide, which actually cites the paper, was very brief and referred to the basic statistics "RMSE", Standard Deviation" and "Pearson Correlation Statistic" which I interpreted differently from the way that Taylor was using the statistics to produce the diagram. See the User's Guide: https://metviewer.readthedocs.io/en/main_v4.1/Users_Guide/taylordiag.html. In retrospect, I should have read the paper to get a better understanding of the diagram before asking all the questions. I really appreciate all your investigation into the diagram which did answer all my questions.

bikegeek · 2024-04-16T22:28:56Z

bikegeek
Apr 16, 2024
Collaborator

Thanks for the feedback, I should get a statistician to add more content to the User's Guide/fill in more background information. I'll create a Github issue for documentation, but it won't make it into the next release: dtcenter/METplotpy#435 Please feel free to make any additional comments on what you would find useful in the documentation under the 'comments' section of the issue. Regards, Minna

…

On Tue, Apr 16, 2024 at 3:47 PM jwraby ***@***.***> wrote: This is very informative. I really need to read the Taylor paper to better understand the diagram. I think the description in the METviewer User's Guide, which actually cites the paper, was very brief and referred to the basic statistics "RMSE", Standard Deviation" and "Pearson Correlation Statistic" which I interpreted differently from the way that Taylor was using the statistics to produce the diagram. See the User's Guide: https://metviewer.readthedocs.io/en/main_v4.1/Users_Guide/taylordiag.html. In retrospect, I should have read the paper to get a better understanding of the diagram before asking all the questions. I really appreciate all your investigation into the diagram which did answer all my questions. — Reply to this email directly, view it on GitHub <#2549 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4UJHRELU6DWYKCTGVMDWLY5WL63AVCNFSM6AAAAABGBIOMI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TCMZVGQ4TA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

METviewer Taylor Diagram graphical values do not match the R data values #2549

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 9 comments 5 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

METviewer Taylor Diagram graphical values do not match the R data values #2549

jwraby Apr 10, 2024

Replies: 9 comments · 5 replies

bikegeek Apr 10, 2024 Collaborator

jwraby Apr 10, 2024 Author

jprestop Apr 10, 2024 Maintainer

bikegeek Apr 11, 2024 Collaborator

jwraby Apr 11, 2024 Author

bikegeek Apr 11, 2024 Collaborator

jwraby Apr 11, 2024 Author

bikegeek Apr 15, 2024 Collaborator

jwraby Apr 16, 2024 Author

bikegeek Apr 16, 2024 Collaborator

jwraby Apr 16, 2024 Author

bikegeek Apr 16, 2024 Collaborator

jwraby Apr 16, 2024 Author

bikegeek Apr 16, 2024 Collaborator

jwraby
Apr 10, 2024

Replies: 9 comments 5 replies

bikegeek
Apr 10, 2024
Collaborator

jwraby Apr 10, 2024
Author

jprestop Apr 10, 2024
Maintainer

bikegeek
Apr 11, 2024
Collaborator

jwraby
Apr 11, 2024
Author

bikegeek
Apr 11, 2024
Collaborator

jwraby
Apr 11, 2024
Author

bikegeek
Apr 15, 2024
Collaborator

jwraby Apr 16, 2024
Author

bikegeek
Apr 16, 2024
Collaborator

jwraby Apr 16, 2024
Author

bikegeek
Apr 16, 2024
Collaborator

jwraby Apr 16, 2024
Author

bikegeek
Apr 16, 2024
Collaborator