Skip to content

Commit

Permalink
Update capitalisation of risk matrix score for consistency
Browse files Browse the repository at this point in the history
  • Loading branch information
tennlee committed Dec 6, 2024
1 parent eef026c commit eda6549
Showing 1 changed file with 5 additions and 7 deletions.
12 changes: 5 additions & 7 deletions tutorials/Risk_Matrix_Score.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"> **CAUTION:**\n",
"> The risk matrix score is a novel metric that is still undergoing mathematical peer review. The implementation may change in line with the peer review process.\n",
"\n",
"The Risk Matrix Score provides a consistent way of scoring forecasts and warnings within a risk matrix framework (Taggart & Wilke 2024). A lower score is better.\n",
"The risk matrix score provides a consistent way of scoring forecasts and warnings within a risk matrix framework (Taggart & Wilke 2024). A lower score is better.\n",
"\n",
"In this tutorial we will illustrate the risk matrix framework using a slimmed down version of the synthetic case study from Taggart & Wilke (2024).\n",
"Consider a simple warning service for hazardous outdoor heat with warning levels \"Nil\", \"Yellow Warning\", \"Orange Warning\" and \"Red Warning\". The warning level that is issued depends on a combination of certainty and severity, as depicted in the matrix below.\n",
Expand All @@ -19,12 +19,11 @@
"\n",
"The certainty categories are defined by probability thresholds (0.1, 0.3, 0.5) and the three nested severity categories (MOD+, SEV+ and EXT) are defined by three temperature thresholds (35, 37, 40) in degrees Celsius. In this example we will assume that the lower probability threshold is included in the certainty category (e.g. \"possible\" correponds to probabilities $p$ satisfying $0.1 \\leq p < 0.5$).\n",
"\n",
"\n",
"A forecaster makes a risk assessment by selecting one certainty category for each of the nested severity categories, according to their probability distribution of daily maximum temperature. This is equivalent to selecting one cell in each of the three right-most columns in the risk matrix. The warning level is determined by the highest colour of the selected cells. For example, if a forecaster selects \"likely\" for MOD+ (yellow), \"possible\" for SEV+ (yellow) and \"possible\" for EXT (orange) then this generates an Orange Warning.\n",
"\n",
"## Risk matrix scores from a synthetic experiment\n",
"\n",
"We now set up a synthetic experiment which will generate forecasts from three different forecasters and corresponding observations. We will score the forecasters using the Risk Matrix Score in two different ways: (1) use a choice of decision weights that evaluates how they fill out the risk matrix, and (2) use a choice of decision weights that evaluates the warning levels they issue.\n",
"We now set up a synthetic experiment which will generate forecasts from three different forecasters and corresponding observations. We will score the forecasters using the risk matrix score in two different ways: (1) use a choice of decision weights that evaluates how they fill out the risk matrix, and (2) use a choice of decision weights that evaluates the warning levels they issue.\n",
"\n",
"Let $N(\\mu,\\sigma^2)$ denote the normal distribution with mean $\\mu$ and variance $\\sigma^2$. Suppose that each observed daily maximum\n",
"temperature $y$ is generated from random variables $y_1$, $y_2$ and $y_3$ using the formula $y = y_1 +y_2 +y_3$, where each random\n",
Expand Down Expand Up @@ -2671,7 +2670,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "pymc_env",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -2686,9 +2685,8 @@
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
},
"orig_nbformat": 4
}
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}

0 comments on commit eda6549

Please sign in to comment.