Skip to contents

Evaluation of forecasts using a Triptych

Source: R/triptych.R
triptych.Rd

A triptych visualizes three important aspects of predictive performance: Economic utility via Murphy curves, miscalibration via reliability curves, and discrimination ability via ROC curves. The triptych S3 class has plotting methods for ggplot2.

Usage

triptych(x, y_var = "y", ..., y = NULL)

Arguments

x

A data frame, list, matrix, or other object that can be coerced to a tibble. Contains numeric forecasts, and observations (optional).

y_var

A variable in x that contains observations. Specified as the argument varin dplyr::pull().

...

Additional arguments passed to murphy(), reliability(), roc(), and mcbdsc().

y

A numeric vector of observations. If supplied, overrides y_var. Otherwise, defaults to dplyr::pull(x, y_var).

Value

A triptych object, that is a tibble subclass, and contains five columns:

  • forecast: Contains the names.

  • murphy: Contains a vctrs_vctr subclass of Murphy curves.

  • reliability: Contains a vctrs_vctr subclass of reliability curves.

  • roc: Contains a vctrs_vctr subclass of ROC curves.

  • mcbdsc: Contains a vctrs_vctr subclass of score decompositions.

See also

Vector class constructors: murphy(), reliability(), roc(), mcbdsc()

Adding uncertainty quantification: add_consistency(), add_confidence()

Visualization: plot.triptych(), autoplot.triptych()

Examples

data(ex_binary, package = "triptych")

tr <- triptych(ex_binary)
identical(tr, triptych(ex_binary, y))
#> [1] TRUE
identical(tr, triptych(ex_binary, 1))
#> [1] TRUE
tr
#> # A tibble: 10 × 5
#>    forecast          murphy     reliability             roc          mcbdsc
#>    <chr>         <trpt_mur>      <trpt_rel>      <trpt_roc>   <trpt_mcbdsc>
#>  1 X01      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  2 X02      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  3 X03      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  4 X04      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  5 X05      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  6 X06      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  7 X07      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  8 X08      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#>  9 X09      <named list[3]> <named list[3]> <named list[3]> <named list[3]>
#> 10 X10      <named list[3]> <named list[3]> <named list[3]> <named list[3]>

# 1. Choose 4 predictions
# 2. Add consistency bands (for reliability curves)
#    (Bootstrap resampling is expensive, the number of bootstrap samples
#     is small to keep execution times short)
# 3. Create patchwork object
# 4. Adjust the title of the legend
dplyr::slice(tr, 1, 3, 6, 9) |>
  add_consistency(level = 0.9, method = "resampling_Bernoulli", n_boot = 20) |>
  autoplot() &
  ggplot2::guides(colour = ggplot2::guide_legend("Forecast"))