Accessing diagnostic estimate data

Accessing diagnostic estimate data

Usage

estimates(x, at, ...)

# S3 method for triptych_mcbdsc
estimates(x, ...)

# S3 method for triptych_murphy
estimates(x, at = NULL, ...)

# S3 method for triptych_reliability
estimates(x, at = NULL, ...)

# S3 method for triptych_roc
estimates(x, at = NULL, p1 = mean(observations(x)), ...)

Arguments

x

An object from which the estimate information should be extracted.

at

A vector of thresholds where x should be evaluated.

...

Additional arguments passed to other methods.

p1

The unconditional event probability. Used in combination with at to determine the diagonal lines along which to determine the estimate.

Value

A tibble with the relevant information describing the diagnostic estimate (Murphy curve, reliability curve, ROC curve, score decomposition) for all supplied forecasting methods.

For a Murphy curve, a tibble with columns: forecast, knot (the threshold value), limit ("left" or "right" in knot, only present if at = NULL), mean_score.

For a reliability curve, a tibble with columns: forecast, CEP, x (the knots of the isotonic regression estimate).

For a ROC curve, a tibble with columns: forecast, FAR (false alarm rate), HR (hit rate).

For a MCBDSC decomposition, a tibble with columns: forecast, mean_score, MCB (miscalibration), DSC (discrimination), UNC (uncertainty).

See also

regions(), forecasts(), observations()

Examples

data(ex_binary, package = "triptych")
tr <- triptych(ex_binary)

estimates(tr$murphy)
#> # A tibble: 20,018 × 4
#>    forecast     knot limit mean_score
#>    <chr>       <dbl> <chr>      <dbl>
#>  1 X01      0        left    0       
#>  2 X01      0        right   0       
#>  3 X01      1.19e-23 left    1.24e-23
#>  4 X01      1.19e-23 right   1.24e-23
#>  5 X01      3.84e-23 left    4.00e-23
#>  6 X01      3.84e-23 right   3.99e-23
#>  7 X01      9.45e-22 left    9.83e-22
#>  8 X01      9.45e-22 right   9.81e-22
#>  9 X01      9.82e-20 left    1.02e-19
#> 10 X01      9.82e-20 right   1.02e-19
#> # ℹ 20,008 more rows
estimates(tr$reliability)
#> # A tibble: 374 × 3
#>    forecast   CEP        x
#>    <chr>    <dbl>    <dbl>
#>  1 X01      0     1.19e-23
#>  2 X01      0     5.70e- 2
#>  3 X01      0.1   5.77e- 2
#>  4 X01      0.1   1.24e- 1
#>  5 X01      0.167 1.25e- 1
#>  6 X01      0.167 1.54e- 1
#>  7 X01      0.3   1.56e- 1
#>  8 X01      0.3   1.84e- 1
#>  9 X01      0.321 1.87e- 1
#> 10 X01      0.321 3.06e- 1
#> # ℹ 364 more rows
estimates(tr$roc)
#> # A tibble: 197 × 3
#>    forecast    FAR    HR
#>    <chr>     <dbl> <dbl>
#>  1 X01      1      1    
#>  2 X01      0.395  1    
#>  3 X01      0.308  0.990
#>  4 X01      0.280  0.983
#>  5 X01      0.266  0.977
#>  6 X01      0.197  0.941
#>  7 X01      0.176  0.929
#>  8 X01      0.148  0.906
#>  9 X01      0.140  0.900
#> 10 X01      0.0766 0.843
#> # ℹ 187 more rows
estimates(tr$mcbdsc)
#> # A tibble: 10 × 5
#>    forecast mean_score     MCB    DSC   UNC
#>    <chr>         <dbl>   <dbl>  <dbl> <dbl>
#>  1 X01          0.0827 0.00474 0.172  0.250
#>  2 X02          0.127  0.0233  0.146  0.250
#>  3 X03          0.134  0.0172  0.132  0.250
#>  4 X04          0.194  0.0587  0.114  0.250
#>  5 X05          0.222  0.0723  0.100  0.250
#>  6 X06          0.180  0.00494 0.0748 0.250
#>  7 X07          0.212  0.0211  0.0590 0.250
#>  8 X08          0.235  0.0263  0.0410 0.250
#>  9 X09          0.303  0.0818  0.0282 0.250
#> 10 X10          0.312  0.0772  0.0148 0.250