Evaluation of probabilistic disease forecasts

G Hughes, FJ Burnett

Research output: Contribution to journalArticle

4 Citations (Scopus)
3 Downloads (Pure)

Abstract

The statistical evaluation of probabilistic disease forecasts often involves calculation of metrics defined conditionally on disease status, such as sensitivity and specificity. However, for the purpose of disease management decision making, metrics defined conditionally on the result of the forecast—predictive values—are also important, although less frequently reported. In this context, the application of scoring rules in the evaluation of probabilistic disease forecasts is discussed. An index of separation with application in the evaluation of probabilistic disease forecasts, described in the clinical literature, is also considered and its relation to scoring rules illustrated. Scoring rules provide a principled basis for the evaluation of probabilistic forecasts used in plant disease management. In particular, the decomposition of scoring rules into interpretable components is an advantageous feature of their application in the evaluation of disease forecasts.
Original languageEnglish
Pages (from-to)1136 - 1143
Number of pages8
JournalPhytopathology
Volume107
Issue number10
Early online date14 Jul 2017
DOIs
Publication statusFirst published - 14 Jul 2017

Fingerprint

disease control
plant diseases and disorders
decision making
degradation

Bibliographical note

1031389

Keywords

  • Brier score
  • Divergence score
  • Expected mutual information
  • G2 test
  • McFadden's R2
  • PSEP
  • Reliability
  • Resolution
  • Uncertainty

Cite this

Hughes, G ; Burnett, FJ. / Evaluation of probabilistic disease forecasts. In: Phytopathology. 2017 ; Vol. 107, No. 10. pp. 1136 - 1143.
@article{9fce6a6a32864ed88845af293a045092,
title = "Evaluation of probabilistic disease forecasts",
abstract = "The statistical evaluation of probabilistic disease forecasts often involves calculation of metrics defined conditionally on disease status, such as sensitivity and specificity. However, for the purpose of disease management decision making, metrics defined conditionally on the result of the forecast—predictive values—are also important, although less frequently reported. In this context, the application of scoring rules in the evaluation of probabilistic disease forecasts is discussed. An index of separation with application in the evaluation of probabilistic disease forecasts, described in the clinical literature, is also considered and its relation to scoring rules illustrated. Scoring rules provide a principled basis for the evaluation of probabilistic forecasts used in plant disease management. In particular, the decomposition of scoring rules into interpretable components is an advantageous feature of their application in the evaluation of disease forecasts.",
keywords = "Brier score, Divergence score, Expected mutual information, G2 test, McFadden's R2, PSEP, Reliability, Resolution, Uncertainty",
author = "G Hughes and FJ Burnett",
note = "1031389",
year = "2017",
month = "7",
day = "14",
doi = "10.1094/PHYTO-01-17-0023-FI",
language = "English",
volume = "107",
pages = "1136 -- 1143",
journal = "Phytopathology",
issn = "0031-949X",
publisher = "American Phytopathological Society",
number = "10",

}

Evaluation of probabilistic disease forecasts. / Hughes, G; Burnett, FJ.

In: Phytopathology, Vol. 107, No. 10, 14.07.2017, p. 1136 - 1143.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evaluation of probabilistic disease forecasts

AU - Hughes, G

AU - Burnett, FJ

N1 - 1031389

PY - 2017/7/14

Y1 - 2017/7/14

N2 - The statistical evaluation of probabilistic disease forecasts often involves calculation of metrics defined conditionally on disease status, such as sensitivity and specificity. However, for the purpose of disease management decision making, metrics defined conditionally on the result of the forecast—predictive values—are also important, although less frequently reported. In this context, the application of scoring rules in the evaluation of probabilistic disease forecasts is discussed. An index of separation with application in the evaluation of probabilistic disease forecasts, described in the clinical literature, is also considered and its relation to scoring rules illustrated. Scoring rules provide a principled basis for the evaluation of probabilistic forecasts used in plant disease management. In particular, the decomposition of scoring rules into interpretable components is an advantageous feature of their application in the evaluation of disease forecasts.

AB - The statistical evaluation of probabilistic disease forecasts often involves calculation of metrics defined conditionally on disease status, such as sensitivity and specificity. However, for the purpose of disease management decision making, metrics defined conditionally on the result of the forecast—predictive values—are also important, although less frequently reported. In this context, the application of scoring rules in the evaluation of probabilistic disease forecasts is discussed. An index of separation with application in the evaluation of probabilistic disease forecasts, described in the clinical literature, is also considered and its relation to scoring rules illustrated. Scoring rules provide a principled basis for the evaluation of probabilistic forecasts used in plant disease management. In particular, the decomposition of scoring rules into interpretable components is an advantageous feature of their application in the evaluation of disease forecasts.

KW - Brier score

KW - Divergence score

KW - Expected mutual information

KW - G2 test

KW - McFadden's R2

KW - PSEP

KW - Reliability

KW - Resolution

KW - Uncertainty

U2 - 10.1094/PHYTO-01-17-0023-FI

DO - 10.1094/PHYTO-01-17-0023-FI

M3 - Article

VL - 107

SP - 1136

EP - 1143

JO - Phytopathology

JF - Phytopathology

SN - 0031-949X

IS - 10

ER -