Table 7 Comparison of different disease severity assessment methods in relation to accuracy-related measures, statistical methods and scale type and resolution used for method validation

Visual assessment

Images traced from slide projections for contrast and image analyzed

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope coefficient (a and b)

R² = 0.913 to 0.960;

a = 1.064 to 12.958;

b = 1.029 to 1.245

Martin and Rybicki (1998

Images on paper cut by symptom border and weighed

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

ρ_c = 0.51–0.99

Nita et al. (2003)

Manual image analysis

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope coefficient (a and b)

R² = 0.51 to 0.93;

a = −1.90 to 41.38;

b = 0.65 to 1.24

Godoy et al. (2006)

Manual image analysis

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope coefficient (a and b)

LCC (ρ_c)

Depending on symptom types:

R² = 0.59 to 0.88;

a = − 1.52 to 2.83;

b = 0.10 to 1.21;

ρ_c = 0.85 to 0.94

Bock et al. (2008a)

Manual image analysis

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope coefficient (a and b)

R² = 0.88 to 0.98;

a = −6.68 to 5.09;

b = 0.75 to 0.94

Michereff et al. (2009)

Manual image analysis

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

Means of ρ_c = 0.76 to 0.98 depending on fruit perspective and use of SADs

Spolti et al. (2011)

Manual image analysis

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

Mean of ρ_c = 0.79 and 0.89 (with and without SADs)

Yadav et al. (2013)

Manual image analysis

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

ρ_c = 0.83 to 1.00 (mean = 0.95)

Bardsley and Ngugi (2013)

Manual image analysis

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

Means of ρ_c = 0.53, 0.87, 0.86 and 0.87 (without SADs, with SADs, and with color or black and white SADs)

Schwanck and Del Ponte (2014)

VIS (RGB) image analysis

Images on paper cut by symptom border and weighed

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope coeffficient (a and b)

R² = 0.996;

a = −0.91;

b = 0.99

Lindow (1983)

Planimeter measurement, various pathosystems

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope

R² = 0.976 to 0.992;

a = 0.914 to 1.06;

b = − 0.17 to −4.35

Lindow and Webb (1983)

Images traced from slide projections for contrast and image analyzed

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope

R² = 0.971 to 0.985;

a = − 0.877 to 0.610;

b = 0.999 to 1.045

Martin and Rybicki (1998)

Manual image analysis

Area in pixels

–

Regression, intercept and slope

R² = 0.980;

a = 0.901;

b = 16,097

Peressotti et al.( 2011)

Severity measured with multiplex real-time PCR

Ratio scale

0 to 100% (100)

100

Regression

R² = 0.9945

De Coninck et al.( 2012)

Visual (pixels)

Ratio scale

0 to 100% (100)

100

Accuracy (%)

Overall accuracy = 96%

Barbedo (2014)

Visual ratings by 16 raters (inspection deemed image analysis was accurate)

Ratio scale

0 to 100% (100)

100

LCC (ρ_c)

ρ_c = 0.76–0.99 (mean = 0.92)

Stewart and McDonald (2014)

Visual (Pixels)

Ratio scale

0 to 100% (100)

100

PCC (r)

r = 0.60–0.90

Clément et al.( 2015)

Manual segmentation using Photoshop

Ratio scale

0 to 100% (100)

100

Quality of segmentation (Q_s)

Q_s = 84.17%

Hu et al. (2017)

Visual

Ordinal (4 classes)

Healthy stage, early stage, middle stage, end stage

Classification accuracy (%)

Classification accuracy:

Healthy stage = 100%;

Early stage = 93.1%;

Middle stage = 83.3%;

End stage = 97.0%

Wang et al.( 2017)

Visual

Ordinal (5 classes)

Healthy, very low, low, high, very high

Classification accuracy (%) compared to other diseases and severities

Accuracy of severity measurement = 78.57–86.51% (depending on architecture of CNN)

Esgario et al. (2019)

Visual

Ordinal (2 classes)

Mild, severe

Classification accuracy (%) compared to other diseases and severities

Severe symptoms = 70.4%;

Mild symptoms = 29.4%

Ramcharan et al. (2019)

Visual

Ordinal (3 classes)

Healthy, general, serious

Proportion accurately classified

0.91

Liang et al. (2019)

Multspectral (MSI) and Hyperspectral (HSI)

Visual

Ordinal (3 classes)

Low, medium high

Classification accuracy (%)

71 to 91%, depending on class (a 5-class scale had accuracy = 11 to 40%)

Coops et al. (2003)

Visual

Ratio scale

0 to 100% (100)

100

Percentage results with error ≥ 5%

24.1%

Larsolle and Muhammed (2007)

Visual

Ordinal (9 classes)

0, 1, 10, 20, 30, 45, 60, 80% or 100%

Regression, intercept and slope

R² = 0.91;

a = 2.40;

b = − 721.22

Huang et al. (2007)

Visual

Ordinal (4 classes)

Severe, medium, light, non-visible

None given

None given

Cui et al. (2009, 2010

Visual

Ordinal (3 classes)

Healthy tissue, light mycelium, dense mycelium

Classification accuracy (%) including 3 diseases and their severities

Healthy tissue = 100%;

Overall accuracy with disease = 61.70 to 98.90%

Mahlein et al. (2012b)

Symptom progression

Changing symptoms related to spectral changes

–

Metro maps

–

Wahabzada et al. (2015)

Symptom progression

Changing symptoms related to spectral changes

–

Leaf traces (similar to above)

–

Kuska et al. (2015)

Visual

Ordinal (9 classes)

0, 1, 5, 10, 20, 40, 60, 80 or 100

Regression (R²)

RMSE

R² > 0.90;

RMSE< 0.15

Wang et al. (2016)

Visual (in-field disease incidence of infected wheat spikes)

Ratio scale

0 to 100% (100)

100

Regression, intercept and slope (depending on VI)

R² = 0.801, 0.828;

a = 0.2902, 0.4572;

b = 0.0013, 0.0020

Kobayashi et al. (2016)

DNA quantification (presymptomatic)

Continuous

DNA content

Regression (R²)

R² = 0.868

Zhao et al. (2017)

DNA quantification (presymptomatic to symptomatic)

Continuous

DNA content

Regression (R²)

R² = 0.72

Thomas et al. (2017)

Length of lesion

mm

mm

–

Predicted lesion length was proportional to the interior lesion length.

Nagasubramanian et al. (2017)

Visual

Ordinal (3 classes)

Low (≤5%), moderate (5 to 20%), severe (> 20%) severity.

Classification accuracy (%)

94.83%

Thomas et al. (2018a, b)