Table 3 Intra-rater-, Inter-rater reliability and accuracy measured by two questionnaire rounds of 20 case vignettes each
Years of working experience in infectious disease surveillance | Intra-rater reliability (κ, 95%-CI) | Inter-rater reliability per hospital (κ, 95%-CI)# | Accuracy (%, First round/Second round) | Accuracy in determination of presence of anastomotic leakage, n = 4. (%, First round/Second round) | Accuracy in determination of absence of anastomotic leakage, n = 16 (%, First round/Second round) | ||
|---|---|---|---|---|---|---|---|
Hospital A | Rater 1 | 4–5 | 0.78 (0.46–1.00) | 0.61 (0.23–0.83) | 95/85 | 75/100 | 93/87 |
Hospital A | Rater 2 | 2–3 | 0.95 (0.74–1.00) | 85/80 | 100/75 | 93/93 | |
Hospital B | Rater 1 | 11–15 | 0.83 (0.49–0.99) | 0.72 (0.42–1.00) | 80/85 | 75/100 | 93/93 |
Hospital B | Rater 2 | 6–10 | 0.73 (0.44–1.00) | 95/90 | 100/100 | 93/93 | |
Hospital C | Rater 1 | 11–15 | 1.00 (1.00–1.00) | 0.94 (0.75–1.00) | 90/90 | 75/75 | 93/93 |
Hospital C | Rater 2 | 11–15 | 0.94 (0.76–1.00) | 90/95 | 75/75 | 93/93 | |
Hospital D | Rater 1 | 0–1 | 0.75 (0.47–1.00) | 0.69 (0.36–0.92) | 90/85 | 100/100 | 93/87 |
Hospital D | Rater 2 | 4–5 | 0.89 (0.72–1.00) | 90/95 | 100/100 | 93/87 | |
Hospital E* | Rater 1 | 2–3 | 0.89 (0.59–1.00) | 0.65 (0.38–0.92) | 80/80 | 100/100 | 93/93 |
Hospital E* | Rater 2 | 4–5 | 0.73 (0.46–1.00) | 85/70 | 100/100 | 93/81 | |
Hospital F* | Rater 1 | 2–3 | 0.79 (0.57–1.00) | 0.69 (0.34–0.92) | 90/90 | 100/100 | 87/81 |
Hospital F* | Rater 2 | 11–15 | 0.89 (0.59–1.00) | 90/90 | 100/100 | 87/87 | |
Hospital G* | Rater 1 | 0–1 | 0.79 (0.55–1.00) | 0.84 (0.61–1.00) | 90/90 | 100/100 | 87/93 |
Hospital G* | Rater 2 | > 25 | 0.94 (0.75–1.00) | 95/90 | 100/100 | 93/93 |