Table 3 Evaluation results of HAI Network’s BSI Surveillance system attributes, India, 2022
Attribute | Indicator | Evidence collected | Assessment | Overall evaluation |
|---|---|---|---|---|
Simplicity | Ease of collecting data | • Minimum 2 full-time surveillance staff • 80% staff spends > 2 h or more every day in collecting the 23 variables and the lab confirmation for case confirmation from paper-based reports | Time-consuming | Simple |
Ease of applying case definition | • 15/20 (75%) staff rate it as very easy or easy | Easy | ||
Ease of online reporting | • 18/20 (90%) rate it as very easy or easy • 10–15 min to submit one CRF, described as “user-friendly” | Easy | ||
Levels of reporting | • Two a) Local (hospital administration), and b) National (AIIMS, New Delhi) | Easy | ||
Stability | Reliability | • All 40 staff surveyed are trained in protocol • All 21 sites reviewed collected denominator data on all days including weekends/holidays • All 21 sites reviewed have access to a 24 by 7 working laboratory • 16/21 sites reviewed have access to all positive cultures, required for classifying BSI type • 5/21 sites reviewed reported not having LIS/ HMIS, reported using manual registers • 15/21 (71%) sites review every positive culture at month-end to capture missing cases | Reliable | Stable if funding is available |
Availability | • Available from 2017 including during second wave of COVID-19 in India • Decreased reporting to 84/131 (64%) in ICUs and surveillance stopped in 22/39 (56%) sites seen briefly during April 2020 | Available | ||
Sustainability | • 26/39 (67%) of the sites are funded by US CDC • Reporting decreased to 63/131 (48%) in the quarter 4, 2021 when funding was interrupted | Funding stability a concern | ||
Acceptability | Willingness of stakeholders to participate | • Started with 20 ICUs in 2017 and has increased to 131 ICUs in 2019 • Hospital administration of all 10 hospitals interviewed accepts this system is required to control multi-drug resistant pathogens | High | Acceptable |
Proportion of physicians accepting feedback from the surveillance system | • 90% physicians surveyed starting a QI initiative in their ICU based on the feedback received from surveillance |  | ||
Representativeness | Population representative of the participating hospital | • 32/58 (55%) of the febrile episodes reviewed had their blood cultured | Not representative | Not representative |
Data Quality | Data validity | • 280/284 (98%) of CRFs reviewed have correctly applied the case definition | Valid | Data quality is good |
Data completeness | • 259/284 (91%) of CRFs have data filled in each data va without any missing details | Complete | ||
Timeliness | Blood collection | • 27/61 (44%) of febrile episodes reviewed were cultured (blood collection) < 24 h of fever | Not timely | Blood culture collection and feedback to ICU physicians is not timely |
Reporting to network | • 36/39 (92%) sites reported data within 10 days of the reporting month | Timely | ||
Dissemination to key stakeholders | • 6/10 (60%) ICU physicians surveyed reported getting consistent monthly feedback • 13/14 (93%) quarterly reports shared by AIIMS with all key stakeholders within one month of the reporting quarter | Not timely Timely | ||
Detection of BSI outbreaks | • Sites not comfortable in sharing information regarding outbreaks detected in their hospitals, hence this information could not be captured • One outbreak of Burkholderia cepacia detected in the network using surveillance data and controlled (retrieved from published data) | ––– | ||
Sensitivity | True cases detected | • 26/26 (100%) cases reviewed during site visit were correctly identified | Sensitive | Sensitive |
Monitoring trends | • System can identify BSI trends from May 2017 to December 2021 | Sensitive | ||
Early warning signals generated | • System has generated 684 ICU specific BSI rate alerts from May 2017 to December 2021 • Alerts are generated when ICU specific BSI rates are > 20 for that month | Sensitive | ||
Positive Predictive Value | Proportion of true cases with confirmed BSI | • 26/26 (100%) cases reported as BSI for the months of November 2021 to March 2022 were reviewed and found to be true cases | Good | Good PPV |
Usefulness | Monitoring trends Detecting outbreaks Improving IPC | • Quarterly trend analysis done for every quarter from Jan 2018 to Dec 2018 • Three outbreaks detected including one outbreak of Burkholderia cepacia using surveillance data and controlled (retrieved from published data) • 12/39 (31%) sites have completed/ongoing QI projects to improve BSI rates • 7/10 (70%) physicians surveyed reported having increased adherence to recommended Central Line insertion and maintenance practices following feedback from the system | Useful | Useful |