Overview

This page is a resource to help stakeholders working along the COVID-19 evidence-to guidance- to action - to data - to evidence cycle improve their work processes and results (see diagram at the top of this page). The ultimately goal is to a soon as possible, improve COVID-19 care delivery and outcomes.
The tables below are designed to aggregate Collaborative participant recommendations for addressing steps in this COVID-19 knowledge ecosystem. The recommended tools and approaches are being gathered and will evolve over time as stakeholder input is received and community consensus around best practices - and the pandemic itself - evolve.

Recommendation Table Listing

Identify Studies
Synthesize Evidence
Produce Guidance
Make Guidance Computable
Implement Guidance
Analyze Care Results
Leverage Results Analysis (e.g., for Quality Improvement, Reporting, Evidence Generation)

Process for Populating Recommendation Tables

Identify 'leads' for each table who are currently doing extensive, collaborative work around the ecosystem step.
Leads provide pointers to what their collaborative communities to be high value resources for table cells
Other Collaborative participants likewise add comments and suggestions about this emerging information
Formal processes/criteria will be developed by the Collaborative for adding/vetting information in the tables to optimize their value and use (including defining explicit criteria for what belongs in each row), e.g.,
- Input Sources:
- Search Strategies:
- Output Repositories:
- Standards:
- Initiatives:
- Tools/Platforms:
- Other Best Practices:

Recommendation Tables for Knowledge Ecosystem Steps

Identify Studies

	General Recommendations	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks
What to know/do Overview
Input Sources	L*VE/Epistemonikos Cochrane COVID-19 study register NIPH COVID-19 Living Evidence Map MedRxiv ClinicalTrials.gov/COVID-19 New pubmed interface DoctorEvidence CORD-19 Dataset Search EBSCO COVID-19 Information Portal University of Michigan COVID-19 Best Evidence Front Door	NEWS2 predicts severity/mortality at admission Annals IM, hospitalized patient trajectories Sensitivity, Specificity Predictive Values of Tests for COVID-19 in ED PREDI-CO Risk assessment tool	For steroids systematic meta-review, 8 sources have been identified these are (MEDLINE, CORD-19, L-OVE/Episteminokos, NIH iSearch COVID-19, EuropePMC, WHO COVID-19 Database, EMBASE + Prospero)
Search Strategies
Output Repositories
Standards
Initiatives	Librarian Reserve Corps COVID-Evidence Project - building a database to gather all the evidence around a drug (University of Basil - currently focused on hydroxychloroquine, but model could be adaptable to other targets) Identification of sources to identify primary studies - validation study of specialized COVID-19 databases (systematic reviewers are going to different sources - this effort is to identify best practices) Advocate for librarian representation in searches and reviews - leverage skillsets/best practices in this work [SLMC]: close gaps between needs that clinicians are seeing on the front line and topics covered in reviews guidelines COKA Evidence/ Tools WG [Project Google Drive](work in process)
Tools/ Platforms
Other Best Practices	Framework slides: Applying Standards to the Evidence Domain (from the COKA Evidence Ecosystem Liaison WG)

Synthesize Evidence

	General Recommendations	Anticoagulation	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks	includes evaluation of quality of evidence also needs to address processing of real world evidence [weave in ICER - search ISER opioids]
What to know/do Overview	For people creating evidence syntheses See if work on planned topic has been done already - search L*VE/Epistemonikos (2K reviews - cover many questions - only a fraction registered in PROSPERO; most reviews are out of date - only 3% have all the trials they should), PROSPERO for what's in the pipeline. Make sure that any existing reviews look at outcomes that are pertinent to your needs [leverage COMET - outcome measures], etc. that of interest for the need at hand. Very hard for user to identify a review that's up to date. COVID-END repository of Best Evidence Syntheses can be helpful here. Need to have access to raw data that fed into the review so it's more re-usable. important to understand end-user needs so that the review can be aimed at addressing them AHRQ has methods guides, Cochrane has them. Are there tools for this on COVID - e.g., Resources and tools for researchers considering and conducting COVID-19 evidence syntheses (From COVID-END Synthesizing WG) For organizations consuming evidence syntheses For selecting/using synthesized evidence: Is there a GRADE evidence profile? Do they make quality assessments? How comprehensive are they? Search date
Input Sources	See Sources/Repositories from Identify Studies table ClinicalTrials.gov/COVID-19; New pubmed interface; DoctorEvidence CORD-19 Dataset Search PROSPERO	Underlying justification and evidence tables (critically ill and acutely ill) from ASH Anticoagulation Living Guidelines L*VE/Epistemonikos search on antithrombotics; can send email when new study/review/guideline is published. Rapid Cochrane review on anticoagulation in COVID-19 patients	BMJ EBM rapid review on COVID tests
Output Repositories	SRDR+ COVID NMA Project COVID-END Best Evidence Syntheses Inventory ('best evidence syntheses' on clinical management) reviews that are living, up to date, high quality and where there is a GRADE evidence profile. LIVING Project - living systematic review of COVID-19 interventions; L*VE/Epistemonikos, VA COVID-19 Evidence Reviews; GIN COVID-19 Evidence Resources EvidenceAid Usher Network for COVID-19 Evidence Reviews
Standards	EBM on FHIR/COKA (standards include: Evidence, EvidenceReport, EvidenceVariable, Group, Citation Vocabulary Map for Evidence-related resources[ Project Google Drive ])
Initiatives				COKA Steroids for COVID-19 Systematic Meta-Review Protocol; slides about this effort
Tools/ Platforms	Resources and tools for researchers considering and conducting COVID-19 evidence syntheses (From COVID-END Synthesizing WG) GIN COVID-19 Evidence Resources SRDR+, Covidence, Distiller SR, RevMan, JBI Sumari Doctor Evidence Analytics Tool
Other Best Practices	COVID-NMA has initiated communication with all trialists to try to ensure consistent approaches e.g. selection of outcomes, reduction of risk of bias, and to invite them to contribute missing data. Framework slides: Applying Standards to the Evidence Domain (from the COKA Evidence Ecosystem Liaison WG)

Produce Guidance

	General Recommendations	Anticoagulation	Testing/Triage
Key Definitions and Frameworks
What to know/do Overview
Input Sources	See Output Repositories from Synthesize Evidence table	ASH Living Guideline on COVID anticoagulation and underlying justification and evidence tables (critically ill and acutely ill) NIH Guideline AU living Guideline NEJM vignette with expert opinion	BMJ/Remote Triage in Ambulatory Care
Output Repositories	Living Map of COVID-19 recommendations (Canadian Institutes of Health Research) ECRI COVID-19 Guideline Resources GIN will publish (has published?) a registry of guidelines in development (COVID and beyond)
Standards
Initiatives
Tools/ Platforms	GIN COVID-19 Guidance Resources MAGICapp; COVID-END Rapid Response Model and Tips/Tools GRADE International Patient Decision Aid Standards 9/15/20 Annals IM article on COVID guidelines. AGREE II for assessing guideline quality and reporting; Africa Centre for Evidence Living Hub of COVID-19 Knowledge Hubs
Other Best Practices	COVID-END Recommending WG guidance on producing COVID-19 Guidelines [to be released soon] 2011 NAM (IOM) Standards for Guidelines Guideline development resources: GIN/McMaster Checklist, Australia Guidelines for Guidelines Handbook Methodologies for the Development of CHEST Guidelines and Expert Panel Reports

Make Guidance Computable

	General Recommendations	Anticoagulation	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks	[CPG on FHIR and BPM+ IGs do this separately - need to be combined and simplified (e.g., with helpful graphics); e.g., leverage L1-L4] [Approach section from CPG on FHIR IG might be helpful in framing scope]
What to know/do Overview
Input Sources	See output repositories under Produce Guidance
Output Repositories	CDS Connect Repository
Standards	CPG on FHIR (working to coordinate with BPM+, and don't yet incorporate value sets); value could be enhanced by better connection through connecting what's in CPG on FHIR to links before and after it in the ecosystem cycle. BPM+ Logica-led Care Management Implementation Guide [standards and guidance] to cultivate synergies between CPG on FHIR and BPM+ [in development - e.g., see info on Care Management Track at 2020-09 Connectathon]) CDS Hooks; SMART on FHIR COVID-19 Interoperability Alliance Value Sets Logica COVID-19 FHIR Profile Library IG [under development] MCBK Standards WG Metadata work
Initiatives
Tools/ Platforms	CDS Connect Authoring Tool
Other Best Practices	From C19HCC Digital Guideline WG: Using CPG-on-FHIR standard for representing/ expressing the full intent of the Guidance in computer-interpretable artifacts (part of HL7 CPG-IG) Using the Agile Approach to CPG Development (inclusive of Integrated Process) to concurrently Produce Guidance and Make Guidance Computable (part of HL7 CPG-IG) Use Agile Knowledge Engineering methods, principles, and tools Cross-functional Integrated team (Agile CPG Team) Leverage composite nature of CPGs (e.g. can develop logic for inferences on patient information- CPG_CaseFeatures) to build incrementally and iteratively with rapid feedback Pull knowledge engineers into Content design/reviews; pull domain SMEs into knowledge representation design/reviews Leverage CPG-on-FHIR as a faithful expression of Guidance and its ability to create computationally derived CDS and Cognitive Support, patient-specific, practice-level digital Quality Measures/Metrics, eCaseReports, etc. to create computable artifacts used downstream in the Learning Health System and to provide closed-loop feedback/feedforward. Leverage established tools and capabilities (e.g. BPM+ process and tooling, Clinical Ontology) to author computable Guidance and Open Source tooling to translated into HL7 CPG-on-FHIR to leverage derivative and native compute tips: Use established standards and work with standards community (to understand and evolve as needed) Engage consumers/ users early and often Engage downstream vendors (e.g Terminology vendor USED in the EHRs) early Just because everyone everyone is using the same terminology systems doesn't mean they're agreeing how to use the actual terms- this needs to be considered and addressed to make ecosystem/supply chain work properly (feedforward from Evidence, but also feedback of data semantics back into evidence) Learn from related communities of practice (e.g. Agile Software Engineering)

Implement Guidance (e.g., as CDS, eCQMs)

	General Recommendations	Anticoagulation	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks	Implement Guidance includes integrating the computable guidance into organizational information system infrastructure, and deploying the intervention to users, and maintaining the interventions over time. Includes looking at 'leading' indicators (e.g., process changes) regarding intervention use and results. (as opposed to the next "Analyze Care Results" table below that addresses 'lagging indicators' (e.g., clinical outcomes))
What to know/do Overview
Input Sources	See output repositories for Make Guidance Computable
Standards
Initiatives	C19 Digital Guidelines WG developing and implementation guide for COVID-19 interventions
Tools/ Platforms
Other Best Practices	HITEQ Center Guide to Improving Care Processes and Outcomes HIMSS Improving Outcomes with CDS

Analyze Care Results (Within and Across Care Delivery Organizations)

	General Recommendations	Anticoagulation	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks	Includes gathering the data that will be analyzed 'Results' include 'lagging indicators' such as clinical outcomes. Analyzing/addressing 'leading indicators' (e.g., process changes) is addressed under "Implement Guidance"
What to know/do Overview	Curate data into a structured, centralized resource - applying specific instructions about how to obtain and document the data (historically a manual process to locate information but becoming more automated) Need semantic interoperability so that results can be aggregated/compared across CDOs; should be based on widely used, open data and standards Put information into registry (institution-specific or cross-institution) Generate report - to get feedback about effectiveness, safety, etc. of various interventions
Input Sources	MedMorph IG draft
Search Strategies
Output Repositories	Registries, e.g., from specialty societies. (discuss with Frank Opelka, ACS)
Standards	[under development] MCBK Standards WG Metadata work standards that do (or could) underpin registries
Initiatives	MedMorph, DEQM FHIR IG
Tools/ Platforms	eCQI Resource Center UMich/MCBK Knowledge Grid efforts MCBK UK ‘knowledge hackathons’ to implement NICE guidelines AHRQ-funded toolkit for ePROs in patient care NQF project to create a measurement framework linking quality of care delivered by telehealth, healthcare system readiness, and health outcomes in a disaster
Other Best Practices

Leverage Results Analysis (e.g., for Quality Improvement, Reporting, and Evidence Generation)

	General Recommendations	Anticoagulation	Testing/Triage	Other (Long COVID, Vaccine, Steroids)
Key Definitions and Frameworks
What to know/do Overview	Use reports (see analyze care results) Review reports (see analyze care results) with key stakeholders
Input Sources
Output Repositories
Current Standards	EBM on FHIR/COKA (standards include: Evidence, EvidenceReport, EvidenceVariable, Group, Citation) Vocabulary Map for Evidence-related resources [Project Google Drive] DEQM FHIR IG FHIR Electronic Case Reporting
Initiatives	MedMorph
Tools/ Platforms
Other Best Practices