This page introduces work by ACTS Collaborative Learning Community participants to improve how evidence and guidance is handled (e.g., made more computable and standards-based) to enhance how 'living' CDS interventions and eCQMs for specific COVID-19 targets (and beyond) are developed and updated in a virtuous knowledge ecosystem cycle.

• Section A is an overview figure illustrating enhanced information flow around the Evidence/Knowledge/Quality Ecosystem Cycle - i.e., driven by more computable and standards-based evidence and guidance
• Section B is an outline of Collaborative participants joint efforts to better detect and managing evidence/guidance updates that could potentially alter living CDS interventions that participants have deployed
• Section C outlines ecosystem enhancement needs and opportunities, as well as the notes on a potential concept demo toolkit (in the 4th table column) for addressing those needs 
• Section D is a diagram illustrating where and how a concept demo toolkit that makes evidence and guidance more computable and standards-based could enhance the Evidence/Knowledge/Quality Ecosystem (and Learning Health System) Cycle
• Section E is an outline with more details about a potential concept demo toolkit for improved evidence/guidance computability
• Section F is notes from a 10/13/20 email about a concept demo among Collaborative participants for enhancing and connecting links in the knowledge ecosystem

A. Evidence Ecosystem Enhancement - Overview Diagram

Enhanced Ecosystem Concept Demo Opportunity 9.25.20.pptx

UMN=University of Minnesota, ASH=American Society for Hematology, SCCM=Society for Critical Care MedicineeCR=eCase Report

[DOC search hyperlink]

B: Near-term Approach for Propagating Down the Knowledge Supply Chain Notifications about Potentially Important Updates Pertinent to Collaborative Participants' Living CDS Interventions 

Goal:

Illustrate how for sample targets (anticoagulation and/or COVID-19 testing/triage) we can signal to care teams (through 'living' CDS interventions) when a change in the evidence-review-guidance supply chain content for this target indicates a change in recommended care. Or this change indicates that the strength of evidence/guidance supporting a recommendation has changed. (The latter is important so this new information can be factored into patient-clinician shared decision making accordingly.)

• As part of this work, lay foundations for improving the evidence/guidance supply chain timeliness, efficiency and effectiveness by making its components (e.g., studies, reviews, guidelines) computable through standardized terminologies.

Approach:

• UMN/VA teams serve as a core for addressing the goal; they each manage all facets of this supply chain and also consume its results via CDS interventions that support care for their patient populations.
  • Cultivate synergies between these 'full cycle' efforts and related Collaborative participant efforts - e.g., NACHC (testing/triage in health centers), ACEP/EvidenceCare COVID-19 Severity Classification/Triage/Disposition tool (see here), Australia Living COVID-19 Guidelines (anticoagulation), U Melbourne COVID-CARE and related efforts, etc.
  • Have teams responsible in each of these organizations for evidence surveillance/synthesis, guidance development/updating, and CDS development/updating/deployment collaborate among themselves and with other organizations in the Collaborative on this 'update notification' process and tooling.
  • Other targets of interest being explored by Collaborative participants: assessment/management of 'Long COVID' (e.g., at VA - see this background/guidance article); steroids for COVID-19 meta-systematic review (pursued by COKA)
• Consider ways to coordinate/advance current efforts:  SRDR/COKA, C19HCC Digital Guideline WG, COVID-NMA, COVID-END, AU Living Guidelines, and related efforts collaborate to demonstrate notification system that suggests to living CDS owners/implementers that updates should be considered.
  • see below sampling of sources that could be leveraged
  • For overview presentation on making evidence computable see: AMDIS Ted Talk EBM on FHIR 2020 Oct 1 PPT
• Document how evidence/guidance changes are detected and addressed in current VA/UMN/other processes
  • enhancing these approaches to include a notification function (applicable across approaches) that propagates supply chain updates to all pertinent stakeholders throughout the chain, including those responsible for developing/maintaining CDS interventions.  
• Phased enhancement approach
  • Document current state: VA/UMN/Australia/others (business-driven requirements - what works, pain points, lessons learned)
  • Define/deliver desired future state (Individual/collective)
    • quick/easy enhancements to processes/tools/resources used for detecting and managing evidence/guidance updates  (e.g., leverage pilot 'web diff tool' to detect changes on target web pages)
    • deeper architecture refinements to achieve more computable/standards-based evidence/guidance processing (build on COKA/SRDR explorations)

Enhanced Process Template (under construction):

For Target - optimize work/output of people/process/technology:

DocSearch -> L*VE/Epistemonikos → Abstractor -> SRDR -> COVID-NMA > AU Living Guidelines → C19 Digital Guideline WG - UMN CDS Implementation Team → Evaluation → [back to beginning]

Sampling of External Sources to Check for Updated Evidence/Guidance on Targets

• NIH anticoagulation adaptive clinical trials announced 9/20
• COVID-NMA (search for heparin)
• VA COVID Reviews (search for triage, testing, anticoagulation)
• L*VE Epistimonikos
• COVID END Best Evidence Synthesis (see testing/triage here, and others/anticoagulants here)
• NIH Guideline on Antithrombotic therapy
• CDC Phone Triage Guidance
• ACEP/EvidenceCare computable guidance on ED triage (based on this guide)
• AU Living Guidelines - see 'definition of disease severity' and 'VTE' [leaders of this effort are engaged in this exploration and they will send their processes (e.g., search tools/strategies)]
• IDSA Guideline on Serologic Testing (frequently updated)
• [other resources listed on Collaborative's evidence/guidance processing Knowledge Ecosystem Recommendations page]

C. Ecosystem Needs, Enhancement Opportunities and Potential Concept Demo Outline

¹By those doing the work - e.g., EPCs, VA/UMN/Health Centers, Agile KE teams, NACHC/ACEP/EvidenceCare, many others

²More information about 'building blocks' for creating components of this 'fantasy' (e.g., standards, sources for inputs/outputs, tools/methods platforms) is on the Community of Practice webpages (see navigation bar left side of this page), and in this emerging catalog from the COVID-END project

D. Overview Diagram for Proof of Concept Demo Toolkit (computable evidence slide.pptx)

E. Notes on a More Comprehensive Proof of Concept Software Toolset 

The 4 proof of concept tools and related repository outlined below can be placed on an open source developmental website for public dissemination. Content development for these tools will be driven by Collaborative participant current efforts, e.g., focused on COVID-19 testing/triage in ambulatory and ED settings and anticoagulation in inpatient settings.

Tool 1: Create/Store/Access Computable Study Results Representation   

• Develop tools that enable users (e.g., via web interfaces) and systems (e.g., via APIs) to input to/output from a proof of concept repository. The repository could be an AHRQ development website created for purposes of the concept demo. This tool will enter/store/retrieve standards-based, computable data about a study related to a COVID-19 target, e.g., triage/testing, anticoagulation. Input sources can include medRxiv, journals, and article repositories, e.g., CORD-19. The data input tool user could be a study author/publisher, or someone else (e.g., in an EPC) for studies already published. Data output tool use could include systematic review developers and other individuals and applications requesting the study results. These proof of concept capabilities could be designed to align with SRDR+ functionality enhancements under consideration/development. 
  • This proof of concept tool demonstrates an open mechanism for capturing and presenting (e.g., via web-based viewer or API) summary results of individual studies in a manner that is re-usable by many different systems. This reduces redundant work currently required to extract key study variables that aren’t standardized and therefore aren’t interoperable or re-usable. 
  • The tool could also demonstrate linkages with clinicaltrials.gov, e.g., for visibility into what trials on the topic of interest (COVID-19 testing/triage) are underway and completed.
  • It also demonstrates a ‘notification feature’ that notifies ‘subscribers’ to the repository (e.g., systematic reviewers) that a new study pertinent to the topic of interest (COVID-19 ambulatory triage/testing) is entered. 

Tool 2: Create/Store/Access Computable Systematic Review Representation

• Develop tools that enable users (e.g., via web interfaces) and systems (e.g., via APIs) to input to/output from a repository (e.g., an AHRQ development website created for proof of concept purposes) systematic review information related to COVID-19 ambulatory triage/testing in a computable, standardized format.
• Includes notification tool/function to notify ‘subscribers’ (e.g., guideline developers) that there’s a new systematic review on the topic of interest.

Tool 3: Create/Store/Access Computable Rationale for Guidance

• Develop a proof of concept web interface and an API that use the FHIR standard for data exchange (i.e., EvidenceReport and PlanDefinition Resources maintained by the COKA/EBM on FHIR/CPG on FHIR HL7 projects) to encode a recommendation's evidence basis, i.e., pertinent studies and systematic reviews encoded by tools 1 (computable study results) and 2 (computable systematic reviews) above. This computable guidance rationale tool will also encode the rationale and confidence for the recommendation.  
• This enhanced, automated evidence processing for recommendations could be demonstrated to support data entry into the 'Evidence Supporting Recommendation' section of the C19HCC knowledge elicitation tool (see excerpt in diagram below). [Note - there are other templates for producing guidance, e.g., this one from GRADEpro, WHO "Digital Accelerator Kits", and others]
• Two functions that could be demonstrated are support for: 1) gathering and synthesizing the evolving evidence base pertinent to a recommendation and 2) assigning standard codes for the rationale behind and confidence in the recommendation.

Excerpt from Knowledge DRAFT Elicitation Tool:

[intervening portions omitted]

• Likewise, the proof concept tool could also demonstrate web-based connection to other guideline development/presentation software to facilitate use of standards-based, computable evidence in guideline development. For example, MAGICapp, used in the Australia National COVID-19 Living Guidelines initiative - see disease severity, section 4, pertinent to triage.
• The tool could contain a feature that notifies CDS and guideline developers/implementers that new systematic reviews on the topic of interest (testing/triage) is available.
• This tool uses EBM on FHIR and CPG on FHIR: leads for these efforts are closely engaged in the ACTS COVID Collaborative and could be involved in developing this toolkit
• Outputs from the use of the C19HCC knowledge elicitation tool are provided to the C19HCC Agile Knowledge Engineering teams, who create L3 guideline representations (i.e., in CQL and BPMN) and CDS/eCQMs/eCaseReports that are deployed in clinical settings.
• The resulting CDS interventions could be placed in CDS Connect; consider synergies between living computable evidence/ guidance as demonstrated in the proof of concept toolkit and the CDS Connect Authoring Tool. 

Tool 4: Identify/Store/Access Terminology for Computable Recommendation Definition

• Develop a proof of concept tool that enables guideline and CDS developers to identify/gather (e.g., via web interfaces, APIs) appropriate standard coded concepts and/or value sets for the required data elements needed for guideline execution. 
• For example, exposures, symptoms, high risk conditions, and special circumstances listed in the CDC Phone Line Advice triage algorithm (see page 6) that underlies part of the NACHC CDS intervention. 
• Proof of concept tool addresses challenges that knowledge engineers currently face in identifying code sets and value sets needed to make concepts and data elements in clinical guidelines computable. In a leading effort to put the CPG on FHIR implementation guide for computable guidance into practice, members of the C19HCC Digital Guideline WG are using a knowledge elicitation tool (see figure below) to elicit these concepts/terms from SMEs and then manually searching code sets/value sets to obtain codes needed to make recommendations computable. This part of the concept demo toolkit provides an automated lookup function that maps concepts/terms to potentially matching items from pertinent value/code sets. These codes/value are needed to access pertinent data from EHRs to execute the recommendation. 
• For example, the new tool could leverage the data dictionary being developed under the NACHC project to help computable guidance developers search code set repositories (e.g., from the COVID-19 Interoperability Alliance and others) to express guideline-related clinical data elements (e.g., body temperature) using specific, standard code sets (e.g., LOINC code, ICD-10, SNOMED, etc.).

Excerpt from Knowledge DRAFT Elicitation Tool: 

Section F: notes from a 10/13/20 email about a concept demo among Collaborative participants for enhancing and connecting links in the knowledge ecosystem

COVID-19 patient management targets being pursued by ACTS Collaborative participants (anticoagulation and testing/triage) present a promising opportunity for a concept demonstration of making the knowledge supply chain/ecosystem more integrated, efficient and computable. For example:

• The University of Minnesota (UMN) currently has anticoagulation CDS deployed for hospitalized patients with COVID-19 that they seek to make more 'living' and FHIR-enabled. (They're generating evidence that this guidance has favorable mortality implications when followed.) 
• Evidence/guidance UMN is using is on this page - in the process of being updated. 
• UMN is working with the C19HCC Digital Guidelines WG on leveraging CPG on FHIR for computable guidance development and implementation.
• UMN evidence processing efforts are increasingly supported by the UMN EPC (which is doing this work outside its AHRQ-funded efforts). 
• ACTS Collaborative participants are driving initiatives that could support these evidence ecosystem enhancements efforts at UMN in a concept demonstration. 
• L*VE/Epistemonikos has a regularly updated search on antithrombotics; can send email when new study/review/guideline is published.
• COKA and SRDR are exploring ways to make evidence more computable and interoperable.
• Sandy Lewis and David Tovey are reaching out to the Evidence/Guidance Community (e.g., GIN, COVID-NMA, COVID-END, others) to help synthesize best practices and tools for supporting evidence/guidance portions of the COVID evidence/knowledge ecosystem. In ad hoc support, David Tovey (former Cochrane Editor in Chief) sent me a Rapid Cochrane review on anticoagulation in COVID-19 patients
• The AU Evidence Task Force has a living guideline on anticoagulation, expressed in the MAGIC app.
• The COVID-END digitizing WG had email explorations into leveraging APIs to exchange evidence information among apps following the WG's meeting last Friday (see thread below). 
• Regarding COVID testing/triage, 
• ACEP/EvidenceCare has produced an evidence-based severity classification tool and seeks strategies and tools to keep this guidance in sync with evolving evidence.
• NACHC is likewise supporting deployment of testing/triage CDS in health centers across the US, and seeks better mechanisms for keeping this up to date. 
• VA also interested in this topic and in ways to better interconnect it's evidence ecosystem (e.g., evidence synthesis program, VA/DOD guidelines) to support living CDS interventions. 
• University of Melbourne has a related patient self-monitoring initiative, is likewise interested in evidence ecosystem enhancement efforts, and is planning to leverage this ACTS Participant Window as a focal point for enhancing how AU focused efforts leverage and support similar work by others in the Collaborative.
• Collaborative participants have already begun outlining ways to demonstrate enhanced knowledge ecosystem/supply chain functioning - see sections A-E on this page.  

This all suggests that the timing is ripe to define one or more specific demonstrations for better refining and interconnecting the supply chain links outlined above to deliver enhanced care processes and outcomes. Starting in the near term, and building toward more robustness and scaling as the explorations mature.