Introduction to Pilot Activities

Three distinct but interconnected ACTS pilots were launched by AHRQ in 2020 to develop a foundation for producing the AHRQ DKP (Figure 10. AHRQ DKP). As outlined in the ACTS Roadmap, the goal for the AHRQ DKP—and these three pilots—is to drive progress toward an AHRQ-supported Knowledge Ecosystem Cycle Figure 11. Digital Healthcare Knowledge Ecosystem) where DKPs from AHRQ and others interoperate smoothly to enable LHSs and the Quintuple Aim (Figure 7. LHS Functions That the Knowledge Ecosystem Supports).
Each pilot will produce collaborations and deliverables that further define, coordinate, consolidate, analyze, and refine knowledge ecosystem to more resources within it more FAIR, computable, and useful in ways that produce seamless interoperability and information flow. This will speed up the broad application of evidence into practice to enhance public health, care delivery and improvement, and other health outcomes. The information gathered from these pilots will continue to refine the ecosystem to help build a framework supporting integration with other public and private knowledge platforms for improved knowledge creation, interoperability, and use at the point of need.

The three pilots include:

  1. The ACTS COVID19 Evidence to Guidance to Action Collaborative (Collaborative) Pilot that is using the COVID pandemic as a use case for a learning community to make information flow around the ecosystem cycle faster and more efficient and effective in the near term and set the stage for making this information flow more computable in the longer term
  2. The Evidence/Guidance Computability Tools Requirements Pilot that is developing requirements for tools to help make evidence and guidance computable—based on the use case of a Collaborative-linked NACHC ambulatory COVID risk assessment and case management project
  3. The VCU-led, Patient-driven Care Plan Tool Refinement and Implementation Pilot that is enhancing a patient-focused preventive care tool to demonstrate the feasibility of making evidence and recommendations from AHRQ and others more computable in a standards-based fashion—and evaluating knowledge ecosystem and clinical implications from developing this tool and deploying it in patient care

Table E-1. Pilots at a Glance provides a high-level description of the pilots and the corresponding ecosystem components impacted.

Pilot

Description

Ecosystem Components

ACTS COVID19 Evidence to Guidance to Action Collaborative

The goal of the Collaborative pilot is to collect, collaborate, and synthesize approaches (suggested or used) for each of the ecosystem components. These approaches will enhance information flow around the ecosystem cycle. The first use case to test the ecosystem components and the suggested approaches is the COVID pandemic test case. The results of the COVID use case will lay a foundation for achieving the broader AHRQ DKP and will drive progress toward the LHS.

All Components

Evidence/Guidance Computability Tools Requirements Pilot

The Tool Requirements Pilot is responsible for developing a requirements document for four key focus areas supporting tool development in alignment with the components of the ecosystem. This pilot will develop requirements to make studies, reviews, and guidance more computable in a standards-based fashion. The key focus areas for requirements include:

  1. Create/Store/Access Computable Study Results
  2. Create/Store/Access Computable Systematic Review Results
  3. Create/Store/Access Computable Rationale for Guidance
  4. Identify/Store/Access Terminology for Computable Recommendation Definition
  • Do Research Studies (Create/Store/Access Computable Study Results Tool)
  • Synthesize/Aggregate Evidence (Store/Access Computable Systematic Reviews Tool)
  • Produce Guidance (Store/Access Computable Systematic Reviews Tool)
  • Create Tools (Identify/Store/Access Terminology for Computable Recommendation Definition Tool)

VCU-led Patient-driven Care Plan Tool Refinement and Implementation Pilot

This pilot will focus on the process of converting an existing AHRQ-funded guidance on prostate cancer screening to a health technology standard (HL7 FHIR). The goal is to make AHRQ guidance available to broader set of stakeholders using a standards-based approach.

  • Create Tools
  • Disseminate tools
  • Integrate Tools and Systems
  • Use Tools to Support Care Decisions/Actions
  • Gather Data About Care Processes/Results

Timeline for Piloting Activities

All initial pilot activities will occur throughout 2021, with completion expected in October. Each pilot activity's results and outcomes will be evaluated and consolidated into a final report focusing on lessons learned challenges and next steps—in the context of producing an AHRQ DKP and other elements of executing the ACTS Roadmap. Table E-2 Pilot Draft Timeline outlines the approximate completion date for each of the high-level steps in pilot activities.

Table E-2 Pilot Draft Timeline Applies to each of the pilots

Activity

Approximate Completion Date

Requirements Gathering

February 2021

Development Activities

May 2021

Testing/Implementation 

September 2021

Evaluation/ Final Report

October 2021

Pilot Details

ACTS COVID19 Evidence to Guidance to Action Collaborative Pilot

Introduction & Background

The Collaborative Pilot addresses all components of the ecosystem and provides a framework and learning community in support of the other pilot activities and will function as a springboard for executing the ACTS Roadmap. The aim of the pilot is continual enhancement of the ecosystem to utilize it for other focus areas (e.g., anticoagulation, risk assessment, triage). The Learning Community and the Collaborative members will provide approaches that can be tested against the ecosystem cycle. There is broad industry recognition that to achieve the goals of the Roadmap, the ecosystem and components needs to be more computable. The computability of the approaches is part of the midterm goals. Beyond reviewing and working to identify considerations for components of the ecosystem, the Collaborative pilot will help evaluate potential pilot activities in support of the pilot portfolio.

Objective

Demonstrate that a learning community can begin improving the ecosystem cycle and components for a few high-priority clinical issues (related to COVID19) in several CDOs in ways that can be scaled broadly to healthcare targets and settings.

Goal

Using the COVID use case, the goal of this pilot is to synthesize suggested and employed approaches for each of the ecosystem components ensuring the input and outputs of the components work together to support the vision of the AHRQ Digital platform. This framework will provide the necessary infrastructure to collect, evaluate, and refine approaches to further the AHRQ digital platform's development. Additionally, the Collaborative pilot will work on ways to make the components of the ecosystem more FAIR and computable as part of mid-term pilot activities. For example, this pilot may look at the components of the ecosystem to help identify processes for identifying people, evaluating guidance, and examining the technology needed to make newly published study results coded and computable, facilitating more automation support for living systematic reviews. These activities would ultimately support more computable, dynamic guidance for CDS interventions and eCQMs, care results, resulting in computable study results.
The goal of the near-term goals of the Collaborative pilot are:

  • Cross-fertilize and accelerate current efforts to develop and deliver the latest COVID19 evidence-based guidance to care teams—and tools to apply this guidance
  • Measurably improve care and outcomes for COVID19 patients and care teams in limited settings for selected targets that can be scaled to other targets and settings
  • The mid-range goal of the Collaborative pilot is to:
  • Identify ways/standards that can be used to make the components of the ecosystem more computable

The longer-term goals of the Collaborative pilot are to:

  • Advance tools, standards, and collaborations that seed the DKPs (from AHRQ/others), knowledge ecosystem, reference architecture, and PPPs supporting the Roadmap
  • Develop a repository of knowledge, suggested approaches, consideration, tools, and projects that support all components of the ecosystem. This repository will function as both a place to collect and consume information in support of an LHS and inform ACTS Roadmap execution.

Approach

In late 2019 and early 2020 the ACTS Roadmap Workgroup was working to recommend pilot activities that would support the AHRQ DKP, knowledge ecosystem, and other ACTS Future Vision components. When the pandemic hit, the Collaborative pivoted (and attracted many additional participants) to focus on near-term ways to support participants' COVID-related ecosystem efforts in ways that could also be leveraged to drive progress toward the AHRQ DKP and ACTS Future Vision. The COVID pandemic represented an opportunity to focus knowledge ecosystem enhancement activities (i.e., making the flow from evidence to guidance to action to data and back to evidence a virtuous cycle) on a specific clinical target of urgent and widespread importance. The ACTS COVID19 Evidence to Guidance to Action Collaborative provides a learning community that is helping a growing list of participants make near-term enhancements to their COVID19-related LHS efforts (see the Collaborative website (55)).

Progress of the Pilot

The initial Collaborative focus was broadly sharing summaries of COVID management guidance from authoritative evidence-based sources (e.g., CDC, WHO) that the Penn Center for Evidence-based Practice had begun producing. In addition to providing these continually updated summaries on a Collaborative website, the pilot produced a tool to make Penn's work developing and maintaining these guidance summaries quicker and more efficient. The tool used a "track change" capabilities to highlight those sections of a web page that had changed, making it easier for providers and researchers to see what specifically changed over the course of a day. As the pandemic unfolded, Penn stopped maintaining these summaries, and the Collaborative's attention turned to examining the entire knowledge ecosystem more comprehensively. That is, surfacing participants' activities, learning, needs, recommendations, and potential support for other collaborative participants for each ecosystem cycle step.
These ecosystem enhancement approaches, needs, and recommendations were initially gathered and presented using basic tables on the Collaborative's website. A much more robust tool for managing this information has been developed and is being used by an increasing number of Collaborative participants to enter and use this information (80)). This database and associated data entry forms, query mechanisms, and data management tools will be used not only to help Collaborative participants and others advance their knowledge ecosystem related efforts for COVID and other targets, but it will also serve as the foundation for developing the playbooks, living evidence and guidance, implementation activities, and infrastructure (e.g., DKP and marketplace) called for in the Roadmap.
There are several specific COVID19-related clinical targets that are of particular focus for Collaborative participants' ecosystem cycle-related efforts to develop guidance and create and implement CDS interventions and monitor care results (e.g., via eCQMs) to drive further improvements. These targets include treatment with anticoagulants and steroids, risk assessment and triage in ambulatory and emergency department settings, and management of post-acute sequalae of COVID19 (PASC or 'long COVID'), among others. In addition to enhancing ecosystem cycle function broadly across targets, the Collaborative is fostering cross fertilization among ecosystem cycle activities focused on these targets in particular (e.g., through the ecosystem enhancement database (80), the weekly Collaborative calls, and other mechanisms). This target-focused work will provide an important foundation for the target-focused activities around living evidence/guidance/CDS/eCQM development, implementation, and evaluation called for in the Roadmap.
Pilot activities are open and transparent, with weekly meetings and posted project updates (webpages/confluence spaces).

Evidence/Guidance Computability Tools Requirements Pilot

Introduction & Background

The Tools Requirement pilot focuses on gathering requirements for tool development to support the components of the ecosystem. This pilot's goal is to develop requirements for tools that make evidence and guidance computable in ways that make it easier to develop and maintain this information and put it into action. The intent of these requirements is to support computable and standards-based evidence to assist developers in creating tools that further interoperability. The output of the Tools Requirement Pilot is a requirements document for use by those developing tools that align with the components of the ecosystem.
Additionally, the requirements document developed as a result of this pilot will inform planning and potential next steps in developing the AHRQ DKP and knowledge ecosystem to achieve LHSs and the Quintuple Aim. The requirements document will describe stakeholder needs, functional requirements, system overview, data element definitions and taxonomies, repository design, user and system interfaces, and other technical and system architecture specifications and design elements necessary to produce guidance in four focus areas. The four focus areas for tool development requirements include Create/Store/Access Computable Study Results, Create/Store/Access Computable Systematic Reviews, Create/Store/Access Computable Rational for Guidance, and Identify/Store/Access Terminology for Computable Recommendations Definition. The requirement document will be developed with participation and input from the Collaborative Pilot.
Details for each of the focus areas covered in the Tools Requirement document include:

  1. Create/Store/Access Computable Study Results Focus Area: This focus area of the Tools Requirement pilot will work to consolidate and expand on the efforts of the EBMonFHIR and COKA initiatives and will work closely with the SRDR+. In addition to working with EBMonFHIR /COKA and SRDR+, this focus area will look to any other efforts in the early stage of tool development to exchange limited information about studies between systems. Using SME's input, this focus area will create, store, and access the full spectrum of computable study results information needed to support computable systematic reviews. The Create/Store/Access Computable Study Results focus area for the Tool Requirements pilot maps to the "Do Research Studies" component of the ecosystem.
  2. Create/Store/Access Computable Systematic Reviews Focus Area: This focus area of the Tools Requirement pilot will build upon the work of the first focus area utilizing the EBM/COKA HL7 FHIR initiative, SRDR+, and other efforts that are making systematic review details more computable. The requirements for this focus area will rely on the work of the previously mentioned projects and the input of SMEs. The input will help develop and refine the requirements document. The Create/Store/Access Computable Systematic Reviews focus area for the Tool Requirements pilot maps to the Synthesize/Aggregate Evidence component of the ecosystem.
  3. Create/Store/Access Computable Rationale for Guidance Focus Area: This focus area of the tools Requirement pilot will take the work of SMEs from the COKA/EBM HL7 FHIR initiative, CPG-on-FHIR, Interoperability Alliance, and C19HCC Digital Guideline Workgroup with a focus on making guidance more computable. This focus area will coordinate with and expand these efforts to define requirements for tools designed to create, store, and access computable representations of the rationale for guidance. The Create/Store/Access Computable Rational for Guidance focus area for the Tool Requirements pilot maps to the Produce Guidance component of the ecosystem.
  4. Identify/Store/Access Terminology for Computable Recommendation Definition Focus Area: This focus area of the Tools Requirement pilot will utilize the work of the COKA/EBMonFHIR, CPG-on-FHIR, Interoperability Alliance, and the C19HCC Digital Guideline Workgroup efforts to make the guidance more computable. This focus area's outcome is to define requirements for tools designed to create, store, and access terminology(ies) needed to make clinical concepts and data elements used in guideline recommendations computable (e.g., automating the manual process of the terminology assignments.) The Identify/Store/Access Terminology for Computable Recommendations Definition focus area for the Tool Requirements pilot maps to the Create Tools component of the ecosystem.

Objective

The objective of this pilot is to produce requirements document that can be used by AHRQ and other knowledge supply chain stakeholders to make their guidance and evidence processing more efficient and the outputs from that guidance more FAIR.

Goal of the Pilot 

This pilot's primary objective and the requirements document is to inform efforts toward the development of a toolkit for making evidence and guidance computable, standards-based, and 'living' as it flows from clinical study results to systematic reviews to guidelines to CDS interventions. Like the other pilot efforts, this pilot is designed to demonstrate a stakeholder-validated approach to reduce significant waste, rework, and inefficiencies in the knowledge supply chain by making its components more computable and standards-based. This pilot will identify recommendations and specific steps that AHRQ (and others) can take to make guidance and evidence-processing tools and efforts more efficient and effective via the AHRQ DKP and knowledge ecosystem. Success metrics of this pilot effort include feedback from a multidisciplinary community of SMEs, input from the Collaborative pilot, lessons learned, and the results of tools developed using the outlined requirements.

Approach

This pilot will work with and utilize the results produced by the HL7 EBMonFHIR and COKA initiatives and with the SRDR+ as well as other stakeholders to create and catalog requirements to make data to evidence to guidance interoperable and computable.

Progress of the Pilot

The Tools Requirement pilot is in the forming stage with plans to formally begin in early 2021. Based on the pilot timeline, the Tools Requirement Pilot will work with SMEs, Collaborative pilot members, and a multidisciplinary community to collect, analyze, and draft requirements. This pilot will be community driven. Pilot activities will be open and transparent, with weekly meetings and posted project updates (webpages/confluence spaces) 

VCU-Led Patient-Driven Care Plan Tool Refinement & Implementation Pilot

Introduction & Background

The VCU Tools Refinement and Implementation pilot is designed to take currently developed AHRQ funded tools and explore the potential of converting tools into industry standards such as such HL7 FHIR, SMART on FHIR applications, CQL (325). The results of this pilot will be housed at CDS Connect (326) for broader dissemination of AHRQ based guidance. The focus of this pilot is twofold:

  1. Convert/refine previously developed My Preventative Care tool (327) to make it more standards based (HL7 FHIR) in order better support the knowledge supply chain using prevention as a target
  2. Focus on the Prostate Cancer Screening App within the My Preventative Care tool as the test case for the conversion/refinement. The output of the pilot is to have a standards-based prostate cancer screening tool that can be used as a "plug-and-play" application available to any health system with a desire to use the tool. It is important to note in this pilot "plug and play" means that the code for the tool is available, can be downloaded and integrated with a health system there is acknowledgement there may need to be some configuration done by developers.

Objectives

The objective of this pilot is to make the AHRQ funded My Preventative Care App more standards based and to develop a standards-based Prostate Screening App to demonstrate the resources, technology and approaches necessary to make components (Create, Disseminate, Integrate and Use Tools) of the ecosystem FAIR and computable

Goal of the Pilot 

The goal of the VCU Tool Refinement and Implementation Pilot is to make the My Preventative Care App and the Prostate Screening App more standards based. The results of this pilot will serve as a test case for making previously developed AHRQ tools more computable, FAIR and useful through standards-based tool development and refinement. Using the work done as part of the My Health Finder this pilot will work to convert the prostate screening information to a health system standard supporting the AHRQ digital platform. The pilot will focus specifically on five components of the ecosystem which include: Create Tools, Disseminate Tools (both locally and widely), Integrate Tools and Systems, Use Tools to Support Care/Decision Actions, and Gather Data about care Processes/Results. The pilot will outline the considerations and lessons learned in converting existing tools into those that support the AHRQ digital platform with the vision of incrementally making AHRQ guidance computable and more widely available. The information gathered by the pilot will be shared with the both the Collaborative pilot as well as the Evidence/Guidance Computability Tools Requirements Pilot and will provide input in to AHRQ Phase 1 concept demo activities (i.e., for AHRQ DKP and other knowledge ecosystem components, and Phase 1 evidence/guidance implementation/evaluation efforts.)

Approach

The VCU Tool Refinement project will work to take the My Preventive Care and Prostate Cancer Screening App and convert the current code from Structured Query Language (SQL) to CQL which can be supported by HL7 FHIR. Once the mapping and conversion is done the VCU Refinement pilot will work to test the outcome of the conversion and create documentation and code that can be stored and accessed on the CDS Connect web. The process will include:

  • Acquisition: Data Retrieval from the patient EHR using FHIR
  • Processing: Utilize CDS Connect Logic to process data against USPSTF preventive care recommendations
  • Delivery: Prepare relevant personal health education for patients use ACTS library APIs to extract resources from AHRQ

Progress of the Pilot

Currently the VCU Tool Refinement and Implementation Pilot is in the process of converting existing local instances of the prostate cancer screening tool in the MyPreventiveCare App (328) to a standard. (From SQL to HL7 FHIR and CQL). The goal is to have the mapping and conversion completed early 2021. Once this conversion/refinement is complete the resulting standards-based App will be tested at VCU. When a provider agrees to test the App, the app will be turned on for use. Patients with a scheduled wellness visit will receive an email asking them to use the Prostate Screening App prior to their visit. The providers will receive A summary of the patients' responses for review prior to the patient's visit. The evaluation metrics of this pilot will include how many eligible patients used the tool prior to their visit, and whether the patient opts to get a PSA based on the Apps recommendations. The output metrics as well as the output of the tool refinement/conversion of this pilot will provide the Evidence/Guidance Computability Tools Requirements Pilot real-world feedback for use when developing evidence and guidance. The outcome of this pilot will also help establish the feasibility of extending tool refinement to other areas of preventative care such as diabetes and lipids. Additionally, this pilot will work to help AHRQ determine where to store the results of these pilots utilizing a library or repository concept where the code sets can be stored, searched, and downloaded by other systems wanting to utilize the outcomes of this pilot.

Pilot Path Forward

These pilots are laying the foundation for—and in some cases beginning to generate—activities and deliverables called for in the ACTS Roadmap. In addition, they are providing knowledge ecosystem stakeholders with collaborations, insights, tools, and requirements today for better improving their activities and results around knowledge ecosystem cycle steps individually and collectively for COVID19 targets and beyond. As AHRQ and other stakeholders review the ACTS Roadmap and consider additional steps they can take toward executing it, these deliberations and actions will be coordinated with these ACTS pilot activities as desired and appropriate. The goal is to further build on momentum toward achieving AHRQ and other stakeholder goals that has begun from developing the ACTS Roadmap and progress so far within these