Resource Developers’ Perspective of the ACTS Future Vision

The future vision workflows and information flows for care delivery and transformation (both at the organizational and national levels) are powered by health IT-enabled tools and resources. These include evidence-based guidance and answers to clinical questions, related tools for developing and executing patient-centered care plans, and many other offerings that support the virtuous LHS cycle. Many different organization types produce these resources aimed at making the right healthcare-related decisions and actions easy including guideline and CDS content developers, EHR vendors, other health IT suppliers, and many others.

In the future vision, knowledge assets (KAs) (e.g., individual clinical studies, systematic reviews, clinical recommendations, training materials, reference information, and many other information types) that these resource developers use in creating their offerings are FAIR, computable, and useful. For example, when manually seeking to find a KA or a group of KAs, resource developers can easily find the right KAs, all the right KAs, and nothing but the right KAs. In circumstances where the desire for the right KAs can be anticipated for the resource development task at hand (e.g., in keeping a clinical recommendation or CDS intervention up to date), the search and retrieval process is proactive, such as through precise, automated searches that notify developers when pertinent new literature is published.

Determination of right in the context of finding the right KAs is multifactorial with factors including relevance to knowledge application (e.g., PICO (41) specification of knowledge sought), certainty of the knowledge (e.g., quality of evidence or strength of the recommendation), compatibility with regulatory and contractual requirements (e.g., certification status, formulary availability), and the format for knowledge expression (e.g., PDF, links, executable code). Determination of right in the context of finding the right KAs is efficient because a person (e.g., a software engineer) seeking information can easily specify concepts of importance (e.g., the metadata dimensions that are tagged to facilitate search: PICO elements, certification status, creation date, artifact type, etc.) when searching. Search results are organized for further refinement based on these factors. Trusted experts and authorities have confirmed the accuracy of these factors, and they are communicated in universal standards for data exchange (to promote interoperability).

In the short-term future vision:

• Many of the KAs are manually discovered and assembled using search portals (see Figure B-1. Mock Portal Example 1—Evidence‑Informed Resource Portal and Figure B-2. Mock Portal Example 2—Evidence‑Informed Resource Portal Results)
• Common open-access points for finding KAs are available both online and through info buttons on health IT screens
• KAs can be filtered and sorted by relevance to appropriate audiences (e.g., patients or care teams)

Eventually, resource developers will have easy (ultimately automatic) access to updated, trusted KAs to inform the production of tools and resources that better address user needs.

In the longer term future vision:

• The assembly process is more automated, leveraging increasingly smaller component handling with granular resource tagging (PICO and other) and automatic identification as technology develops (AI, ML, NLP), and providing precise answers as easily as full documents
• APIs facilitate efficient identification of and access to comprehensive sets of KAs (including “push” notifications for emergent public health threats)
• Solutions provide context-sensitive support for user needs, with automatic population of resources in appropriate contexts, enhancing discovery and access (e.g., leveraging FAIR and computable KAs to support automated gathering of current recommendations and tools to prepopulate draft care plan templates for further processing by SMEs into care plan support tools that will be used by patients and care teams)
• AHRQ and other organizations better support development and updating of KAs and resources built from them because their offerings are validated (e.g., as meeting agreed-upon thresholds of methodological rigor and quality), tagged, packaged, disseminated, accessed, and applied more efficiently and effectively than is the case today
• Seamlessly integrated and interoperable virtual marketplaces enable resource developers and others to find, access, and reuse knowledge assets and tools efficiently
• Tools from resource developers are accompanied by analytics capabilities that provide data about resource use and impact, and this feedback informs continuous improvement in resource development and application, and helps generate new evidence to drive the virtuous LHS cycle

The Resource Developers’ Future Vision is further described in the B.4.1, Detailed Resource Developers’ Perspective of the ACTS Future Vision.

Detailed Resource Developers’ Perspective of the ACTS Future Vision

Purpose of This Section

Achieving the Quintuple Aim (22) and the CDS 5 Rights (29) requires collaboration of all healthcare stakeholders and participants, including those who develop evidence-based guidelines and decision-support resources, as well as health IT suppliers such as EHR and population health management system vendors. This section frames the perspectives and future vision of these resource developers, defined as organizations that create evidence-informed tools and resources to support care delivery and care transformation.

This future vision illustrates how AHRQ-supported and other knowledge artifacts (KAs) (e.g., care plan templates, decision support tools, guideline recommendations, systematic reviews) can be better used to create, update, and deliver these offerings. The future vision addresses new ways evidence-informed resources and the KAs upon which they are based will be created, discovered, and used. In this future vision, resource developers could access AHRQ offerings and other information more efficiently and effectively than they do currently to produce and deliver the decision support interventions required to optimize care delivery and transformation.

Resource Developers’ Future Vision Executive Summary

Suboptimal Current State

In the current state, resource developers seeking KAs to develop their care support tools and resources face tremendous inefficiencies, resulting in products not optimally suited to meeting user needs:

• KAs are scattered across hundreds of databases and websites
• Search solutions that cross hundreds of sources do not have limiters to identify validated and relevant KAs
• Many KAs of interest are not available via APIs (275)to directly support automated retrieval
• When multiple KAs are found, there is no easy method to distinguish duplication (repetition of the same knowledge in a different artifact) from replication (reproduction of results from additional information sources, a finding which increases certainty of the knowledge)
• When multiple KAs are found, there is no easy method to:
  • Determine consistency of knowledge across the KAs
  • Ascertain the certainty of knowledge within them
  • Compare and contrast these multiple KAs on the basis of any standards or quality assessment tools
• There is no consistent interface between KA sources and health IT systems to provide evidence to users in a timely, context-sensitive manner

Ideal Future Vision

In the future vision, these inefficiencies will be largely resolved and resource developers seeking KAs to develop or work with their care support resources will be instantly supported by the following improvements:

• When manually seeking to find a KA or a group of KAs, they can find the right KAs, all the right KAs, and nothing but the right KAs
• In circumstances where the desire for the right KAs can be anticipated, the search and retrieval process is automated and not dependent on manual triggers
• Determination of right in the context of finding the right KAs is multifactorial with factors including the following:
  • Relevance to knowledge application (e.g., PICO (41) specification of knowledge sought)
  • Certainty of the knowledge (e.g., quality of evidence or strength of the recommendation)
  • Compatibility with regulatory and contractual requirements (e.g., certification status, formulary availability)
  • The format for knowledge expression (e.g., PDF, links, executable code)
• Determination of right in the context of finding the right KAs is efficient because:
  • The person (or system) seeking information can easily specify concepts of importance when searching (the metadata dimensions that are tagged to facilitate search (e.g., PICO elements, certification status, creation date, artifact type, etc.)
  • Search results are organized for further refinement based on these factors
  • Trusted experts and authorities have confirmed the accuracy of these factors
  • Factors are communicated in universal standards for data exchange (to promote interoperability)

Current State

Resource developers provide disparate tools that support care plan development and execution, and otherwise support care delivery and care transformation; but they do so piecemeal, which is inefficient and does not optimally meet user needs (including quick and easy access). EHR and other health IT system content depends upon resources provided by guidelines and CDS developers, but those are currently limited, with inconsistent quality and poor discoverability, access, and navigation at the right time and place, especially at the point of care. As a result, implementation and adherence rates (276) (key drivers (277) for optimizing quality of care, clinical outcomes, and cost effectiveness) are low for current guideline recommendations and CDS artifacts.

The time lag between scientific discovery and integration of new knowledge into clinical practice resources remains challenging. Few clinical practice guidelines (CPGs) employ digital data with regular or continuous literature monitoring to keep the evidence base current and thereby provide recommendations that are persistently kept up to date. The same applies to other types of CDS tools. Health IT vendors are not able to classify how up to date various tools and resources they consume and produce are, much less have the capabilities to easily replace outdated ones with new versions.

AHRQ resources are intended to be useful for various audiences including evidence-based guideline developers, CDS providers, patients, healthcare professionals, and others; but there are limitations and barriers:

• The USPSTF guidelines address only prevention so guidelines addressing diagnosis, screening, staging, treatment, management, and other areas must be developed by organizations external to AHRQ. In the United States, these are primarily developed by medical professional societies.
• The National Guidelines Clearinghouse was terminated with no replacement yet authorized, eliminating a popular database for searching and identifying guidelines that meet explicit criteria. Similar considerations apply to the National Quality Measures Clearinghouse.
• EPCs are charged with conducting comprehensive systematic reviews. However, the EPC reports are long and not always as useful for guideline development as many medical professional societies may have hoped. They often do not address the specific PICO questions posed by the requesting organization, since other stakeholders are allowed to modify the PICO elements. It is difficult and time-consuming to update these reviews to keep the guidelines current.
• The concept behind the SRDR+ was intended to help guideline developers update the digital dataset as new evidence emerges, thereby supporting ‘living guidelines’ that are more easily kept up to date. But SRDR+ is mostly technology-focused and missing the services component, resulting in complex and tedious work for guideline developers. The medical professional societies do not have the staff or volunteer resources to extract the data into the digital environment and are not comfortable with sharing the data extracted by others, whose skills they do not know or trust.
• The AHRQ resources alone generally do not cover the full range of information most resource developers need because these resources usually cover only a limited set of topics and healthcare questions (that is, they don’t address all the questions people face when making healthcare decisions). For example, a systematic review addressing the benefits and harms of a surgical procedure may not address recovery time or experience, which are common patient concerns. When the AHRQ resources do approach a topic comprehensively and holistically, the material may not be updated continuously so resource developers still need to search for new evidence, regulatory changes, or other information to have the most current, valid, and relevant content for real-time use.

Future Vision

Short-term Future Vision

The short-term future vision calls for a search portal to efficiently locate and identify knowledge artifacts matching search criteria facilitated by metadata tagging, including PICO and research design elements, and general knowledge artifact types.

Guideline developers will collaborate with CDS creators for guideline-based CDS and other knowledge artifacts. Computable and automatically updated recommendations and CDS resources will be tagged and uploaded into web-based platforms for all healthcare domains, resulting in a one-stop search/browse experience across repositories where these offerings are housed. User-friendly UIs will make it easier for resource developers to access information and tools they process into their offerings. This content will be discoverable through human-triggered, PICO-based searches.

Guideline recommendations and CDS resources will be developed in compliance with accepted standards to improve quality and reduce redundancy. In the case of multiple guideline recommendations or CDS tools addressing the same clinical topic area, there will be a comparison tool based on appropriate criteria for validity, relevance, and trust (e.g., recency of publication, quality of the underlying evidence, geographical target audience, and standards for recommendations, such as some of the latest Appraisal of Guidelines for Research and Evaluation (AGREE) instruments (278) for assessing the quality of guidelines) but also a way to identify consistency or inconsistency when multiple recommendations or tools are relevant and sufficiently valid.

Governance and enforcement of inclusion and exclusion criteria in portals that provide access to vetted content will ensure end-user confidence in retrieved results. For example, validating that structure and content standards are followed confirms that the methodological rigor is sufficient and the KAs are evidence-based; thus, users will be able to trust the quality of the tools and resources. This governance will, therefore, require assessments of adherence to accepted methodological standards. Governance will address intellectual property (IP) rights and subscription-based access. If a KA is not freely provided, pay-per-view or subscription purchase will be options.

Development of user-friendly interfaces is paramount to the short- and long-term success of embedded CDS. CDS creators, EHR/health IT developers, and end users will work together, on a continuous basis, to develop and refine tools which provide context-sensitive, actionable recommendations to patients and their care teams in real time through pertinent information systems (e.g., EHRs, patient portals) with minimal disruption to respective workflows.

The portal (see Figure B-1. Mock Portal Example 1—Evidence‑Informed Resource Portal, Figure B-2. Mock Portal Example 2—Evidence‑Informed Resource Portal Results, and Figure B-3. Care Plan Support Tool Mockup Interface) to make evidence-based resources and knowledge findable and accessible to all stakeholders will be adaptable to meet the needs of different stakeholders. For resource developers, this means the ability to specify the types of knowledge they seek, including both resources designed for end users and those providing “raw materials” that resource developers use to create new tools and products. This also means supporting API access to the portal so that resource developers can create sophisticated and automated methods to search for their needs, including on-demand and continuous search to alert them to new knowledge.

Because resource developers are themselves creating resources to be FAIR (59), it will be most efficient if the resources and “raw materials” they find are poised to be interoperable. In the healthcare domain, the predominant standard for data exchange is FHIR (279). FHIR itself is being extended to support data exchanged for biomedical evidence and statistics with the creation of FHIR Evidence Resources (146). Knowledge resources that contain evidence and statistics can be stored in computable form (that is, the evidence and statistics can be expressed in machine-interpretable granular specifications defined as FHIR Evidence Resources (280)). Knowledge resources that contain executable code (that is, knowledge resources that include UIs or expressions for how data should be processed and displayed) will be provided in SMART (281) frameworks that support interoperable use across electronic health systems.

The Longer Term Future Vision

In both the short- and long-term visions, technology will support user-generated information requests. But only in the long-term vision will automated information automatically populate the right information and tools directly into the pertinent information system when triggered by appropriate PICO preference selections for the offering under development.

Resources developed to inform and support care delivery need to be developed as precisely as possible and in an automated manner to be implemented by healthcare professionals, patients, and care teams. This is a fundamental goal for all resource developers as they aim for their products to be relevant, context-sensitive, and automatically displayed at the right times, in the right places, for the right users, in the right formats, and on the right channels (information systems/places, at the right point in the workflow). The longer term vision defines more granular components of these knowledge artifacts (e.g., logic rules, recommendations, evidence assertions) for computable expression. AI and/or machine learning (ML) will assist in the production and use of a large collection of searchable, reusable knowledge. Where ML/AI is unable to achieve the goal directly, the care resource developer may use ML/AI to facilitate human curation for achievement of this goal.

The key ingredients in the care delivery future vision are integrated, patient-centered care plans that are generated from templates that preassemble and integrate evidence-informed guidance, tools, and resources to address the patient’s clinical issues (including preventive care), individually and collectively. (see Figure B-1. Mock Portal Example 1—Evidence‑Informed Resource Portal, Figure B-2. Mock Portal Example 2—Evidence‑Informed Resource Portal Results, and Figure B-3. Care Plan Support Tool Mockup Interface). Resource developers will need to produce “care plan generators” that provide templates for condition-specific management plans and tools for combining and customizing them for patient-specific goals, preferences, and circumstances.

These care plan generators (and related tools for supporting evidence-informed decisions and actions by all those giving, supporting, and receiving care) will be generated in ways that they can proactively and automatically populate EHR/PHR screens through AI/ML retrieval when triggered by related PICO concepts and/or patient preference terms. These tools will comprehensively address prevention, screening, diagnosis, treatment, and management for acute and chronic conditions, as well as commonly occurring constellations of multiple conditions. They will be able to display the relevant guideline recommendations, CDS tools, and underlying sources. They will also be linked to tracking mechanisms so that intervention effects can be determined. Content will be AI/ML supported to pull all appropriate KAs for display with descriptive information. PICO annotation will drive the underlying identification and tagging of KAs to matching search queries or for automatic population. All clinical taxonomies (Systematized Nomenclature of Medicine [SNOMED], Medical Subject Headings [MeSH], National Cancer Institute Thesaurus [NCIT], International Classification of Diseases, Tenth Revision, Clinical Modification [ICD-10], Current Procedural Terminology [CPT], etc.) will be mapped so that a single term or concept will pull all synonyms. The software would continue to “learn” as it is used.

To help those working to transform care with gathering and applying pertinent information and tools, resource developers will provide a new generation of care transformation support resources. These offerings will help QI teams identify, access, and use the resources to manage entire QI initiatives or address particular components of such initiatives (e.g., configuring health IT tools to support the target or gaining stakeholder engagement in the improvement effort). These care transformation support resources are analogous to the care plan generators in that they precompile high-quality, evidence-informed resources so that they can be accessed and applied more efficiently and effectively to achieve complex goals.

The knowledge management tools mentioned above that resource developers use to create their offerings—together with other LHS ecosystem enhancements described elsewhere in this Roadmap—will support resource developers in creating these care plan generators and related tools.

Decision support tools will be interoperable across settings and information systems. For example, care teams will share evidence-based decision algorithms, treatment decisions, and care plans across EHR platforms and vendors. Similarly, sharing of questionnaires and completed decision aids will be possible between care teams and patients, EHRs and PHRs, and across other health IT platforms.

Resource developers will provide capabilities to generate analytics at a high level for the short-term platform usage but at a much more granular level within the longer term vision in the health IT system (e.g., EHR or PHR/patient portal). To generate insights that contribute to an LHS, systems will monitor, report, and support analyses on:

• The frequency with which users consult specific recommendations
• The frequency with which users adhere to the recommendations
• The likelihood that the recommendation had an impact on the patient outcome (e.g., via by EHR vendors’ enhanced capabilities to better track outcomes as they evolve to include enhanced-support LHSs)

As in the near-term future vision, governance and enforcement of inclusion criteria will be necessary to ensure end user trust, enforcement of standards, and the ability to assess the quality of the tools and resources. Methods and process standards will be judged based on established rules to ensure methodological rigor and that the quality meets accepted standards. These may change over time as real-world evidence becomes more usable in precision medicine and as technologies provide greater analytic capabilities. The governing bodies will need to lead these advancements in the field and support educational awareness and knowledge.

Governance will also maintain intellectual property (IP) rights, but subscription-based access—when KAs are not freely available—will be discouraged so that all relevant resources become freely available to all end users.

Figure B-4. Ecosystem of Resource Development & Feedback is an overview of how inputs to and outputs from resource developers will help deliver the future state for care delivery.

AHRQ & Other Assets to Support the Future Vision

AHRQ resources will become more valuable and useful for resource developers.

The SRDR+ should be serviced only by the trusted methodologists of the EPCs basing the extractions on the relevant PICO elements and other data configuration specifications. Software to aid search, screening, and extraction will be employed to reduce errors and increase efficiencies, while maintaining a focus on accuracy. The extracted data can then be trusted and shared by all users. All methods and processes will be documented in the metadata, including any calculations created or results converted. The digital dataset can be expanded as new research is published, pooling all relevant data for updated analyses, again by trusted EPC methodologists. Previous results can be compared with new results using the same saved workflows. Literature should be monitored regularly and guideline developers alerted when new trial data are published that matches the PICO elements of interest. This can be facilitated by PICO-annotation of the new publications and by AI-identified elements of relevance.

Methodologists would write less of a report and more of an explanation of what each of the analyses reveal, including strengths and limitations in that specific body of evidence. The descriptive explanations of the analytic results would be tagged by PICO element and appear with the results of the meta-analyses or network meta-analyses. Subsequent cohort or sensitivity analyses could be commissioned and would also be accompanied by quality interpretations. Personographs and other data visualizations will be produced for each meta-analysis with the ability for customization. All of this content would be accessible through a PICO-based search of the contents of the SRDR+.

The recommendations that are produced by guideline developers, including those of the USPSTF, would become granular knowledge artifacts that CDS developers use to create tools for shared decision-making. This content (including USPSTF, CDS Connect, other guideline recommendations, and other CDS resources) will be context-specific for patients, physicians and other healthcare professionals, and care managers. The content will be uploaded into a freely available, user-friendly, and easily searchable repository, possibly available through an existing source (e.g., PubMed or Google) or a new source. They will be tagged by the patient characteristics, interventions, and outcomes upon which they were created, and will be associated with the supporting evidence, analyses, EPC explanations or reports, references, and other relevant content, as listed above. Links to FDA alerts, drug labels (kept up to date by regulatory feeds), and reimbursement codes should be tagged to surface for PICO-relevant content.

API methods for CDS suppliers and others to search and receive AHRQ and others’ knowledge artifacts should include assignments to permanent URLs and easy-to-copy reference citations. Tagging the knowledge itself (in addition to the knowledge artifacts) would require tagging the many specific concepts like terms, phrases, statistics, and the many relations across the terms, phrases, and statistics. Eventually, AI techniques might help make this less labor-intensive. This can be facilitated by ontology mapping of all clinical taxonomies to surface all synonyms with single search terms for each concept. All parties would have to use the same standards and systems, such as the current effort to produce a global standard for computable expression of evidence and statistics, known as the HL7 FHIR Resources for EBM Knowledge Assets project (EBMonFHIR) (146).

AHRQ will play an active role in closing the feedback loop by monitoring the usefulness and impact of clinical recommendations. The interface between physicians using embedded CDS and guideline developers analyzing the impact of the recommendations will facilitate the development of evidence-generating medicine as an essential component of the LHS.

All users will have free online access to this master index for all of healthcare, therefore becoming the single best resource to access this content, regardless of which organizations (AHRQ or others) developed the various knowledge artifacts and tools. Eventually, they will surface in the EHR and other health IT screens when appropriate keywords or concepts appear in the proper combinations. Additional resources will also be available through convenient links for users who want to explore these or drill into the underlying evidence. (Refer to the Care Delivery Perspective of the ACTS Future Vision for more details.)

Additional Considerations for AHRQ

It took IOM, now the National Academy of Medicine (NAM), standards to have a significant impact on adopting more evidence-based and standardization methods and processes. It might take a similar high-profile effort to facilitate some of these recommended changes. Furthermore, there should eventually be methodologies that support precision medicine with customized recommendations based on aggregated real-world evidence, but the IOM standards do not currently support these processes. AHRQ should consider sponsoring a new NAM committee to review and update the former standards, within the context of advances in technologies, translation sciences, and precision medicine.