This website is under development - content not reviewed/approved by AHRQ or other stakeholders


Skip to end of metadata
Go to start of metadata

Initial Working DRAFT for Stakeholder Input and Refinement


Problem to be solved

There is no quick, easy, holistic, comprehensive view across the many different sources for clinical guidance/recommendations for a specific patient or group of patients. A holistic view of the recommendations includes the evidence, value judgments, currency, and other key dimensions related to the topic-specific recommendations individually and collectively. There isn't a current or easy way to determine differences between recommendations or when recommendations or evidence underlying them changes. Another challenge is the identification of important gaps in relevant topics or subtopics.

  • Across different guidance sources, the recommendations often differ in content (requiring implementors to make decisions about which to follow), often differ in format (making it difficult for users to gather and compare recommendations across sources), and often have ambiguities in how data to be gathered or actions to be taken are expressed (which can make it difficult to put the guidance into action).  All these adversely affect adoption of evidence-based, patient-focused guidance, and best care practices.
  • Where guidelines differ, it is not easy to distinguish "appropriate" reasons for differences (such as context) from "inappropriate" reasons (such as methodology).
  • Across different guidance sources, there is not an easy way to see what is missing:   recognize concepts for which there are no recommendations.  And if recognized, not an easy way to signal the need from stakeholders (e.g., clinicians, CDS developers) to guideline developers for such recommendations.

What does solving the problem look like - e.g., what new is needed? (future vision)

  • For a given clinical topic, users can easily access, evaluate, and compare clinical management recommendations on the topic provided by various authoritative, evidence-based sources. In addition, users can easily access, evaluate and compare the discrete/explicit considerations that led to the recommendation; e.g., quality of the evidence, and magnitude and balance of benefits/harms/costs associated with implementing the recommendation, etc.  Likewise, information from the guidance supplier about clinical/implementation considerations for applying the guidance can be readily accessed, evaluated and compared. These structured, computable recommendations can be readily adapted into clinical decision support interventions and clinical performance measures (see also CPG authoring portions of concept demo).
  • See Key Functions and User Stories in table below
  • social determinants of health considerations can be more easily factored into applying recommendations to specific patients (e.g., whether home anticoagulation is appropriate for a particular patient)
  • Support (and potentially prompt) broad and diverse input from all stakeholders across research questions, priority setting, guideline development, and implementation.
  • Include representation of the costs of action (and costs of inaction) where available for the recommended actions.
  • For gaps where there are no recommendations, ability to apply priority setting to inform use of resources to fill these gaps.
  • Get broad input into prioritization, e.g. https://ois.lbg.ac.at/en/projects/priority-setting
  • Show differences across guidelines and (to the extent possible) the reasons for the differences which can help distinguish "appropriate" and "inappropriate" differences.
  • (See here for work in progress toward providing this toolset)

Tool Users/Use Cases

Infrastructure needed to produce tools/solve problem

  • Need schema/standard for reporting a recommendation (there are standards emerging for evidence and CDS, but not yet for the "simple" expression of a recommendation and all it's contributing components [i.e., 'considerations' noted above in 'what does solving the problem look like' - e.g., rationale for recommendation, how evidence relates to the recommendation, costs/risks/benefits])
  • Need tools for human data entry so guideline developers can express these considerations without having to deal with JSON coding.
  • Need tools to convert already existing structured data that has these considerations encoded into the common structure.
  • Need tools to view (including search and filter) the data (i.e., 'recommendation summary browser) so users can see what they're interested in.

Other enablers needed to solve problem

  • Guidance developers need to agree to express their recommendations in standards-based, computable format
  • Need to bring health IT developers, guidance developers, and toolset users together to drive user-centered design for toolset
  • Collaboration tools to facilitate re-use, partnerships, and collective efforts in guideline development

Steps to address needed infrastructure / enablers  - Who does what?

  • Bring together stakeholders to define and apply needed standards - e.g., leveraging EBM on FHIR (and COKA) and related HL7 connectathons

How tool(s) fit in Patient Journey

  • See use cases below - e.g., patients and clinicians use toolset to review evidence/guidance related to specific clinical issues on care delivery 'front stage'; on 'backstage,' systematic reviewers and guideline developers/disseminators are using the toolset to create guidance-related information, and CDS developers/implementers are using the toolset to create CDS interventions and measures that are supporting patient and care team decisions and actions

Ecosystem Cycle Step(s) where tool is applicable

  • Aggregate/synthesize evidence
  • Produce Guidance
  • Create Tools
  • Disseminate Tools/guidance


Use Case Table for Recommendation Summary Browser

Tool for this SheetUsersKey FunctionOverall Goalschallenge/problemuser story 1


Recommendation Summary BrowserPatient, Clinician, Systematic reviewer, Guideline developer, CDS developerCreate a summary of relevant recommendations for a clinical situation including common subgroups (e.g. hospitalized vs. outpatient)for a clinical condition: 1) what are the recommended management approaches - and risks, costs and benefits of each - for specific patient groups; how should this management approach be implemented (clinical/implementation considerations - optimizing benefits/minimizing risks)?




Use Case IDUser Personashort namegoals/objectiveschallenge/problemuser story 1user story 2user story 3submitter
R1PatientFind the recommendation for me and the supporting evidence1. Find the specific recommendation for me.
2. Understand what recommendation would maximize benefit and minimize harm.
3. Show the supporting evidence specific to the recommendation.
4. Make an easily understandable form of the linked evidence easily accessible.
1. Evidence not easily accessible to patient and caregivers
2. Evidence not easily linked to a specific recommendation
3. Evidence not easily understandable
For my condition I want to know what is recommended for me.For a recommended treatment (e.g. a specific recommendation for an anticoagulant that applies to treatment for me) I want to know the associated benefits and harms and be able to share this evidence with my doctor.For a recommended treatment (e.g. a specific recommendation for an anticoagulant that applies to treatment for me) I want to know what will reduce my chance of bleeding and my chance of stroke.Janice Tufte
R2ClinicianFind the recommendation for a patient group and a way to see if multiple recommendations are consistent.View recommendations with:
a. dates of currency.
b. signal highlighting inconsistencies.
c. markers for strength of recommendation and certainty of evidence within and across guidelines.
D. supporting evidence
Recommendations from different authoritative sources may disagree or be incompatible.
Recommendations may be "outdated" if they are derived at different times.
For a specific subpopulation (e.g. COVID-19 in critical care vs. in hospital vs. outpatient) I would like a quick summary of recommendations from authoritative sources.

Surbhi Shah
R3Systematic ReviewerQuickly identify differences in recommendations with related factors.
Systematic review priorities may not align with priorities for active clinical questions.
Difficult to know when guidelines disagree.
With limited resources to prioritize work for systematic review development we would like to identify the issues where unanswered questions lead to weak or conflicting recommendations for care.Link to supporting evidence is important to allow independent assessment of quality of evidenceIdentify the priority questions for systematic reivew development. Identify recommendations limited by controversy or insufficient evidence.Mary Butler
R4Guideline developerFind the recommendations and basis from others that match my recommendation in development.1. Identify recommendations from other guidelines with "close enough" match to the recommendation in development.
2. Understand the basis behind recommendations (avoid repeat work)
Hard to find recommendations from other guidelines at the level of specificity to match my recommendation (population specificity and intervention specificity)I want to know what other guideline developers are recommending and what evidence they have gathered and what methodologies they used.Provide feedback loop to recommendation creator (from any other user/user persona).
Sandy Lewis
R5CDS developer/ implementerKnow when changes in recommendations should trigger evaluation for changing my CDS artifactMatch recommendations to CDS artifacts. Recognize when time of recommendation change is after time of evaluation for CDS artifact.Update signal for a guideline is NOT specific enough to know if it relates to my CDS artifactFor the CDS artifacts I have for specific decisions I would like a quick summary of recommendations and notice of times of publication/change to know when to reconsider my CDS artifacts.Represent regulatory changes and warnings (e.g. FDA notices) as "recommendation modifiers" to coordinate with the Recommendation Summary Browser use.
Maria Michaels
R6Guideline disseminatorShow my recommendation in Recommendation Summary BrowserBreadth of dissemination
Ease of dissemination
Hard to reach the right audience with the specific messageI would like to represent my guideline content in the Recommendation Summary Browser.I would like access to user data to learn the breadth of dissemination by user types for my recommendation.

  • No labels