Data-to-Evidence Community Window

This page is for organizations and initiatives focused on the data-to-evidence portion of the knowledge ecosystem cycle. The purpose of the documentation below is to support efforts to use data about care delivery and results to generate evidence, meet reporting requirement, and activities. The first set of tables apply broadly to COVID-19 (and other targets), and the second set of tables refer to evidence and guidance specifically related to the Collaborative’s current focus targets; i.e., COVID-19 testing and triage, and anticoagulant use.

Those providing data for these tables should consider, "What do you want those who seek to implement 'living guidance' to know about 'data-to-evidence' strategies, initiatives and tools to make their efforts more successful? Likewise, "What do you want to know about efforts of others working in the knowledge ecosystem to make your data-to-knowledge efforts more successful?"

10/22 Data to Evidence Session:

• Recording

Notes from 10/22/20 Kick-off Call for Community

     Upshot

• What would be helpful in addressing the opportunity to leverage data across multiple care delivery organizations to support generation of new evidence is a new tool that parallels the 4 tools outlined on this page (Sections D and E) that assists with gathering and processing EHR data in ways that support its appropriate use as a type of evidence to support care guidance and decisions. Next steps to move this forward are TBD.

     Notes

• Maria: some organizations (e.g., Norway initiative) looking at clinical data directly and consider its implications for providing guidance - e.g., augmenting clinical trial results to inform guidance on the topic. We should think about how this fits into the other work we're doing. MedMorph - looking at how to use FHIR to send data to endpoints like public health, registries, research to generate evidence, etc. Maria addressing focusing on processing data across organizations - e.g., as they're doing in Norway and medical societies (e.g., ACS - to determine if a guideline should be updated).
• JO: standards/value sets are a key infrastructure for this part of the cycle - and for making this part of the cycle work smoothly and in concert with the rest of the cycle. 
• Brian walked through slides 5 and 6 in this deck - re: standards about things like patient groups to support flow around the cycle.

• Note from Maria in a separate email after this call as part of a different exchange, but related: "The piece that I brought up is in addition to CPG-on-FHIR. CPG-on-FHIR is about how to create a faithful representation of the guidelines/guidance in computable form (i.e., the computable guideline) and then develop derivatives (e.g., CDS, eCQMs/dQMs, case reports) from the computable guideline. Those items are used to pull together data, for example, from an EHR to meet certain business rules, etc., but there’s also the piece about having the means to exchange with other organizations (e.g., from an EHR (or multiple EHRs) to a registry). That’s the piece that MedMorph brings into the picture for research and public health (eCQMs have the DEQM which is specific to quality measures). This needs to be part of the picture otherwise we’re not moving the data from one entity to another to be able to do the kinds of multi-organizational analyses that we talked about today. All of these pieced fit together (e.g., CPG-on-FHIR, MedMorph, EBM-on-FHIR, even DEQM) because we’ve been working hard to keep them aligned everywhere it makes sense to do so and highlight complementarity where it makes sense to do so… it’s just a matter of making it clearer in this ACTS work so we’re all working off the same playbook."

Follow-up email about this meeting from Brian Alper:

If trying to develop a “tool 5” or a tool to facilitate Data-to-Evidence data exchange a next step could be to consider which of the following “tools” is most desired up front, and other tools may be added as the ecosystem develops.  In this context I used the term “study” to broadly describe the data collection and analysis to produce the evidence, regardless of study quality or whether it is called a study in common language.

1. A tool to share the “study protocol” – the definitional/instructional/guidance concepts not specific to the individual participants, similar in a structural sense to sharing the “guideline”
2. A tool to share the “research subject” – the identifying knowledge artifact that can identify the study participant and provide the links (with necessary security and anonymity) to recognizable identifiers, consent documentation if necessary, and contact information if necessary.
3. A tool to share the “research data” – data in the form needed for a usable dataset for the study
4. A tool to convert “real-world observations” (eg EHR data not captured for research purposes) to “research data”
5. A tool to facilitate “data analysis” – a tool to process the dataset and produce “evidence” (summary of analysis results)

There are multiple large efforts doing things in this area --- much still to be done but it should not be starting from scratch – but it may take a while to figure out which of the various efforts are a “good fit” and “open to collaboration”.

Notes from 10/16-19/20 email exchange

From JO:

"No question that 'data to evidence' is critical to the Collaborative's knowledge ecosystem enhancement effort.  Sandy asked to be included in the exploration, and Sara and Brian mentioned in the chat too - all copied on this note. Also looping in Jenna, whom you noted has been a partner for you in representing this need throughout the ACTS journey, and Ben Hamlin, who may be very interested in this as well. Including Surbi, Chris and Matt, since the UMN anticoagulation results could be part of an effort to demonstrate better flow through this link.

It would be great to strengthen how we're attending to this link in the chain - especially critical in the COVID-19 era. We could establish a new 'Collaborative Participant Window' devoted to this link in the cycle (i.e., "Apply Results (QI/Create Evidence)"). If we do this, would you be able to shepherd contributions to this page (along with other interested) and work with me to make sure we're connecting the best we're surfacing about this link with the other links we're working on in the Collaborative?"

From BA:

"EBMonFHIR (COKA) works at the “evidence part” of the knowledge ecosystem and FHIR.   “Evidence to Guidance” as you know works with CPGonFHIR, CDC ACG, C19, etc. and the HL7 CDS Work Group

In the other direction (“Data to Evidence”) participants include HL7 Biomedical Research and Regulation (BRR) work group and an HL7 FHIR Accelerator named Vulcan.   We are helping BRR map out FHIR ResearchStudy and FHIR ResearchSubject resources."

From MM:

"Partially echoing Brian, the “data to evidence” part of the cycle includes some aspects of research but also things like public health and quality reporting.

So in addition to groups like HL7 BRR and Vulcan, there’s also Da Vinci (quality reporting), and Making EHR Data More Available for Research and Public Health (MedMorph) (research and public health data exchange) that all fall into the “category” that I think we’ve been leaving out in our ACTS discussions."

From MB:

"Related to DaVinci, should include DEQM- Data Exchange for Quality Measurement.

The "Rosetta Stone"/ Holy Grail would be to use the same Concepts/Terms/Codes as appropriate throughout the entire cycle, NOT just the same Terminology Systems.  And I won't even go down the path of using the same Information Model as both are needed to get truly accurate semantics (OM.  CC-in Victor Lee and Andy Kanter in this regard."

From VL:

Piggybacking onto Matt’s comments, the alignment on semantics is in fact the principal goal of the COVID-19 Interoperability Alliance (https://covid19ia.org). In collaboration with various community stakeholders, we’ve authored more than 600 value sets and have made them freely available from both the Alliance website (no registration required, just download away) and VSAC (registration for a free account is required by NLM). If there are specific inquiries or even requests for value sets, I’d be happy to discuss. It’s our way of giving back to the community and making an impact on the pandemic.

From SL:

And to add further to this conversation, the 'data to evidence' portion is the key to precision medicine that we hope to achieve someday. So Big Data play a huge role here. IBM Watson was/is trying to move in this direction. I believe the oncology-focused Flatiron Health is also a player but would need to confirm that. The ASCO CancerLinQ would certainly be in this space. 

Additionally, we should think about all registries. If we could get all registries using the same standards and terminologies in an interoperable way, we could really move the needle.

From JN:

Also measures & common data elements for research (e.g., PhenX, NIH CDE Repository)… ideally aligning data standards across clinical care, registries, public health surveillance & research (to the extent possible).

Several NIH research efforts (e.g., RADx) are aiming to align research measures/CDEs for COVID 19.

From MM:

While it’s not COVID-19-related, cancer reporting is one of the use cases we’re modeling in MedMorph, and mCODE already provides a head start towards semantic interoperability (it’s not quite there yet, but that’s part of the goal). Bouncing off of Sandy’s comments, there might be something there to explore further.

Indirectly COVID-19 related, cancer was one set of the chronic diseases that saw a dip in timely treatment especially earlier on in the pandemic, so there’s probably still a COVID-19 angle even with cancer… plus cancer is actually a lot of different diseases under that one umbrella. On the semantic interoperability end, we could look at cancer data elements that are also COVID-19 data elements (e.g., cross-disease semantic interoperability). Thinking out loud here, but seems like something we can at least discuss. Every state in the U.S. has a cancer registry with cancer reporting required by law.

There’s definitely plenty of opportunity just with COVID-19 too.

From SS:

Absolutely agree with mobilizing the resources and bringing them on a common platform as suggested by all . I am happy to help out in any way. 

Overarching Description of Data-to-Evidence Community Best Practices, Tools, Resources, Initiatives (see especially 'Apply Results' column at the end of these tables)

Stakeholders can place comments at the bottom of any Learning Community page. If you need editing access to these Participant Window pages, please contact support@ahrq-acts.org.

Drill Down into Data-to-Evidence Community Best Practices, Tools, Resources, Initiatives For ACTS Collaborative Focus Targets  (see especially 'Apply Results' column at the end of these tables)

COVID-19 Testing/Triage (ED and ambulatory settings)

Anticoagulation