• Print

Proposed Interoperability Standards Measurement Framework Public Comments

Proposed Interoperability Standards Measurement Framework Public Comments

The comment period for this document ended on 5 p.m. ET on Monday, July 31, 2017. Comments for this document will be posted shortly.

The document solicits feedback on a proposed industry-wide measurement framework for assessing the implementation and use of health care interoperability standards. Currently, stakeholders’ capabilities to measure and report on the use of standards vary significantly across the health IT ecosystem. This framework aims to help health IT developers, health information exchange organizations, and health care providers move towards a set of uniform measures to assess interoperability progress.

Please note that comments and recommendations submitted as part of this process will be made public by ONC.


Carol Chouinard
Tuesday, June 6, 2017 - 19:30
Dear ONC stakeholders,
Thank you for providing us with the opportunity to share comments and feedback on your proposed model. No one can argue against the need and the value of measuring the adoption of interoperability standards. Measuring our progress is a very important activity in the journey toward universal interoperability.

I would offer the following observations:
1) The industry would benefit from a more comprehensive definition of interoperability standards and/or standard levels. Although most stakeholders will refer to “interoperability standards” as a set of data values associated to defined set of data elements, it can also refer to the format used to express these values as well as the standard protocols used to exchange or transport the values.
2) There is limited value in measuring health system or vendor specific standards; the only real way to measure national interoperability progress is to measure the level of adoption of nationally defined and adopted standards (ICD-10 codes, medications and NPIs being the obvious examples). For example, there is limited value nationally in knowing that health systems have standardized on their own patient identifiers, order-sets, or their own list of gender identifiers.
3) The ultimate interoperability limiting factor is the non-standardized, sometimes unstructured, and rapidly evolving nature of health care data. Most health systems have to process non-only “current data”, which may comply with current standards, but they have to process historical data, potentially based on different standards. Then they have to deal with new types of data being introduced by precision medicine technologies, by connected care (e.g. wearables and home monitoring devices) and other patient generated data.

To address these challenges, I would suggest that ONC considers the following recommendations:
- Limit the measures to standards that are well defined and generally accepted nationally and expand that set of standards over time.
- Push for automated reporting – as opposed to using surveys. As you mention, most vendors have the ability to monitor and report on the transactions. They also have meta-data tools that can be leveraged if the scope of the transactions is realistic and practical.
- Consider measuring which portion of the health care data set is actually covered by well-defined and nationally accepted standards – independent of adoption levels.
- Consider adding a “sharing” measure, i.e. monitoring how many health care providers can share a given patient record. With the consolidation taking place in the industry, and increasingly more stakeholders sharing platforms, national interoperability will greatly benefit from the increased level of sharing (as opposed to focusing only on exchanging). And in many cases (between hospitals and affiliated clinics for example), sharing the same platforms and patients is far more efficient than any interoperability solutions – that should be recognized and measure as well!

I hope this is helpful,


Maurice Thomas
Friday, May 19, 2017 - 15:13
Very good framework
Montra May
Thursday, May 11, 2017 - 20:15
I look forward to providing feedback after review of the document.
Cynthia Mofarrah
Thursday, May 4, 2017 - 01:59
I am working in the HIT and interested about standards.
Piet Vlemmix
Thursday, May 4, 2017 - 01:03
Cymry Reardon
Wednesday, May 3, 2017 - 11:54
Doing my first read through, I noticed I had several additional questions I wish this proposal addressed:
When you mention Health IT developers publicly reporting, I'm wondering where to? Additionally, if there is an annual date per year (or if just roughly once a year?)?
Lastly, where is the over site. The pile of data would be fantastic, and I agree knowing the standards of use would be incredibly helpful - but the data needs to be compile, organized and maintained. How do we do that and where? How do we publicly promote this data so that its widely known and available?
Lastly, I was wondering how we encourage industries which are falling behind to speak up and present they're data fully and accurately. I feel like I can find plenty of success stories of large companies investing and succeeding, but how do we spot light the smaller firms, and whats more; support them? The US is covered in plenty of small companies that manage this on a daily basis, and they're data would be incredibly insightful.
1) I think if we're going to implement standards reporting, it has to be mandatory. Otherwise, only the industries and companies which can easily afford to will participate. In order to get the best possible data, from the most widespread sources, we need to implement this for everyone.
2) an annual survay or even a written progress report would be great. Something that not only tracks what software and updates are being used, but also how much, why and how that use is reflected in the interactions with the patient.
3) I think so? I really admire the goals, and think they are desperately needed.
4) I think we're lacking the overview; what we're doing with the end data, and how to interpret what it means.
5) I think personal injury firms, which not only aggregate data but communicate with both doctors and insurance companies; mediate care and manage payment of costs should be a voice in this discussion.
6)I think monitoring this data is something our industry sorely lacks, and while I think it is a huge task we are laying before ourselves, without it we will be charging forward without checking our math, so to speak. I think implementing this kind of standard will create more jobs and grow out field, too. We need to be a loud voice in the industry and checks and balances are a part of that.
7) I definitely believe it's feasible. At this point, many organizations require an annual review of they're performance. Including submitting data to ONC shouldn't be overly burdensome, especially considering we hope to streamline the process and provide stakeholders with resources on the overall effect of policies. I think the challenge may be in nailing down a date/deadline, since so many companies and industries operate differently.
8) Surveys on what standards are used would allow us to monitor and measure only the standards that are absolutely necessary. I think the idea of saying "its not possible to measure everything so we should pick and choose" will limit our data and make the end goal more difficult to determine. We need to know what the entire industry uses, how often, and why. I think we need a broad overview and then we can focus down on what we're actually using after analyzing the data.
9) I think creating a common vocabulary and definitions is part of this process. Would it be possible for ONC to create a vocab list? If the end goal is to great a guidebook, basically, then defining everything accurately so we can all use the same terminology should be the first step in the processes. If ONC is the body starting this processes, I think its feasible that they (you) establish the nomenclature.
10) I think this is the most complicated question asked, at least for me. I've read through the proposal twice and I think that tracking this while not violating the security we've worked to establish would be complicated. But in the end, I think we need some kind of tracking software that monitors use without retaining ePHI or any other confidential data. We need to know in what way software is used, even if we can't/shouldn't track the specifics that are transacted.

kavishwar wagholikar
Tuesday, May 2, 2017 - 18:28
Sherri Douville
Monday, May 1, 2017 - 11:10
Dear Sir or Madam,

This measurement framework is a good talking point and educational tool to help separate some of the issues. A few thoughts from an early stage Health IT vendor perspective:

1) As an early stage digital health company, we would noly integrate with an EHR based on customer request and need only. We do not consider this to be a true a "standards-based" integration. For any integration be useful, it would typically be a custom integration involving the partnership of the customer's IT leadership.

2) As an early stage company, we would consider attestation to standards at this time to be an impractical, enormous regulatory burden prior to FHIR being fully built out (Fast Health Interoperability Resources).

3) We respectfully request that any vendor-facing activity or regulation is enforced only for the results intended. We want to discourage the tracking and reporting of anything that doesn't drive real results for interoperability.

Briefly, the way we perceive the challenges are: There’s existing standards, HL7 v2 and C-CDA, (all pre FHIR) and they are not going away. People are going to continue to use those systems because they can exchange lab orders and lab results, between the EHR and the lab system. In addition, different parts of the FHIR standard for packaging discrete bundles of data are not yet built out, particularly those parts that deal with complex clinical data elements such as specialties and genetics. The complex clinical portions are where interoperability is needed to care for patients with multiple co-morbidities efficiently. As HL7 board member Dr. Russell Leftwich points out in Healthcare Informatics, "What needs to happen is that the entire FHIR community and the clinicians in particular need to get together, one data element at a time, and agree on a model, or how to represent that piece of data. You have to have that to really have interoperability across organizations."

We would encourage ONC to focus on how to make progress with FHIR as quickly as possible with the right incentives. Without FHIR, interoperability amongst different brands of different systems, interoperability between different systems will continue to remain as custom built solutions. Therefore, reporting and tracking from our perspective appears irrelevant. FHIR interfaces will take hours or days instead the weeks it takes to build HL7 interfaces. If there really is a drive to move the needle on interoperability, there needs to be incentives i.e. financially positive for this work to come to completion. Perhaps this could mean making a very large grant available for various conveners such as the CARIN alliance to apply for and organize and drive specific portions, or maybe it's grants to specialty societies (American Academy of Cardiology etc.) who as part of the grant have to pay and assemble the clinicians needed to build specialty and genetics FHIR elements. In our view, there would need to be monetary compensation made available to the clinicians who agree to take on the needed leadership roles to drive representation of data elements and the models. Clinicians already feel overburdened by the current documentation requirements. While clinicians' input is needed to drive interoperability, they should be fairly compensated for their time in our view.


Sherri Douville
CEO, Medigram Inc.