• Print

Proposed Interoperability Standards Measurement Framework Public Comments


Proposed Interoperability Standards Measurement Framework Public Comments

The comment period for this document ended on 5 p.m. ET on Monday, July 31, 2017. Comments for this document will be posted shortly.

The document solicits feedback on a proposed industry-wide measurement framework for assessing the implementation and use of health care interoperability standards. Currently, stakeholders’ capabilities to measure and report on the use of standards vary significantly across the health IT ecosystem. This framework aims to help health IT developers, health information exchange organizations, and health care providers move towards a set of uniform measures to assess interoperability progress.

Please note that comments and recommendations submitted as part of this process will be made public by ONC.

 

Michael Prevoznik
Tuesday, July 25, 2017 - 14:52
Please see attached file.
qd_comments_onc_smf_final.docx
Sarah Willis-Garcia
Monday, July 24, 2017 - 16:52
On behalf of the more than 30 member companies of the Electronic Health Record Association (EHRA), we are pleased to offer our comments on the Office of the National Coordinator for Health Information Technology’s proposed Interoperability Standards Measurement Framework (Framework). We appreciate this opportunity to provide input to establishing a framework for measuring interoperability and the use of standards.

ehr_association_comments_to_onc_proposed_interoperability_standards_measurment_framework.pdf
Mary Beth Kurilo
Thursday, July 20, 2017 - 12:53
Thank you for the opportunity to provide feedback. Our comments and suggestions are attached.

aira_letter_and_comments_-_onc_measurement_framework_-_final_signed.pdf
Jeffery Smith
Thursday, July 20, 2017 - 12:47
John Travis
Wednesday, July 19, 2017 - 17:24
Thank you for the opportunity to provide input to the draft Interoperability Standards Measurement Framework. Please see our comments attached.

interoperability_standards_measurement_framework_rfi_-_cerner_response.pdf
Frank Diaz
Wednesday, July 19, 2017 - 11:44
None.
abdeljalil mekkaoui
Tuesday, July 18, 2017 - 14:15
PROGRAMME FILING
Dan Tran
Friday, July 14, 2017 - 13:16
1) Is a voluntary, industry-based measure reporting system the best means to implement this framework? What barriers might exist to a voluntary, industry-based measure reporting system, and what mechanisms or approaches could be considered to maximize this system’s value to stakeholders?

Minimizing the friction to leverage/integrate this framework will be key in ensuring high adoption rates, e.g. by having ONC certification test labs gather this information as a voluntary part of the certification process, or by having a web portal where you can send non-PHI data samples and standards will be automatically recorded to a central registry.

Additional incentives would also help to increase the likelihood of participation. For example, providing a digital badge that vendors can use that displays all of their supported ONC-verified interoperability standards.

Instead of implementing a voluntary system why not create an API that takes a C-CDA summary document which represents code system standards. EHR systems are currently expected to report quarterly to their certifying entity. As part of this process a summary C-CDA that contained only new codes, added since the last version of a coding system, could be mandated to be contained in the C-CDA summary sample. This would provider a clear indication of 1) the coding systems being used in the EHR and 2) the version of the coding systems being used.

---

2) What other alternative mechanisms to reporting on the measurement framework should be considered (for example, ONC partnering with industry on an annual survey)?

As mentioned above, having ONC proctors record this information during certification would reduce the burden on vendors. Having a web portal that vendors can use to update their supported standards (e.g. by uploading non-PHI data samples) would also be a potential option.

---

3) Does the proposed measurement framework include the correct set of objectives, goals, and measurement areas to inform progress on whether the technical requirements are in place to support interoperability?

The standards themselves have versions as well, and it’s unclear how those will be recorded via the metrics.

---

4) What, if any gaps, exist in the proposed measurement framework?

Although the level of conformance is a very important statistic, it may be hard to measure. If this is (number of conforming systems / total number of systems), what is the definition of a system? Customizations on top of this add another level of complexity.

---

5) Are the appropriate stakeholders identified who can support collection of needed data? If not, who should be added?

Not sure what the involvement of ONC certification test labs can play in recording this information, but that might be another organic forum for collecting standards information.

---

6) Would health IT developers, exchange networks, or other organizations who are data holders be able to monitor the implementation and use of measures outlined in the report? If not, what challenges might they face in developing and reporting on these measures?

This depends on the specific tools and resources available to assist with the process. Clear guidance and having resources to assist with metric collection and reporting will make it easier for data holders to implement this framework.

---

7) Ideally, the implementation and use of interoperability standards could be reported on an annual basis in order to inform the Interoperability Standards Advisory (ISA), which publishes a reference edition annually. Is reporting on the implementation and/or use of interoperability standards on an annual basis feasible? If not, what potential challenges exist to reporting annually? What would be a more viable frequency of measurement given these considerations?

Having a website or live portal similar to CHPL would allow the standards to be reported on a rolling basis. Updates on things like “percentage of certified products that have leveraged the standards measurement framework” could then be included in the ISA publication.

---

8) Given that it will likely not be possible to apply the measurement framework to all available standards, what processes should be put in place to determine the standards that should be monitored?

An initial broad survey can be sent out to create a baseline of currently implemented standards. The results of that survey can be used to generate the initial list of standards. Additional standards can then be added at the ONC’s discretion.

---

9) How should ONC work with data holders to collaborate on the measures and address such questions as: How will standards be selected for measurement? How will measures be specified so that there is a common definition used by all data holders for consistent reporting?

Direct communications and surveys (e.g. sending things out via mailing lists, emailing all products currently in CHPL, etc.) should help, as well as having open forums and meetings, etc.

---

10) What measures should be used to track the level of “conformance” with or customization of standards after implementation in the field?

As mentioned above, these metrics will be challenging to measure and record in a meaningful way. An easy way to track customizations would be with a simple “Do you have customizations on top of the standard” instead of trying to have data holders report a percentage.

Paul Wilson
Thursday, July 13, 2017 - 15:31
These responses are being submitted from The National Council for Prescription Drug Programs (NCPDP) as reviewed by The Education, Legislation and Regulations Task Group.

Responses:
1) Is a voluntary, industry-based measure reporting system the best means to implement this framework? What barriers might exist to a voluntary, industry-based measure reporting system, and what mechanisms or approaches could be considered to maximize this system’s value to stakeholders?
a. Although incentives may need to be offered to ensure a high-level of response, a voluntary reporting system appears to be the best option. Barriers include participation of the reporting entities and the standardization of what is being counted and measured.
b. NCPDP recommends ONC look at current models that exist for surveying the use of standards and interoperability. For example, the MN eHealth Initiative is one such entity that has been surveying eHealth adoption since 2004. Reviewing current models will allow for the data being collected to be aligned with that of existing reporting systems.
c. Proper communication of this effort, the benefits, and outcomes of collecting the data will be essential as many entities consider their data proprietary and confidential.
2) What other alternative mechanisms to reporting on the measurement framework should be considered (for example, ONC partnering with industry on an annual survey)?
a. Partnering with the industry on an Annual Survey is an option if the voluntary reporting survey from question 1 does not result in sufficient data or responses.
b. An alternative may be to mandate reporting through existing certification programs or appropriate regulation.
3) Does the proposed measurement framework include the correct set of objectives, goals, and measurement areas to inform progress on whether the technical requirements are in place to support interoperability?
a. NCPDP recommends ONC work with each Standards Development Organization (SDO) to determine objectives, goals and appropriate measurement areas.
b. It may not be appropriate to have product development plan details reported as a measurement, due to the proprietary nature of the vendor specific information.
4) What, if any gaps, exist in the proposed measurement framework?
a. A gap may exist if ONC is unable to successfully determine the denominator for all measures, e.g. all system vendors, all prescribers, etc.
b. Additionally, there appears to be a gap due to confusion about how the data will be used. Once the stakeholders clearly understand how the data will be used, the voluntary reporting will likely increase.
5) Are the appropriate stakeholders identified who can support collection of needed data? If not, who should be added?
a. Your potential list of stakeholders is a good start. ONC will need to identify and remove any redundancies in the data to ensure accurate measurement and reporting.
b. ONC may want to contact states (e.g. MN) or other organizations (e.g. eHealth Initiative) that are already collecting similar data for guidance.
6) Would health IT developers, exchange networks, or other organizations who are data holders be able to monitor the implementation and use of measures outlined in the report? If not, what challenges might they face in developing and reporting on these measures?
a. Developers may not have insight into when their customers are going to implement a specific version that includes the functionality ONC wants to track, or at least not enough detailed insight to track the implementation.
b. Measures should be reported based on which transactions and how many transactions are exchanged without involving the end user. Depending on the method of reporting transactions, reaching the end user may be very challenging.
7) Ideally, the implementation and use of interoperability standards could be reported on an annual basis in order to inform the Interoperability Standards Advisory (ISA), which publishes a reference edition annually. Is reporting on the implementation and/or use of interoperability standards on an annual basis feasible? If not, what potential challenges exist to reporting annually? What would be a more viable frequency of measurement given these considerations?
a. The volume of some transactions for a specific standard is currently tracked annually (e.g. MN and Surescripts National Progress Report). Reporting on an annual basis would depend on the complexity of the reporting requirements. Timing of the reporting would need to be considered as the timeline for the submission of these reports would be dependent on the requirements. Stakeholders may need to alter internal systems to ensure transactions are properly counted and reported, which could cause reporting delays. It is impossible to determine feasibility of any proposed deadlines until the requirements are known.
b. Links to some current industry reports are noted below:
i. Council for Affordable Quality Healthcare (CAQH): www.caqh.org/explorations/caqh-index/
ii. Cover My Meds: epascorecard.covermymeds.com/
iii. eHealth Initiative: www.ehidc.org/
iv. State of Minnesota: www.health.state.mn.us/e-health/assessment/index.html
v. Surescripts: http://surescripts.com/news-center/national-progress-report-2016/
8) Given that it will likely not be possible to apply the measurement framework to all available standards, what processes should be put in place to determine the standards that should be monitored?
a. NCPDP recommends ONC start with standards named in any Federal regulation. Additional standards may be considered in future phases, based on their status within the ONC Interoperability Standards Advisory.
b. ONC may want to consider prioritizing the SCRIPT standard transactions, which have a more direct impact to patient care and outcomes, (e.g. NEWRx, ePA) over those that address administrative and billing transactions or less critical exchanges of data.
9) How should ONC work with data holders to collaborate on the measures and address such questions as: How will standards be selected for measurement? How will measures be specified so that there is a common definition used by all data holders for consistent reporting?
a. NCPDP recommends ONC work with the SDOs to assist in identifying the standards to be reported and the stakeholders involved in the implementation of those standards.
b. ONC may also want to consider working with the identified stakeholders on specific measures as they apply to each standard.
c. The SDOs should assist in establishing the common measurement definitions for consistent reporting.
10) What measures should be used to track the level of “conformance” with or customization of standards after implementation in the field?
a. NCPDP recommends ONC work with the SDOs, who likely have mechanisms for addressing non-conformance with their standard(s).

valita fredland
Thursday, July 13, 2017 - 08:40
....

Pages