Printer Friendly, PDF & Email Printer Friendly, PDF & Email

§170.315(c)(2) Clinical quality measures (CQMs) — import and calculate

Version 1.3 Updated on 06-29-2018
Revision History
Version # Description of Change Version Date
1.0

Final Test Procedure

07-29-2016
1.1

Removed note to test one of the two alternatives and renumbered steps for paragraph (c)(2)(ii).

01-10-2017
1.2

Added numbers to the Execute Any Time section for paragraph (c)(2)(i) and continued the numbering for the next sections (Setup and Import).

Updated the numbering to restart at 1 for the SUT section of Calculate under paragraph (c)(2)(ii).

Added a continued number to the section of Alternative: Cypress Certification API under the TLV of paragraph (c)(2)(ii).

In paragraph (c)(2)(ii), updated the Alternative section to align with the correct steps a tester would take if using the Cypress Certification API.

04-06-2018
1.3

Updated ‘step #’ to reference the actual step description and added clarifying language in the Alternative API section.

06-29-2018
Regulation Text
Regulation Text

§170.315 (c)(2) Clinical quality measures—import and calculate

  1. Import. Enable a user to import a data file in accordance with the standard specified in §170.205(h)(2) for one or multiple patients and use such data to perform the capability specified in paragraph (c)(2)(ii) of this section. A user must be able to execute this capability at any time the user chooses and without subsequent developer assistance to operate.
  2. Calculate each and every clinical quality measure for which it is presented for certification.
Standard(s) Referenced

Please consult the Final Rule entitled: 2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health Record (EHR) Definition, and ONC Health IT Certification Program Modifications for a detailed description of the certification criterion with which these testing steps are associated. We also encourage developers to consult the Certification Companion Guide in tandem with the test procedure as they provide clarifications that may be useful for product development and testing.

Note: The order in which the test steps are listed reflects the sequence of the certification criterion and does not necessarily prescribe the order in which the test should take place.
 

Testing components

No GAP Icon Documentation Icon Visual Inspection Icon Test Tool Icon ONC Supplied Test Data Icon

 

Paragraph (c)(2)(i)

System Under Test Test Lab Verification

Execute Any Time

  1. The health IT developer supplies documentation outlining how a user can execute the import capability described in (c)(2)(i) any time the user chooses and without subsequent developer assistance to operate.

Setup

  1. The Health IT Module provides the following information in order to enable the creation of the (c)(2)(i) “Cypress Gold Standard Test Data” which includes instructions to enable the recording of CQM data within patient record(s):
    1. Name of the health IT developer;
    2. Name of the Product;
    3. List of CQMs to be certified; and
    4. List of certification criteria to be tested

Import

  1. Using the "Cypress Gold Standard Test Data" a user demonstrates the importing of reports formatted in accordance with the standard specified at § 170.205(h)(2) HL7 CDA® R2 Implementation Guide for: Quality Reporting Document Architecture – Category I (QRDA I); Release 1, DSTU Release 3, Volume 1 for all of the data needed to calculate each of the clinical quality measures (CQMs) presented for testing, for one or multiple patients.

Execute Any Time

  1. The tester verifies that the health IT developer supplied documentation outlines that a user can perform an import as specified in (c)(2)(i) any time the user chooses and without subsequent developer assistance to operate.

Setup

  1. The tester creates the "Cypress Gold Standard Test Data" based upon the information provided by the health IT developer, in Cypress, which will create a new (c)(2) test instance.

Import

  1. Using visual inspection, the tester verifies that the Health IT Module can demonstrate the importing of CQM data specified in accordance to the standard at § 170.205(h)(2).

Paragraph (c)(2)(ii)

System Under Test Test Lab Verification

Calculate

  1. The user calculates the aggregate reports for each of the CQMs for which they are seeking certification, based upon the imported and de-duplicated data set.
  2. The Health IT Module submits an aggregate report for each of the CQMs to be certified.

Calculate

  1. The tester verifies that the Health IT Module can use the imported CQM data to calculate the aggregate data and display the report in accordance at a minimum to the standard at § 170.205(k)(1) Quality Reporting Document Architecture Category III, Implementation Guide for CDA Release 2 and § 170.205(k)(2) Errata to the HL7 Implementation Guide for CDA® Release 2: Quality Reporting Document Architecture—Category III, DSTU Release 1 using visual inspection.
  2. Using Cypress, the tester:
    1. uploads the aggregate report(s) submitted by the Health IT Module; and
    2. displays and evaluates the accuracy of the submitted CQM results.

Packaging of Results

  1. The tester generates a test artifact containing the following into a single archived file and any additional notes that the tester deems important:
    1. all of the test data used to test (c)(2); and
    2. all of the data generated by the Health IT Module.

Cypress Certification API (Alternative)

  1. A tester may use the Cypress Certification API to upload the data file submitted by the Health IT Module. The tester can verify the results in Cypress as normal, however, the tester should manually verify that the Health IT Module can use the imported CQM data to calculate the aggregate data and display the report appropriately as well as the accuracy of the submitted CQM results for at least one CQM to ensure this functionality is present. The tester should still generate a test artifact containing the test data and all data generated by the Health IT Module into a single archive file, as usual. Testers are permitted to randomly select, at their discretion, which individual patient from the batch entry recording will be used to export a QRDA Category I file in order to demonstrate the functionality is present. The tester instructs the health IT developer to export the selected patient and confirms that this patient was exported.

Alternative Test Method

Summary Alternative Test Method File Test Tool Updated On

NCQA was approved as an ONC-Approved Alternate Test Method June 19, 2017 for testing § 170.315(c)(2), § 170.315(c)(3), and § 170.315(c)(4).

NCQA tests and validates the integrity of an HIT organization’s software code that calculates eCQM results and produces Quality Reporting Document Architecture (QRDA) reports.

Our eMeasure Certification program uses the same ONC-approved methodology, but the certification is used to reduce the audit on supplemental HEDIS data and is not part of the ONC program.

Version 1.3 Updated on 08-04-2017
Revision History
Version # Description of Change Version Date
1.0

Initial Publication

10-29-2015
1.1

Included clarifications regarding testing and certification to versions of standards associated with the CMS annual measure updates, de-duplication, and testing for calculation.

01-10-2017
1.2

Revised to include clarification about testing, certification, and surveillance expectations for paragraph (c)(2)(i).

03-17-2017
1.3

Revised to include an ONC approved alternative test procedure, tool, and data offered by the National Committee for Quality Assurance (NCQA).

08-04-2017
Regulation Text
Regulation Text

§170.315 (c)(2) Clinical quality measures—import and calculate

  1. Import. Enable a user to import a data file in accordance with the standard specified in §170.205(h)(2) for one or multiple patients and use such data to perform the capability specified in paragraph (c)(2)(ii) of this section. A user must be able to execute this capability at any time the user chooses and without subsequent developer assistance to operate.
  2. Calculate each and every clinical quality measure for which it is presented for certification.
Standard(s) Referenced

Certification Companion Guide: Clinical quality measures (CQMs) — import and calculate

This Certification Companion Guide (CCG) is an informative document designed to assist with health IT product development. The CCG is not a substitute for the 2015 Edition final regulation. It extracts key portions of the rule’s preamble and includes subsequent clarifying interpretations. To access the full context of regulatory intent please consult the 2015 Edition final rule or other included regulatory reference. The CCG is for public use and should not be sold or redistributed.
 

 

Certification Requirements

Privacy and Security: This certification criterion was adopted at § 170.315(c)(2). As a result, an ONC-ACB must ensure that a product presented for certification to a § 170.315(c) “paragraph (c)” criterion includes the privacy and security criteria (adopted in § 170.315(d)) within the overall scope of the certificate issued to the product.

  • The privacy and security criteria (adopted in § 170.315(d)) do not need to be explicitly tested with this specific paragraph (c) criterion unless it is the only criterion for which certification is requested.
  • As a general rule, a product presented for certification only needs to be tested once to each applicable privacy and security criterion (adopted in § 170.315(d)) so long as the health IT developer attests that such privacy and security capabilities apply to the full scope of capabilities included in the requested certification. However, exceptions exist for § 170.315(e)(1) “VDT” and (e)(2) “secure messaging,” which are explicitly stated.

Design and Performance: The following design and performance certification criteria (adopted in § 170.315(g)) must also be certified in order for the product to be certified.

  • When a single quality management system (QMS) is used, the QMS only needs to be identified once. Otherwise, the QMS’ need to be identified for every capability to which it was applied.
  • When a single accessibility-centered design standard is used, the standard only needs to be identified once. Otherwise, the accessibility-centered design standards need to be identified for every capability to which they were applied; or, alternatively the developer must state that no accessibility-centered design was used.
Table for Privacy and Security
Technical Explanations and Clarifications

 

Applies to entire criterion

Clarifications:

  • The specific version, number, and type of clinical quality measures (CQMs) presented for certification are determined at the developer’s discretion. We recommend developers consult any CMS or other programs’ requirements around the specific version, number, or type of CQMs required for providers in determining the CQMs presented for certification.
  • Certain CMS programs require or provide the option for electronic CQM (eCQM) reporting. These programs include the EHR Incentive Program, the Physician Quality Reporting System, the Hospital Inpatient Quality Reporting Program, the Comprehensive Primary Care (CPC) initiative, CPC Plus, and the Value-Based Payment Modifier Program. Each year, CMS issues annual updates to eCQMs (herein referred to as the “CMS annual measure update(s)”) which are published on the Electronic Clinical Quality Improvement (eCQI) Resource Center. The CMS annual measure updates rely upon a specific version of the Quality Reporting Document Architecture (QRDA) Category I standard. Each year’s QRDA Category I standard is referenced in the corresponding CMS QRDA Implementation Guide (IG) associated with that program year and CMS annual measure update. The CMS QRDA IG also contains additional programmatic form and manner requirements necessary for reporting to CMS programs, which make it necessary for the corresponding testing tool to keep pace with these measure updates and CMS reporting requirements. Thus, health IT developers are permitted to be tested and certified to the applicable CMS annual measure update and use the corresponding version of QRDA Category I standard referenced in the CMS QRDA IG. Note that for this criterion at § 170.315(c)(2), the testing tools are only capable of validating the correct calculation of CQMs for reports submitted in the corresponding QRDA Category III format. ONC will evaluate the need for future rulemaking to align the version of QRDA Category I standard required for this certification criterion with the version of QRDA Category I standard in the CMS annual measure update.
  • After technology is certified to specific CQMs for this 2015 Edition certification criterion at § 170.315(c)(2), technology is not required to recertify to the annual measure specification updates CMS issues to maintain 2015 Edition certification unless that product is relabeled. Said another way, other programs, such as the EHR Incentive Programs, may require developers upgrade their technology to the newest CQM specifications, but the technology is not required to be retested or recertified, unless explicitly specified in other program requirements. [ONC FAQ #42] It is expected that all systems will test all measure and standards updates as a best practice. The testing tools are available for each CMS annual measure update and when there are late standards errata or CMS requirement changes to facilitate additional testing.
  • Systems that present for certification to § 170.315(c)(1), (c)(2), (c)(3) must explicitly demonstrate this criterion at § 170.315(c)(2). Those systems previously considered “self-contained” according to the 2014 Edition policy will no longer be deemed to meet certification for § 170.315(c)(2). [see also 80 FR 62650]
  • For the purposes of automated testing to meet certification requirements, only errors (but not warnings) generated during testing would constitute a failure to meet certification requirements.

Paragraph (c)(2)(i)

Technical outcome – A user can import a data file formatted in accordance with HL7 QRDA Category I Release 3 or the corresponding version of the QRDA standard for the CMS annual measure update being certified for one or multiple patients in order to perform calculations on the CQMs presented for certification.

Clarifications:

  • A user of this health IT must be able to import a data file at any time selected without additional assistance from health IT developers. We are not prescribing how data is imported into a system (e.g., mapped to backend database or viewable to a provider as part of the patient record) and leave the determination at the discretion of the developer. [80 FR 62650]
    • Testing and Certification. Successful testing and certification does not require the evaluation of the time required to process a CQM data file. To illustrate, a delay between when a user initiates an export and receives the resulting data file would not, by itself, preclude successful testing of the technology or the issuance of a certification on the basis of those successful test results.
    • Surveillance. While the CQM export capability does not require that data be received instantaneously, a non-conformity would exist if surveillance revealed that processing or other delays were likely to substantially interfere with the ability of a provider or health system to view and verify their CQM results for quality improvement on a near real-time basis. [80 FR 62650] Similarly, a non-conformity would exist if delays were causing or contributing to users being presented with data files that no longer contained current, accurate, or valid data. To avoid these implementation issues and ensure that capabilities support all required outcomes, health IT developers should seek to minimize processing times and other delays to the greatest extent possible. We note also that any delays must be disclosed in accordance with 170.523(k)(1).
  • This provision will streamline testing and certification by importing QRDA Category I files avoiding the need for systems to manually enter test patient data. It will also promote quality improvement and data sharing between systems by providing the systems the ability to import CQM data from other systems in a standardized format. [see also 80 FR 62650]
  • Providers and health systems should determine the protocols around when and how providers import CQM data. For testing, the health IT would need to demonstrate a user can import data formatted to QRDA Category I for one or more patients without needing additional developer support. [see also 80 FR 62651]
  • We would expect that health IT must be able to de-duplicate patient records, but do not prescribe how systems would demonstrate de-duplication. Developers have the discretion to determine the most suitable method for deduplication. [see also 80 FR 62651] De-duplication testing will require systems to identify two files with the same or similar patient identifiers but different QRDA document identifiers and adjudicate information which is identical and combine information which is not.
  • Testing will more robustly test the pathways by which a patient is included in the numerator and/or denominator, including exclusions and exceptions, of a measure. [see also 80 FR 62651]

Paragraph (c)(2)(ii)

Technical outcome – The health IT must be able to calculate each CQM presented for certification.

Clarifications:

  • The testing tools are only capable of validating the correct calculation of CQMs for reports submitted in the QRDA Category III format.

Regulation Text
Regulation Text

§170.315 (c)(2) Clinical quality measures—import and calculate

  1. Import. Enable a user to import a data file in accordance with the standard specified in §170.205(h)(2) for one or multiple patients and use such data to perform the capability specified in paragraph (c)(2)(ii) of this section. A user must be able to execute this capability at any time the user chooses and without subsequent developer assistance to operate.
  2. Calculate each and every clinical quality measure for which it is presented for certification.
Testing Tool
Content last reviewed on July 13, 2018
Was this page helpful?