Printer Friendly, PDF & Email Printer Friendly, PDF & Email

§170.315(c)(3) Clinical quality measures (CQMs) — report

Version 1.2 Updated on 06-29-2018
Resource Documents
Revision History
Version # Description of Change Version Date
1.0

Final Test Procedure

07-29-2016
1.1

Removed ‘report with raw data’ from SUT step 2 in paragraph (c)(3)(i).

01-10-2017
1.2

Updated ‘step #’ to reference the actual step description.

06-29-2018
Regulation Text
Regulation Text

§170.315 (c)(3) Clinical quality measures—report

Enable a user to electronically create a data file for transmission of clinical quality measurement data:

  1. At a minimum, in accordance with the standards specified in §170.205(h)(2) and §170.205(k)(1) and (2).
  2. Optional. That can be electronically accepted by CMS.
Standard(s) Referenced
Testing
Testing Tool
Criterion Subparagraph Test Data
(c)(3)(i)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported QRDA Category I files created during (c)(1)(ii) and generated QRDA Category I files created as a result of (c)(2)(i) Import of CQMs.

(c)(3)(ii)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported QRDA Category I files created during (c)(1)(ii) and generated QRDA Category I files created as a result of (c)(2)(i) Import of CQMs.

Please consult the Final Rule entitled: 2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health Record (EHR) Definition, and ONC Health IT Certification Program Modifications for a detailed description of the certification criterion with which these testing steps are associated. We also encourage developers to consult the Certification Companion Guide in tandem with the test procedure as they provide clarifications that may be useful for product development and testing.

Note: The order in which the test steps are listed reflects the sequence of the certification criterion and does not necessarily prescribe the order in which the test should take place.

Testing components

No GAP Icon Documentation Icon Visual Inspection Icon Test Tool Icon ONC Supplied Test Data Icon

 

Paragraph (c)(3)(i)

System Under Test Test Lab Verification

QRDA Category III Report

  1. The user can generate an aggregate report (QRDA Category III) with calculated summary data for the patient population of the clinical quality measures calculated in the Execute test (§ 170.315(c)(2)), which at a minimum is in accordance with the standard specified in § 170.205(k)(1) Quality Reporting Document Architecture Category III, Implementation Guide for CDA Release 2 and § 170.205(k)(2) Errata to the HL7 Implementation Guide for CDA® Release 2: Quality Reporting Document Architecture – Category III, DSTU Release 1.

QRDA Category I Report

  1. A user can generate a de-duplicated archive of patient documents in the QRDA Category I format of the clinical quality measures calculated in the Execute test (§ 170.315(c)(2)), which at a minimum is in accordance with the standard specified in § 170.205(h)(2) HL7 CDA® R2 Implementation Guide: Quality Reporting Document Architecture – Category I (QRDA I); Release 1, DSTU Release 3, Volume 1.

Data File for Transmission

  1. The health IT developer submits the quality measurement data file consisting of the data created by the generation of the QRDA Category III aggregate report(s) and the de-duplicated QRDA Category I report(s) for verification.

Test Lab Setup

  1. Prior to beginning this test, the tester creates and exports data using Cypress, and the health IT developer imports the data into their Health IT Module.

QRDA Category III Report

  1. Using the Cypress supplied XML Schema validation, the tester:
    1. uploads the aggregate report(s) submitted by the health IT developer; and
    2. runs the Cypress supplied XML schema validation for each aggregate report.
  2. The tester verifies that all of the QRDA Category III aggregate report(s), submitted by the health IT developer are at a minimum in accordance with the standard specified at § 170.205(k)(1) and (2) through evaluation of the Cypress validation report.

QRDA Category I Report

  1. The tester verifies that all of the de-duplicated QRDA Category I report(s) submitted by the health IT developer are at a minimum in accordance with the standard specified at § 170.205(h)(2) through evaluation of the Cypress validation report.

Data File for Transmission

  1. The tester verifies via visual inspection that the data file for transmission submitted with clinical quality measurement data includes both de-duplicated QRDA Category I and aggregate QRDA Category III report(s).

Paragraph (c)(3)(ii) Optional

System Under Test Test Lab Verification

The QRDA reports created in paragraph (c)(3)(i) are validated using Cypress to validate they can be electronically accepted by CMS.

The tester verifies that the QRDA reports can be electronically submitted to CMS based on Cypress validation.


Alternative Test Method

Summary Alternative Test Method File Test Tool Updated On

NCQA was approved as an ONC-Approved Alternate Test Method June 19, 2017 for testing § 170.315(c)(2), § 170.315(c)(3), and § 170.315(c)(4).

NCQA tests and validates the integrity of an HIT organization’s software code that calculates eCQM results and produces Quality Reporting Document Architecture (QRDA) reports.

Our eMeasure Certification program uses the same ONC-approved methodology, but the certification is used to reduce the audit on supplemental HEDIS data and is not part of the ONC program.

Version 1.3 Updated on 06-15-2020
Resource Documents
Revision History
Version # Description of Change Version Date
1.0

Initial Publication

10-26-2015
1.1

Revised to include clarification about testing and certification to versions of standards associated with the CMS annual measure updates.

Updated incorrect links.

01-10-2017
1.2

Revised to include an ONC approved alternative test procedure, tool, and data offered by the National Committee for Quality Assurance (NCQA).

08-04-2017
1.3

Updated the Security requirements per 21st Century Cures Act. 

06-15-2020
Regulation Text
Regulation Text

§170.315 (c)(3) Clinical quality measures—report

Enable a user to electronically create a data file for transmission of clinical quality measurement data:

  1. At a minimum, in accordance with the standards specified in §170.205(h)(2) and §170.205(k)(1) and (2).
  2. Optional. That can be electronically accepted by CMS.
Standard(s) Referenced
Testing
Testing Tool
Criterion Subparagraph Test Data
(c)(3)(i)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported QRDA Category I files created during (c)(1)(ii) and generated QRDA Category I files created as a result of (c)(2)(i) Import of CQMs.

(c)(3)(ii)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported QRDA Category I files created during (c)(1)(ii) and generated QRDA Category I files created as a result of (c)(2)(i) Import of CQMs.

Certification Companion Guide: Clinical quality measures (CQMs) — report

This Certification Companion Guide (CCG) is an informative document designed to assist with health IT product development. The CCG is not a substitute for the 2015 Edition final regulation. It extracts key portions of the rule’s preamble and includes subsequent clarifying interpretations. To access the full context of regulatory intent please consult the 2015 Edition final rule or other included regulatory reference. The CCG is for public use and should not be sold or redistributed.
 

 

Certification Requirements

Privacy and Security: This certification criterion was adopted at § 170.315(c)(3). As a result, an ONC-ACB must ensure that a product presented for certification to a § 170.315(c) “paragraph (c)” criterion includes the privacy and security criteria (adopted in § 170.315(d)) within the overall scope of the certificate issued to the product.

  • The privacy and security criteria (adopted in § 170.315(d)) do not need to be explicitly tested with this specific paragraph (c) criterion unless it is the only criterion for which certification is requested.
  • As a general rule, a product presented for certification only needs to be tested once to each applicable privacy and security criterion (adopted in § 170.315(d)) so long as the health IT developer attests that such privacy and security capabilities apply to the full scope of capabilities included in the requested certification. However, exceptions exist for § 170.315(e)(1) “VDT” and (e)(2) “secure messaging,” which are explicitly stated.
  • § 170.315(d)(2)(i)(C) is not required if the scope of the Health IT Module does not have end-user device encryption features.

Design and Performance: The following design and performance certification criteria (adopted in § 170.315(g)) must also be certified in order for the product to be certified.

  • When a single quality management system (QMS) is used, the QMS only needs to be identified once. Otherwise, the QMS’ need to be identified for every capability to which it was applied.
  • When a single accessibility-centered design standard is used, the standard only needs to be identified once. Otherwise, the accessibility-centered design standards need to be identified for every capability to which they were applied; or, alternatively the developer must state that no accessibility-centered design was used.
Table for Privacy and Security
Technical Explanations and Clarifications

 

Applies to entire criterion

Clarifications:

  • The specific version, number, and type of clinical quality measures (CQMs) presented for certification are determined at the developer’s discretion. We recommend developers consult any CMS or other programs’ requirements around the specific version, number, or type of CQMs required for providers in determining the CQMs presented for certification.
  • Certain CMS programs require or provide the option for electronic CQM (eCQM) reporting. These programs include the EHR Incentive Program, the Physician Quality Reporting System, the Hospital Inpatient Quality Reporting Program, the Comprehensive Primary Care (CPC) initiative, CPC Plus, and the Value-Based Payment Modifier Program. Each year, CMS issues annual updates to eCQMs (herein referred to as the “CMS annual measure update(s)”) which are published on the Electronic Clinical Quality Improvement (eCQI) Resource Center. The CMS annual measure updates rely upon specific versions of the Quality Reporting Document Architecture (QRDA) Category I and Category III standards. Each year’s QRDA standards are referenced in the corresponding CMS QRDA Implementation Guide (IG) associated with that program year and CMS annual measure update. The CMS QRDA IG also contains additional programmatic form and manner requirements necessary for reporting to CMS programs, which make it necessary for the corresponding testing tools to keep pace with these measure updates and CMS reporting requirements. Thus, health IT developers are permitted to be tested and certified to the applicable CMS annual measure update and use the corresponding versions of QRDA Category I and Category III standards as referenced in the CMS QRDA IG. ONC will evaluate the need for future rulemaking to align the versions of QRDA standards required for this certification criterion with the versions of QRDA standards in the CMS annual measure update.
  • After technology is certified to specific CQMs for this 2015 Edition certification criterion at § 170.315(c)(3), technology is not required to recertify to the annual measure specification updates CMS issues to maintain 2015 Edition certification unless that product is relabeled. Said another way, other programs, such as the EHR Incentive Program, may require developers upgrade their technology to the newest CQM specifications, but the technology is not required to be retested or recertified unless explicitly specified in other program requirements. [see also ONC FAQ #42] It is expected that all systems will test all measure and standards updates as a best practice. The testing tools are available for each CMS annual measure update and when there are late standards errata or CMS requirement changes to facilitate additional testing.
  • For the purposes of automated testing to meet certification requirements, only errors (but not warnings) generated during testing would constitute a failure to meet certification requirements.

Paragraph (c)(3)(i)

Technical outcome – A user can create a data file for transmission of CQM data in QRDA Category I (for individual level reports) and Category III (for aggregate reports) as specified in the HL7 CDA® Release 2 Implementation Guide for: Quality Reporting Document Architecture – Category I (QRDA I), DSTU Release 3 and Implementation Guide for CDA® Release 2: Quality Reporting Document Architecture—Category III, DSTU Release 1 (US Realm) with September 2014 Errata, respectively or the corresponding version of the QRDA standard for the CMS annual measure update being certified.

Clarifications:

  • No additional clarifications available.

Paragraph (c)(3)(ii) Optional

Technical outcome – As an optional provision, a user can create a data file for transmission of CQM data that can be electronically accepted by CMS.

Clarifications:

  • The requirements for CMS reporting will be published by CMS in documents such as the CMS QRDA Implementation Guide. CMS will indicate in its regulations and program guidance whether this optional provision is required for participation in its programs. (Refer also to clarification above which applies to the entire criterion.)
  • This provision is optional because providers may use the measures and the technology certified to this criterion for calculating and reporting measures to entities other than CMS programs. [see also 80 FR 62652]
  • The testing tool(s) for 2015 Edition CQM criteria support testing to the CMS QRDA Implementation Guide requirements for the measures intended for certification. Conformance failures against the CMS QRDA Implementation Guide are displayed as warnings. The following links are references to CMS CQM reporting resources:

Version 1.0 Updated on 06-30-2020
Resource Documents
Revision History
Version # Description of Change Version Date
1.0

Initial Publication

06-15-2020
1.1

Corrected Standards References, removed Real World Testing bullet, added clarification for 2020 CMS QRDA Category I and Category III

06-30-2020
Regulation Text
Regulation Text

§170.315(c)(3) Clinical quality measures—report

Enable a user to electronically create a data file for transmission of clinical quality measurement data in accordance with the applicable implementation specifications specified by the CMS implementation guides for Quality Reporting Document Architecture (QRDA), category I, for inpatient measures in § 170.205(h)(3) and CMS implementation guide for QRDA, category III for ambulatory measures in § 170.205(k)(3).

Standard(s) Referenced
Testing
Testing Tool
Criterion Subparagraph Test Data
(c)(3)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported CMS QRDA Category I IG files created during section (c)(1)(ii) and generated CMS QRDA Category I IG files created as a result of section (c)(2)(i) Import of Clinical Quality Measures (CQMs).

Certification Companion Guide: Clinical quality measures (CQMs) — report (Cures)

This Certification Companion Guide (CCG) is an informative document designed to assist with health IT product development. The CCG is not a substitute for the 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program Final Rule (ONC Cures Act Final Rule). It extracts key portions of the rule’s preamble and includes subsequent clarifying interpretations. To access the full context of regulatory intent please consult the ONC Cures Act Final Rule or other included regulatory reference. The CCG is for public use and should not be sold or redistributed.

 

Certification Requirements

Privacy and Security: This certification criterion was adopted at § 170.315(c)(3). As a result, an ONC-ACB must ensure that a product presented for certification to a § 170.315(c) “paragraph (c)” criterion includes the privacy and security criteria (adopted in § 170.315(d)) within the overall scope of the certificate issued to the product.

  • The privacy and security criteria (adopted in § 170.315(d)) do not need to be explicitly tested with this specific paragraph (c) criterion unless it is the only criterion for which certification is requested.
  • As a general rule, a product presented for certification only needs to be tested once to each applicable privacy and security criterion (adopted in § 170.315(d)) so long as the health IT developer attests that such privacy and security capabilities apply to the full scope of capabilities included in the requested certification. However, an exceptions exist for § 170.315(e)(1) “VDT” and (e)(2) “secure messaging,” which are explicitly stated.
  • § 170.315(d)(2)(i)(C) is not required if the scope of the Health IT Module does not have end-user device encryption features.

Design and Performance: The following design and performance certification criteria (adopted in § 170.315(g)(4) and § 170.315(g)(5)) must also be certified in order for the product to be certified.

  • When a single quality management system (QMS) is used, the QMS only needs to be identified once. Otherwise, when different QMS are used, each QMS needs to be separately identified for every capability to which it was applied.
  • When a single accessibility-centered design standard is used, the standard only needs to be identified once. Otherwise, the accessibility-centered design standards need to be identified for every capability to which they were applied; or, alternatively the developer must state that no accessibility-centered design was used.
Table for Privacy and Security
Technical Explanations and Clarifications

 

Applies to entire criterion

Technical outcome – Enable a user to electronically create a data file for transmission of CQM data in accordance with the CMS QRDA Category I IG for inpatient measures as adopted in § 170.205(h)(3) and CMS QRDA Category III IG for ambulatory measures as adopted in § 170.205(k)(3).

 

Clarifications:

  • The specific version, number, and type of clinical quality measures (CQMs) presented for certification are determined at the developer’s discretion. We recommend developers consult any CMS or other programs’ requirements around the specific version, number, or type of CQMs required for providers in determining the CQMs presented for certification.
  • Certain CMS programs require or provide the option for electronic CQM (eCQM) reporting. These programs include the EHR Incentive Program, the Physician Quality Reporting System, the Hospital Inpatient Quality Reporting Program, the Comprehensive Primary Care (CPC) initiative, CPC Plus, and the Value-Based Payment Modifier Program. Each year, CMS issues annual updates to eCQMs (herein referred to as the “CMS annual measure update(s)”) which are published on the Electronic Clinical Quality Improvement (eCQI) Resource Center. The CMS annual measure updates rely upon specific versions of the HL7 Quality Reporting Document Architecture (QRDA) Category I and Category III standards. Each year’s HL7 QRDA Category I and Category III standards are referenced in the corresponding CMS QRDA Implementation Guides (IGs) associated with that program year and CMS annual measure update. The CMS QRDA Category I and Category III IGs also contain additional programmatic form and manner requirements necessary for reporting to CMS programs, which make it necessary for the corresponding testing tools to keep pace with these measure updates and CMS reporting requirements.
  • Compliance with this certification criterion is dependent on the update to the CMS QRDA Category I and Category III IGs. It is our expectation that health IT developers will update Health IT Modules to align with the 2020 CMS QRDA Category I and Category III IGs in a timely fashion that allows users to report eCQMs to CMS in early 2021. Health IT developers should continue to update Health IT Modules to align with the CMS annual update to the CMS QRDA Category I and Category III IGs for continued compliance with this certification criterion.
  • For details on the latest CMS QRDA Category I and Category III IGs, we refer readers to the eCQI Resource Center website.
  • After technology is certified to specific CQMs for this 2015 Edition certification criterion at § 170.315(c)(3), technology is not required to recertify to the annual measure specification updates CMS issues to maintain 2015 Edition certification unless that product is relabeled. Said another way, other programs, such as the EHR Incentive Program, may require developers upgrade their technology to the newest CQM specifications, but the technology is not required to be retested or recertified unless explicitly specified in other program requirements. [see also the Health IT Certification Program Overview] It is expected that all systems will test all measure and standards updates as a best practice. The testing tools are available for each CMS annual measure update and when there are late standards errata or CMS requirement changes to facilitate additional testing.
  • For the purposes of automated testing to meet certification requirements, only errors (but not warnings) generated during testing would constitute a failure to meet certification requirements.
  • In order to prevent unintended burden by tailoring the requirements to the type of measures being tested ONC has provided the following clarifications:  For the updated certification criterion “CQMs—report” in § 170.315(c)(3) a Health IT Module testing only inpatient measures would test only with the CMS QRDA Category I IG and a Health IT Module testing only ambulatory measures would test only with the CMS QRDA Category III IG. A Health IT Module supporting both inpatient and ambulatory measures would be required to test to both the CMS QRDA Category I and Category III IGs.
  • The following links are references to CMS CQM reporting resources:

Version 1.1 Updated on 06-15-2020
Resource Documents
Revision History
Version # Description of Change Version Date
1.0

Final Test Procedure

06-01-2020
1.1

Test Procedure corrections for consistency across (c)(3) Certification Companion Guide

06-15-2020
Regulation Text
Regulation Text

§170.315(c)(3) Clinical quality measures—report

Enable a user to electronically create a data file for transmission of clinical quality measurement data in accordance with the applicable implementation specifications specified by the CMS implementation guides for Quality Reporting Document Architecture (QRDA), category I, for inpatient measures in § 170.205(h)(3) and CMS implementation guide for QRDA, category III for ambulatory measures in § 170.205(k)(3).

Standard(s) Referenced
Testing
Testing Tool
Criterion Subparagraph Test Data
(c)(3)

Cypress Gold Standard Test Data created using Cypress - generated to match the submitted exported CMS QRDA Category I IG files created during section (c)(1)(ii) and generated CMS QRDA Category I IG files created as a result of section (c)(2)(i) Import of Clinical Quality Measures (CQMs).

Please consult the Final Rule entitled: 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program for a detailed description of the certification criterion with which these testing steps are associated. Developers are encouraged to consult the Certification Companion Guide in tandem with the Test Procedure, as they both provide clarifications that may be useful for product development and testing.

Note: The order in which the test steps are listed reflects the sequence of the certification criterion and does not necessarily prescribe the order in which the test should take place.

Testing components

No GAP Icon Documentation Icon Visual Inspection Icon Test Tool Icon ONC Supplied Test Data Icon

Paragraph (c)(3) – (Conditional – For Modules Certified to 2015 Edition (c)(3))

System Under Test
ONC-ACB Verification

A Health IT Module currently certified to the 2015 Edition 170.315(c)(3) Clinical quality measures - report will attest directly to the ONC-ACB to conformance with the updated 170.315(c)(3) requirements outlined in the 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program Final Rule.

The ONC-ACB verifies the Health IT Module certified to 2015 Edition 170.315(c)(3) Clinical quality measures - report attests conformance to 2015 Edition Cures Update 170.315(c)(3) criteria requirements.


Paragraph (c)(3)

System Under Test Test Lab Verification

CMS Quality Reporting Document Architecture (QRDA) Category III Implementation Guide (IG) Report

  1. The user can generate an aggregate report  with calculated summary data for the patient population of the clinical quality measures calculated in the Execute test (§ 170.315(c)(2)), which at a minimum is in accordance with the standard specified in  § 170.205(k)(3). CMS Implementation Guide for Quality Reporting Document Architecture: Category III; Eligible Clinicians and Eligible Professionals Programs; Implementation Guide for 2019.

CMS QRDA Category I IG Report

  1. A user can generate a de-duplicated archive of patient documents in the CMS QRDA Category I IG format of the clinical quality measures calculated in the Execute test (§ 170.315(c)(2)), which at a minimum is in accordance with the standard specified in § 170.205(h)(3). CMS Implementation Guide for Quality Reporting Document Architecture: Category I; Hospital Quality Reporting; Implementation Guide for 2019.

Data File for Transmission

  1. The health IT developer submits the quality measurement data file consisting of the data created by the generation of the CMS QRDA Category III IG aggregate report(s) and the de-duplicated CMS QRDA Category I IG report(s) for verification.

Test Lab Setup

  1. Prior to beginning this test, the tester creates and exports data using Cypress, and the health IT developer imports the data into its Health IT Module.

CMS QRDA Category III IG Report

  1. Using the Cypress supplied XML Schema validation, the tester:
    1. uploads the aggregate report(s) submitted by the health IT developer; and
    2. runs the Cypress supplied XML schema validation for each aggregate report.
  2. The tester verifies that all of the QRDA Category III aggregate report(s) submitted by the health IT developer are at a minimum in accordance with the standard specified in § 170.205(k)(3) through evaluation of the Cypress validation report.

CMS QRDA Category I IG Report

  1. The tester verifies that all of the de-duplicated CMS QRDA Category I IG report(s), submitted by the health IT developer are at a minimum in accordance with the standard specified in § 170.205(h)(3) through evaluation of the Cypress validation report.

Data File for Transmission

  1. The tester verifies via visual inspection that the data file for transmission submitted with clinical quality measurement data includes both de-duplicated CMS QRDA Category I IG and aggregate CMS QRDA Category III IG report(s).

Content last reviewed on June 22, 2020
Was this page helpful?