§170.405 Real World Testing

Version 1.1 Updated on 08-07-2020
Revision History
Version # Description of Change Version Date
1.0

Initial Publication

06-15-2020
1.1

Updated date in regulation text based on correction notice

08-07-2020
Regulation Text
Regulation Text

§170.405 Real World Testing.

  1. Condition of Certification requirement. A health IT developer with Health IT Module(s) certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h) must successfully test the real world use of those Health IT Module(s) for interoperability (as defined in 42 U.S.C. 300jj(9) and § 170.102) in the type of setting in which such Health IT Module(s) would be/is marketed.
  2. Maintenance of Certification requirements-
    1. Real world testing plan submission. A health IT developer with Health IT Module(s) certified to any one or more of the criteria referenced in paragraph (a) of this section must submit to its ONC-ACB an annual real world testing plan addressing each of those certified Health IT Modules by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the plan on CHPL no later than December 15  of each calendar year.
      1. The plan must be approved by a health IT developer authorized representative capable of binding the health IT developer for execution of the plan and include the representative’s contact information.
      2. The plan must include all health IT certified to any one or more of the criteria referenced in paragraph (a) as of August 31 of the year in which the plan is submitted, and address the real world testing to be conducted in the calendar year immediately following plan submission.
      3. The plan must address the following for each of the certification criterion identified in paragraph (a) of this section that are included in each Health IT Module’s scope of certification:
        1. The testing method(s)/methodology(ies) that will be used to demonstrate real world interoperability and conformance to the full scope of the certification criterion’s requirements, including scenario- and use case-focused testing;
        2. The care setting(s) that will be tested for real world interoperability and an explanation for the health IT developer’s choice of care setting(s) to test;
        3. For any standards and implementation specifications referenced by the criterion that the developer has chosen to certify to National Coordinator-approved newer versions pursuant to paragraph (b)(8) or (9) of this section, a description of how the developer will test and demonstrate conformance to all requirements of the criterion using all versions of the adopted standards to which each Health IT Module was certified as of August 31 of the year in which the real world testing plan is due.
        4. A schedule of key real world testing milestones;
        5. A description of the expected outcomes of real world testing;
        6. At least one measurement/metric associated with the real world testing; and
        7. A justification for the health IT developer’s real world testing approach.
    2. Real world testing results reporting.
      1. If in the course of conducting real world testing the developer discovers one or more non-conformities with the full scope of any certification criterion under the Program, the developer must report that non-conformity to the ONC-ACB within 30 days.
      2. For real world testing activities conducted during the immediately preceding calendar year, a health IT developer must submit to its ONC-ACB an annual real world testing results report addressing each of its certified Health IT Modules that include certification criteria referenced in paragraph (a) of this section by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the results report on CHPL no later than March 15 of each calendar year. The real world testing results must report the following for each of the certification criteria identified in paragraph (a) of this section that are included in the Health IT Module’s scope of certification:
        1. The method(s) that was used to demonstrate real world interoperability;
        2. The care setting(s) that was tested for real world interoperability;
        3. The voluntary updates to standards and implementation specifications that the National Coordinator has approved through the Standards Version Advancement Process.
        4. A list of the key milestones met during real world testing;
        5. The outcomes of real world testing including a description of any challenges encountered during real world testing; and
        6. At least one measurement/metric associated with the real world testing.
    3. USCDI Updates for C-CDA. A health IT developer with health IT certified to § 170.315(b)(1), (b)(2), (e)(1), (g)(6), (f)(5), and/or (g)(9) on May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of these criteria adopted in this final rule; and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(3)(i) of this section by May 2, 2022.
    4. C-CDA Companion Guide Updates. A health IT developer with health IT certified to § 170.315(b)(1), (b)(2), (b)(9), (e)(1), (g)(6), and/or (g)(9) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of the Program criteria in the 2015 Edition; and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(4)(i) of this section by May 2, 2022.
    5. Electronic Prescribing. A health IT developer with health IT certified to § 170.315(b)(3) prior to June 30, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of this criteria adopted at § 170.315(b)(3)(ii); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(5)(i) of this section by May 2, 2022.
    6. Security tags. A health IT developer with health IT certified to § 170.315 (b)(7) and/or § 170.315 (b)(8) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of the criteria adopted in § 170.315(b)(7) and/or the revised versions of the criteria adopted in § 170.315(b)(8); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(6)(i) of this section by May 2, 2022.
    7. ASTM updates. A health IT developer with health IT certified to § 170.315(d)(2), (d)(3), and/or (d)(10) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with § 170.210(e)(1) and the standard specified in § 170.210(h); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(7)(i) of this section by May 2, 2022.
    8. Standards Version Advancement Process – voluntary updates of certified health IT to newer versions of standards and implementation specifications. A health IT developer subject to paragraph (b) of this section is permitted to update Health IT Module(s) certified to any one or more of the certification criteria referenced in paragraph (a) of this section to a newer version of any adopted standard or implementation specification included in the criterion, provided that newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. A developer that pursues such updates of its certified Health IT Module(s) must:
      1. Provide advance notice to all affected customers and its ONC-ACB-
        1. Expressing its intent to update the certified Health IT Module(s) to the National Coordinator-approved advanced version of the standard implementation specification;
        2. The developer’s expectations for how the update will affect real world interoperability for the Health IT Module(s);
        3. Whether the developer intends to continue to support the certificate(s) for the existing certified Health IT Module(s) version(s) for some period of time and how long or if the existing certified Health IT Module(s) version(s) will be deprecated; and
      2. Successfully demonstrate conformance with approved more recent versions of the standard(s) or implementation specification(s) included in each certification criterion under which the developer chooses to update its certified Health IT Module(s).
      3. Maintain the updated certified Health IT Module(s) in full conformance with all applicable Program requirements.
    9. Standards Version Advancement Process - voluntary certification to newer versions of standards and implementation specifications. A Health IT developer is permitted to seek certification for its Health IT Module(s) to any one or more of the certification criteria referenced in paragraph (a) of this section using a newer version of any adopted standard(s) or implementation specification(s) included in the criterion without first obtaining certification to the version of that adopted standard or implementation specification that is incorporated by reference in § 170.299, provided that the newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. Developers may, for each standard and implementation specification included in each criterion, choose on an itemized basis whether to seek certification to the version incorporated by reference in § 170.299, or to one or more newer version(s) approved by the National Coordinator for use in Health IT Module(s) certifications issued pursuant to section 3001(c)(5) of the Public Health Service Act, or to both.
Standard(s) Referenced
Standards Referenced

None.

Certification Companion Guide: Real World Testing

This Certification Companion Guide (CCG) is an informative document designed to assist health IT developers to meet the Conditions and Maintenance of Certification requirements.  The CCG is not a substitute for the 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Program Final Rule (ONC Cures Act Final Rule). It extracts key portions of the ONC Cures Act Final Rule’s preamble and includes subsequent clarifying interpretations. To access the full context of regulatory intent please consult the ONC Cures Act Final Rule or other included regulatory reference. The CCG is for public use and should not be sold or redistributed.

Attestation Requirements

Outlined below is a summary of the attestation requirements for the Condition and Maintenance of Certification for § 170.405 Real World Testing.  For additional details related to the requirements please refer to the ONC Cures Act Final Rule.

  • The health IT developer with Health IT Modules certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h) submits real world test plans by an authorized representative that meets the requirements outlined in § 170.405(b)(1).
  • The health IT developer submits the real world test results that meet the requirements outlined in § 170.405(b)(2) and reports any non-conformities identified during testing to the ACB within 30 days of identifying the non-conformity.   
  • The health IT developer with health IT certified to § 170.315(b)(1), (b)(2), (b)(3), (b)(7), (b)(8), (b)(9), (d)(2), (d)(3), (d)(10), (e)(1), (g)(6), (f)(5), and/or (g)(9) updates the certified health IT to be compliant with the revised 2015 Edition Cures Update version of these criteria and provides customers with the updated 2015 Edition Cures Update version by May 2, 2022 (with enforcement discretion until  August 2, 2022).
Certification Requirements

Applicability of Condition/Maintenance: Health IT Modules certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h).

Condition Explanations and Clarifications

Applies to Entire Criterion

Clarifications:

  • While health IT developers are not required to test their certified health IT in each and every setting in which it is intended for use we would expect a developer’s real world testing plan to address each type of clinical setting for which its health IT is marketed.

Paragraph (b)(1)

Clarifications:

  • Health IT developers(s) must submit an annual real world testing plan to its ONC-ACB by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the plan on CHPL no later than December 15 of each calendar year.  Initial Real World Testing plans may be posted through December 15, 2020 (with enforcement discretion until March 15, 2021).

Paragraph (b)(2)(i)

Clarifications:

  • If a health IT developer discovers one or more non-conformities with the full scope of any certification criterion, the developer must report that non-conformity to the ONC-ACB within 30 days of identifying the non-conformity.

Paragraph (b)(2)(ii)

Clarifications:

  • A health IT developer must post an annual real world testing results report to its ONC-ACB by a date that enables the ONC-ACB to publish a publicly available hyperlink to the results report on CHPL no later than March 15 of each calendar year. Initial real world testing from the 2021 performance year may be posted up through March 15, 2022 (with enforcement discretion until June 15, 2022).
  • The real world testing results report must address each of its certified Health IT Module(s) that meet applicable certification criteria.

Paragraph (b)(8)(i)

Clarifications:

  • A health IT developer that intends to update to its certified Health IT Module(s) under the Standards Version Advancement Process must provide advance notice to all affected customers and its ONC-ACB.

Content last reviewed on August 11, 2020
Was this page helpful?