§170.405 Real World Testing

Updated on 03-12-2021
Revision History
Version # Description of Change Version Date
1.0

Initial Publication

06-15-2020
1.1

Updated date in regulation text based on correction notice

 

08-07-2020
1.2

Updated compliance date per the Interim Final Rule with Comment Period (IFR), Information Blocking and the ONC Health IT Certification Program: Extension of Compliance Dates and Timeframes in Response to the COVID-19 Public Health Emergency; removed (f)(5) from C-CDA related requirements

11-02-2020
1.3

Updated to provide additional clarity on the Attestations Condition and Maintenance of Certification requirements. 

03-12-2021
Regulation Text
Regulation Text

§ 170.405 Real World Testing.

  1. Condition of Certification requirement. A health IT developer with Health IT Module(s) certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h) must successfully test the real world use of those Health IT Module(s) for interoperability (as defined in 42 U.S.C. 300jj(9) and § 170.102) in the type of setting in which such Health IT Module(s) would be/is marketed.
  2. Maintenance of Certification requirements-
    1. Real world testing plan submission. A health IT developer with Health IT Module(s) certified to any one or more of the criteria referenced in paragraph (a) of this section must submit to its ONC-ACB an annual real world testing plan addressing each of those certified Health IT Modules by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the plan on CHPL no later than December 15  of each calendar year.
      1. The plan must be approved by a health IT developer authorized representative capable of binding the health IT developer for execution of the plan and include the representative’s contact information.
      2. The plan must include all health IT certified to any one or more of the criteria referenced in paragraph (a) as of August 31 of the year in which the plan is submitted, and address the real world testing to be conducted in the calendar year immediately following plan submission.
      3. The plan must address the following for each of the certification criterion identified in paragraph (a) of this section that are included in each Health IT Module’s scope of certification:
        1. The testing method(s)/methodology(ies) that will be used to demonstrate real world interoperability and conformance to the full scope of the certification criterion’s requirements, including scenario- and use case-focused testing;
        2. The care setting(s) that will be tested for real world interoperability and an explanation for the health IT developer’s choice of care setting(s) to test;
        3. For any standards and implementation specifications referenced by the criterion that the developer has chosen to certify to National Coordinator-approved newer versions pursuant to paragraph (b)(8) or (9) of this section, a description of how the developer will test and demonstrate conformance to all requirements of the criterion using all versions of the adopted standards to which each Health IT Module was certified as of August 31 of the year in which the real world testing plan is due.
        4. A schedule of key real world testing milestones;
        5. A description of the expected outcomes of real world testing;
        6. At least one measurement/metric associated with the real world testing; and
        7. A justification for the health IT developer’s real world testing approach.
    2. Real world testing results reporting.
      1. If in the course of conducting real world testing the developer discovers one or more non-conformities with the full scope of any certification criterion under the Program, the developer must report that non-conformity to the ONC-ACB within 30 days.
      2. For real world testing activities conducted during the immediately preceding calendar year, a health IT developer must submit to its ONC-ACB an annual real world testing results report addressing each of its certified Health IT Modules that include certification criteria referenced in paragraph (a) of this section by a date determined by the ONC-ACB that enables the ONC-ACB to publish a publicly available hyperlink to the results report on CHPL no later than March 15 of each calendar year. The real world testing results must report the following for each of the certification criteria identified in paragraph (a) of this section that are included in the Health IT Module’s scope of certification:
        1. The method(s) that was used to demonstrate real world interoperability;
        2. The care setting(s) that was tested for real world interoperability;
        3. The voluntary updates to standards and implementation specifications that the National Coordinator has approved through the Standards Version Advancement Process.
        4. A list of the key milestones met during real world testing;
        5. The outcomes of real world testing including a description of any challenges encountered during real world testing; and
        6. At least one measurement/metric associated with the real world testing.
    3. USCDI Updates. A health IT developer with health IT certified to § 170.315(b)(1), (b)(2), (e)(1), (g)(6), and/or (g)(9) on May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of these criteria adopted in this final rule; and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(3)(i) of this section by December 31, 2022.
    4. C-CDA Companion Guide Updates. A health IT developer with health IT certified to § 170.315(b)(1), (b)(2), (b)(9), (e)(1), (g)(6), and/or (g)(9) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of the Program criteria in the 2015 Edition; and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(4)(i) of this section by December 31, 2022.
    5. Electronic prescribing. A health IT developer with health IT certified to § 170.315(b)(3) prior to June 30, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of this criteria adopted at § 170.315(b)(3)(ii); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(5)(i) of this section by December 31, 2022.
    6. Security tags. A health IT developer with health IT certified to § 170.315 (b)(7) and/or § 170.315 (b)(8) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with the revised versions of the criteria adopted in § 170.315(b)(7) and/or the revised versions of the criteria adopted in § 170.315(b)(8); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(6)(i) of this section by December 31, 2022.
    7. ASTM updates. A health IT developer with health IT certified to § 170.315(d)(2), (d)(3), and/or (d)(10) prior to May 1, 2020 must:
      1. Update their certified health IT to be compliant with § 170.210(e)(1) and the standard specified in § 170.210(h); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(7)(i) of this section by December 31, 2022.
    8. Standards Version Advancement Process – voluntary updates of certified health IT to newer versions of standards and implementation specifications. A health IT developer subject to paragraph (b) of this section is permitted to update Health IT Module(s) certified to any one or more of the certification criteria referenced in paragraph (a) of this section to a newer version of any adopted standard or implementation specification included in the criterion, provided that newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. A developer that pursues such updates of its certified Health IT Module(s) must:
      1. Provide advance notice to all affected customers and its ONC-ACB-
        1. Expressing its intent to update the certified Health IT Module(s) to the National Coordinator-approved advanced version of the standard implementation specification;
        2. The developer’s expectations for how the update will affect real world interoperability for the Health IT Module(s);
        3. Whether the developer intends to continue to support the certificate(s) for the existing certified Health IT Module(s) version(s) for some period of time and how long or if the existing certified Health IT Module(s) version(s) will be deprecated; and
      2. Successfully demonstrate conformance with approved more recent versions of the standard(s) or implementation specification(s) included in each certification criterion under which the developer chooses to update its certified Health IT Module(s).
      3. Maintain the updated certified Health IT Module(s) in full conformance with all applicable Program requirements.
    9. Standards Version Advancement Process - voluntary certification to newer versions of standards and implementation specifications. A Health IT developer is permitted to seek certification for its Health IT Module(s) to any one or more of the certification criteria referenced in paragraph (a) of this section using a newer version of any adopted standard(s) or implementation specification(s) included in the criterion without first obtaining certification to the version of that adopted standard or implementation specification that is incorporated by reference in § 170.299, provided that the newer version is approved by the National Coordinator for use in certifications issued under the ONC Health IT Certification Program. Developers may, for each standard and implementation specification included in each criterion, choose on an itemized basis whether to seek certification to the version incorporated by reference in § 170.299, or to one or more newer version(s) approved by the National Coordinator for use in Health IT Module(s) certifications issued pursuant to section 3001(c)(5) of the Public Health Service Act, or to both.
    10. Clinical quality measures - report. A health IT developer with health IT certified to § 170.315(c)(3) prior to June 30, 2020, must:
      1. Update their certified health IT to be compliant with the revised versions of this criteria adopted in § 170.315(c)(3); and
      2. Provide its customers of the previously certified health IT with certified health IT that meets paragraph (b)(10)(i) of this section by December 31, 2022.
Standard(s) Referenced
Standards Referenced

None.

Certification Companion Guide: Real World Testing

Attestation Requirements

Outlined below is a summary of the attestation requirements for the Real World Testing Condition and Maintenance of Certification (45 CFR § 170.405). This attestation is a part of the Attestations Condition and Maintenance of Certification requirements and will be available for developers to attest alongside the other attestation requirements in 45 CFR § 170.406 beginning on April 1, 2022, and semiannually thereafter. For additional details related to the attestation requirements please refer to the Attestations Condition and Maintenance of Certification CCG.

  • The developer with Health IT Modules certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h) submits Real World Testing plans by an authorized representative that meets the requirements outlined in § 170.405(b)(1).
  • The developer submits the Real World Testing results that meet the requirements outlined in § 170.405(b)(2) and reports any non-conformities identified during testing to the ACB within 30 days of identifying the non-conformity.   
  • The developer with health IT certified to § 170.315(b)(1), (b)(2), (b)(3), (b)(7), (b)(8), (b)(9), (c)(3), (d)(2), (d)(3), (d)(10), (e)(1), (g)(6), and/or (g)(9) updates the certified health IT to be compliant with the revised 2015 Edition Cures Update version of these criteria and provides customers with the updated 2015 Edition Cures Update version by December 31, 2022.
Certification Requirements

Applicability of Condition/Maintenance: Health IT Modules certified to any one or more 2015 Edition certification criteria in § 170.315(b), (c)(1) through (3), (e)(1), (f), (g)(7) through (10), and (h).

Condition Explanations and Clarifications

Applies to Entire Criterion

Clarifications:

  • While developers are not required to test their certified health IT in each and every setting in which it is intended for use, it is expected a developer’s Real World Testing plan will address each type of clinical setting for which its health IT is marketed.

Paragraph (b)(1) Real World Testing plan submission

Clarifications:

  • Certified Health IT Developers(s) must submit an annual real world testing plan to its ONC-ACB by a date determined by the ONC Authorized Certification Body (ONC-ACB) that enables the ONC-ACB to publish a publicly available hyperlink to the plan on CHPL no later than December 15 of each calendar year.  Initial Real World Testing plans may be posted through December 15, 2021.

Paragraph (b)(2)(i) Real World Testing results reporting

Clarifications:

  • If a developer discovers one or more non-conformities with the full scope of any certification criterion, the developer must report that non-conformity to the ONC-ACB within 30 days of identifying the non-conformity.

Paragraph (b)(2)(ii) Real World Testing results reporting

Clarifications:

  • A developer must post an annual Real World Testing results report to its ONC-ACB by a date that enables the ONC-ACB to publish a publicly available hyperlink to the results report on CHPL no later than March 15 of each calendar year. Initial Real World Testing from the 2022 performance year may be posted up through March 15, 2023. 
  • The Real World Testing results report must address each of its Certified Health IT Module(s) that meet applicable certification criteria.

Paragraph (b)(8)(i) Standards Version Advancement Process - voluntary updates of certified health IT to newer versions of standards and implementation specifications

Clarifications:

  • A developer that intends to update to its Certified Health IT Module(s) under the Standards Version Advancement Process (SVAP) must provide advance notice to all affected customers and its ONC-ACB.