ONC HIT Certification Program Test Results Summary for ... · 410-823-2201 Developer/Vendor Contact: Larry Walsh. Test Results Summary for 2014 Edition EHR Certification Version EHR-Test-144

Test Results Summary for 2014 Edition EHR Certification Version EHR-Test-144 Rev 17-Feb-2014

Page 1 of 11

ONC HIT Certification Program

Test Results Summary for 2014 Edition EHR Certification

Part 1: Product and Developer Information

1.1 Certified Product Information

Product Name: Cyfluent

Product Version: 3.1

Domain: Ambulatory

Test Type: Complete EHR

1.2 Developer/Vendor Information

Developer/Vendor Name: Cyfluent

Address: 400 East Joppa Road Towson MD 21286

Website: www.cyfluent.com

Email: [email protected]

Phone: 410-823-2201

Developer/Vendor Contact: Larry Walsh


Page 2 of 11

Part 2: ONC-Authorized Certification Body Information

2.1 ONC-Authorized Certification Body Information

ONC-ACB Name: Drummond Group

Address: 13359 North Hwy 183, Ste B-406-238, Austin, TX 78750

Website: www.drummondgroup.com


Phone: 817-294-7339

ONC-ACB Contact: Bill Smith

This test results summary is approved for public release by the following ONC-Authorized Certification Body Representative:

Bill Smith

Certification Committee Chair

ONC-ACB Authorized Representative Function/Title

6/20/2014

Signature and Date

2.2 Gap Certification

The following identifies criterion or criteria certified via gap certification

§170.314

(a)(1) (a)(17) (d)(5) (d)(9)

(a)(6) (b)(5)* (d)(6) (f)(1)

(a)(7) (d)(1) (d)(8)

*Gap certification allowed for Inpatient setting only

x No gap certification

http://www.drummondgroup.com/

mailto:[email protected]


Page 3 of 11

2.3 Inherited Certification

The following identifies criterion or criteria certified via inherited certification

§170.314

x (a)(1) x (a)(14) (c)(3) x (f)(1)

x (a)(2) x (a)(15) x (d)(1) x (f)(2)

x (a)(3) (a)(16) Inpt. only x (d)(2) x (f)(3)

x (a)(4) (a)(17) Inpt. only x (d)(3) (f)(4) Inpt. only

x (a)(5) x (b)(1) x (d)(4)

(f)(5) Optional & Amb. only x (a)(6) x (b)(2) x (d)(5)

x (a)(7) x (b)(3) x (d)(6)

(f)(6) Optional & Amb. only x (a)(8) x (b)(4) x (d)(7)

x (a)(9) x (b)(5) x (d)(8) (g)(1)

x (a)(10) (b)(6) Inpt. only (d)(9) Optional x (g)(2)

x (a)(11) x (b)(7) x (e)(1) x (g)(3)

x (a)(12) (c)(1) x (e)(2) Amb. only x (g)(4)

x (a)(13) (c)(2) x (e)(3) Amb. only

No inherited certification


Page 4 of 11

Part 3: NVLAP-Accredited Testing Laboratory Information

Report Number: TEB-06162014-2846

Test Date(s): 6/16/2014

3.1 NVLAP-Accredited Testing Laboratory Information

ATL Name: Drummond Group EHR Test Lab

Accreditation Number: NVLAP Lab Code 200979-0

Address: 13359 North Hwy 183, Ste B-406-238, Austin, TX 78750

Website: www.drummondgroup.com


Phone: 512-335-5606

ATL Contact: Beth Morrow

For more information on scope of accreditation, please reference NVLAP Lab Code 200979-0.

Part 3 of this test results summary is approved for public release by the following Accredited Testing Laboratory Representative:

Timothy Bennett

Test Proctor

ATL Authorized Representative Function/Title

6/20/2014

Nashville, TN Signature and Date Location Where Test Conducted

3.2 Test Information

3.2.1 Additional Software Relied Upon for Certification

Additional Software Applicable Criteria Functionality provided by Additional Software

Emdeon a.2, b.3 e-Prescribing, drug interaction checks

http://ts.nist.gov/standards/scopes/2009790.htm

http://www.drummondgroup.com/


http://ts.nist.gov/standards/scopes/2009790.htm


Page 5 of 11

Additional Software Applicable Criteria Functionality provided by Additional Software

No additional software required

3.2.2 Test Tools

Test Tool Version

x Cypress 2.4.1

x ePrescribing Validation Tool 1.0.3

HL7 CDA Cancer Registry Reporting Validation Tool 1.0.3

HL7 v2 Electronic Laboratory Reporting (ELR) Validation Tool 1.8

x HL7 v2 Immunization Information System (IIS) Reporting Validation Tool

1.7.1

x HL7 v2 Laboratory Results Interface (LRI) Validation Tool 1.7

x HL7 v2 Syndromic Surveillance Reporting Validation Tool 1.7

x Transport Testing Tool 174

x Direct Certificate Discovery Tool 2.1

No test tools required

3.2.3 Test Data

Alteration (customization) to the test data was necessary and is described in Appendix [insert appendix letter]

No alteration (customization) to the test data was necessary

3.2.4 Standards

3.2.4.1 Multiple Standards Permitted

The following identifies the standard(s) that has been successfully tested where more than one standard is permitted

Criterion # Standard Successfully Tested

(a)(8)(ii)(A)(2)

§170.204(b)(1)

HL7 Version 3 Implementation Guide: URL-Based Implementations of the Context-Aware Information Retrieval (Infobutton) Domain

§170.204(b)(2)

HL7 Version 3 Implementation Guide: Context-Aware Knowledge Retrieval (Infobutton) Service-Oriented Architecture Implementation Guide


Page 6 of 11


(a)(13)

x §170.207(a)(3)

IHTSDO SNOMED CT® International Release July 2012 and US Extension to SNOMED CT® March 2012 Release

§170.207(j)

HL7 Version 3 Standard: Clinical Genomics; Pedigree

(a)(15)(i)

x §170.204(b)(1)

HL7 Version 3 Implementation Guide: URL-Based Implementations of the Context-Aware Information Retrieval (Infobutton) Domain

§170.204(b)(2)

HL7 Version 3 Implementation Guide: Context-Aware Knowledge Retrieval (Infobutton) Service-Oriented Architecture Implementation Guide

(a)(16)(ii) §170.210(g)

Network Time Protocol Version 3 (RFC 1305)

§170. 210(g)


(b)(2)(i)(A)

§170.207(i)

The code set specified at 45 CFR 162.1002(c)(2) (ICD-10-CM) for the indicated conditions

x §170.207(a)(3)


(b)(7)(i)

§170.207(i)

The code set specified at 45 CFR 162.1002(c)(2) (ICD-10-CM) for the indicated conditions

x §170.207(a)(3)


(e)(1)(i)

Annex A of the FIPS Publication 140-2

[list encryption and hashing algorithms]

AES SHA-1

(e)(1)(ii)(A)(2) §170.210(g)


x §170. 210(g)


(e)(3)(ii)

Annex A of the FIPS Publication 140-2

[list encryption and hashing algorithms]

AES SHA-1

Common MU Data Set (15)

x §170.207(a)(3)


§170.207(b)(2)

The code set specified at 45 CFR 162.1002(a)(5) (HCPCS and CPT-4)


Page 7 of 11


None of the criteria and corresponding standards listed above are applicable

3.2.4.2 Newer Versions of Standards

The following identifies the newer version of a minimum standard(s) that has been successfully tested

Newer Version Applicable Criteria

No newer version of a minimum standard was tested

3.2.5 Optional Functionality

Criterion # Optional Functionality Successfully Tested

x (a)(4)(iii) Plot and display growth charts

(b)(1)(i)(B) Receive summary care record using the standards specified at §170.202(a) and (b) (Direct and XDM Validation)

(b)(1)(i)(C) Receive summary care record using the standards specified at §170.202(b) and (c) (SOAP Protocols)

(b)(2)(ii)(B) Transmit health information to a Third Party using the standards specified at §170.202(a) and (b) (Direct and XDM Validation)

(b)(2)(ii)(C) Transmit health information to a Third Party using the standards specified at §170.202(b) and (c) (SOAP Protocols)

x (f)(3)

Ambulatory setting only – Create syndrome-based public health surveillance information for transmission using the standard specified at §170.205(d)(3) (urgent care visit scenario)


Express Procedures according to the standard specified at §170.207(b)(3) (45 CFR162.1002(a)(4): Code on Dental Procedures and Nomenclature)


Express Procedures according to the standard specified at §170.207(b)(4) (45 CFR162.1002(c)(3): ICD-10-PCS)

No optional functionality tested


Page 8 of 11

3.2.6 2014 Edition Certification Criteria* Successfully Tested

Criteria # Version

Criteria # Version

TP** TD*** TP TD

(a)(1) 1.2 1.5 x (c)(3) 1.6 1.6

(a)(2) 1.2 (d)(1) 1.2

(a)(3) 1.2 1.4 (d)(2) 1.5

(a)(4) 1.4 1.3 (d)(3) 1.3

(a)(5) 1.4 1.3 (d)(4) 1.3

(a)(6) 1.3 1.4 (d)(5) 1.2

(a)(7) 1.3 1.3 (d)(6) 1.2

(a)(8) 1.2 (d)(7) 1.2

(a)(9) 1.3 1.3 (d)(8) 1.2

(a)(10) 1.2 1.4 (d)(9) Optional 1.2

(a)(11) 1.3

(e)(1) 1.8 1.5

(a)(12) 1.3 (e)(2) Amb. only 1.2 1.6

(a)(13) 1.2 (e)(3) Amb. only 1.3

(a)(14) 1.2 (f)(1) 1.2 1.2

(a)(15) 1.5 (f)(2) 1.3 1.7.1

(a)(16) Inpt. only 1.3 1.2 (f)(3) 1.3 1.7

(a)(17) Inpt. only 1.2 (f)(4) Inpt. only 1.3 1.7

(b)(1) 1.7 1.4

(f)(5) Optional & Amb. only

1.2 1.2 (b)(2) 1.4 1.6

(b)(3) 1.4 1.2

(f)(6) Optional & Amb. only

1.3 1.0.3 (b)(4) 1.3 1.4

(b)(5) 1.4 1.7 (g)(1) 1.7 1.9

(b)(6) Inpt. only 1.3 1.7 (g)(2) 1.7 1.9

(b)(7) 1.4 1.6 (g)(3) 1.3 x (c)(1) 1.6 1.6 (g)(4) 1.2

x (c)(2) 1.6 1.6

No criteria tested

*For a list of the 2014 Edition Certification Criteria, please reference http://www.healthit.gov/certification (navigation: 2014 Edition Test Method)

**Indicates the version number for the Test Procedure (TP)

***Indicates the version number for the Test Data (TD)

http://www.healthit.gov/certification


Page 9 of 11

3.2.7 2014 Clinical Quality Measures*

Type of Clinical Quality Measures Successfully Tested:

x Ambulatory

Inpatient

No CQMs tested

*For a list of the 2014 Clinical Quality Measures, please reference http://www.cms.gov (navigation: 2014 Clinical Quality Measures)

Ambulatory CQMs

CMS ID Version CMS ID Version CMS ID Version CMS ID Version

x 2 v3 x 90 v3 x 136 v3 x 155 v2

x 22 v2 x 117 v2 x 137 v2 x 156 v2

x 50 v2 x 122 v2 x 138 v2 x 157 v2

x 52 v2 x 123 v2 x 139 v2 x 158 v2

x 56 v2 x 124 v2 x 140 v2 x 159 v2

x 61 v3 x 125 v2 x 141 v3 x 160 v2

x 62 v2 x 126 v2 x 142 v2 x 161 v2

x 64 v3 x 127 v2 x 143 v2 x 163 v2

x 65 v3 x 128 v2 x 144 v2 x 164 v2

x 66 v2 x 129 v3 x 145 v2 x 165 v2

x 68 v3 x 130 v2 x 146 v2 x 166 v3

x 69 v2 x 131 v2 x 147 v2 x 167 v2

x 74 v3 x 132 v2 x 148 v2 x 169 v2

x 75 v2 x 133 v2 x 149 v2 x 177 v2

x 77 v2 x 134 v2 x 153 v2 x 179 v2

x 82 v1 x 135 v2 x 154 v2 x 182 v3

Inpatient CQMs

CMS ID Version CMS ID Version CMS ID Version CMS ID Version

9 71 107 172

26 72 108 178

30 73 109 185

31 91 110 188

32 100 111 190

53 102 113

55 104 114

60 105 171

http://www.cms.gov/


Page 10 of 11

3.2.8 Automated Numerator Recording and Measure Calculation

3.2.8.1 Automated Numerator Recording

Automated Numerator Recording Successfully Tested

(a)(1) (a)(9) (a)(16) (b)(6)

(a)(3) (a)(11) (a)(17) (e)(1)

(a)(4) (a)(12) (b)(2) (e)(2)

(a)(5) (a)(13) (b)(3) (e)(3)

(a)(6) (a)(14) (b)(4)

(a)(7) (a)(15) (b)(5)

x Automated Numerator Recording was not tested

3.2.8.2 Automated Measure Calculation

Automated Measure Calculation Successfully Tested

x (a)(1) x (a)(9) (a)(16) (b)(6)

x (a)(3) x (a)(11) (a)(17) x (e)(1)

x (a)(4) x (a)(12) x (b)(2) x (e)(2)

x (a)(5) x (a)(13) x (b)(3) x (e)(3)

x (a)(6) x (a)(14) x (b)(4)

x (a)(7) x (a)(15) x (b)(5)

Automated Measure Calculation was not tested

3.2.9 Attestation

Attestation Forms (as applicable) Appendix

x Safety-Enhanced Design* A

x Quality Management System** B

x Privacy and Security C

*Required if any of the following were tested: (a)(1), (a)(2), (a)(6), (a)(7), (a)(8), (a)(16), (b)(3), (b)(4)

**Required for every EHR product

3.3 Appendices

Attached below.


Page 11 of 11

Test Results Summary Document History

Version Description of Change Date

17-Feb-2014 Edited: section header page 3; contact info page 4

17-Feb-2014

10-Feb-2014 Modified layout 10-Feb-2014

20-Nov-2013 Updated test tool sections 20-Nov-2013

25-Oct-2013 Corrected numbering of 3.2.8 section 25-Oct-2013

15-Oct-2013 Modified layout slightly 15-Oct-2013

01-Oct-2013 Initial Version 01-Oct-2013

2014 Edition Test Report Summary

USER CENTER DESIGN REPORT – TEST REPORT UPDATE

This test report was updated in December 2015 to satisfy User Center Design Report specifications by ONC. The new Test Report ID is amended as follows: “Part 3: NVLAP-Accredited Testing Laboratory Information: Report Number” plus the suffix “_Dec2015”.

5-Dec-13

EHR Usability Test Report

EHR Usability Test Report of Cyfluent, Version 3.0

Report based on ISO/IEC 25062:2006 Common Industry Format for Usability Test Reports

Cyfluent Version 3.0

Date of Usability Test: 12/03/2013

Date of Report: 12/05/2013

Report Prepared By: Cyfluent

Elaine Law, VP of Operations

210-823-1614

[email protected]

400 Joppa Road

Towson, MD 21204


Page 2 of 37

Table of Contents

EXECUTIVE SUMMARY ................................................................................................................................ 3

INTRODUCTION ............................................................................................................................................ 8

PARTICIPANTS .......................................................................................................................................... 8

STUDY DESIGN ......................................................................................................................................... 9

TASKS .......................................................................................................................................................... 9

PROCEDURES ......................................................................................................................................... 11

TEST LOCATION ...................................................................................................................................... 11

TEST ENVIRONMENT............................................................................................................................. 12

PARTICIPANT INSTRUCTIONS ............................................................................................................ 12

USABILITY METRICS .............................................................................................................................. 14

DATA SCORING ........................................................................................................................................... 14

RESULTS ....................................................................................................................................................... 16

DATA ANALYSIS AND REPORTING.................................................................................................... 16

DISCUSSION OF THE FINDINGS ........................................................................................................ 19

EFFECTIVENESS ................................................................................................................................ 19

EFFICIENCY ......................................................................................................................................... 19

SATISFACTION .................................................................................................................................... 19

MAJOR FINDINGS ............................................................................................................................... 20

AREAS FOR IMPROVEMENT ........................................................................................................... 20

APPENDICES ............................................................................................................................................... 21

Appendix 1: PARTICIPANT DEMOGRAPHICS ................................................................................... 22

Appendix 2: SYSTEM USABILITY SCALE QUESTIONNAIRE ......................................................... 23

Page 3 of 37

EXECUTIVE SUMMARY

A usability test of Cyfluent Version 3.0, Ambulatory was conducted December 3-5 via WebEx by Cyfluent. The purpose of this test was to test and validate the usability of the current user interface, and provide evidence of usability in the EHR Under Test (EHRUT).

During the usability test, 5 healthcare providers and users matching the target demographic criteria served as participants and used the EHRUT in simulated, but representative tasks. This study collected performance data on 27 tasks typically conducted on an EHR:

1) 170.314(a)(1) Computerized provider order entry

a) Create a lab order

b) Enter lab results

c) View radiology images

2) 170.314(a)(2) Drug-drug, drug-allergy interaction checks

a) View drug-drug and drug-allergy interaction checks

3) 170.314(a)(6) Medication list

a) Issue a reported medication

4) 170.314(a)(7) Medication allergy list

a) Enter medication allergies

5) 170.314(a)(8) Clinical decision support

a) Select an existing patient record and perform actions that generate CDS interventions, triggered

based each of the following data categories within that patient record and at least one

combination of data from two or more of the following data categories:

i) Problem list

ii) Medication list

iii) Medication allergy list

iv) Demographics

v) Laboratory tests and values/results

vi) Vital signs

6) 170.314(b)(3) Electronic prescribing

7) 170.314(b)(4) Clinical information reconciliation

During the 60 minute one-on-one usability test, each participant was greeted by the

administrator. Participants had prior experience with the EHR. The administrator introduced

the test, and instructed participants to complete a series of tasks (given one at a time) using the

EHRUT. During the testing, the administrator timed the test and, along with the data logger(s)

recorded user performance data on paper and electronically. The administrator did not give the

participant assistance in how to complete the task.

Page 4 of 37

Participant screens, head shots and audio were recorded for subsequent analysis.

The following types of data were collected for each participant:

Number of tasks successfully completed within the allotted time without assistance

Time to complete the tasks

Number and types of errors

Path deviations

Participant’s verbalizations

Participant’s satisfaction ratings of the system

All participant data was de-identified – no correspondence could be made from the identity of

the participant to the data collected. Following the conclusion of the testing, participants were

asked to complete a post-test questionnaire and were not compensated for their time. Various

recommended metrics, in accordance with the examples set forth in the NIST Guide to the

Processes Approach for Improving the Usability of Electronic Health Records, were used to

evaluate the usability of the EHRUT. Following is a summary of the performance rating data

collected on the EHRUT.

Task N Task

Success Path Deviation Task

Time Errors Task

ratings

# Mean (SD)

Deviations (Observed/Optimal)

Mean (SD)

Mean (SD)

Mean (SD)

1. Enter a reported medication of Avandia 2 mg

5 1.0 40/40 58 0 4.8

2. Change the reported medication from Avandia 2 mg to Avandia 4 mg

5 1.0 51/40 62 0 4.4

3. Access the Avandia 4 mg medication

5 1.0 5/5 6.8 0 5.0

4. Open the most recent encounter for patient

5 1.0 13/5 24 0 5.0

5. Enter a lab order of Creatinine 24 Hour

5 1.0 21/20 26.4 0 5.0

6. Edit the lab order of Creatinine 24 hours and add results value of 8

5 1.0 25/20 33.8 0 5.0

7. Access the lab order of Creatinine 24 hours

5 1.0 12/10 15 0 4.8

Page 5 of 37

Task N Task Success

Path Deviation Task Time

Errors Task ratings

# Mean (SD)


Mean (SD)

Mean (SD)

Mean (SD)

8. Enter a radiology order of Chest X-Ray

5 1.0 21/20 17 0 5.0

9. Edit a radiology order of Chest X-Ray and assign to Medical Assistant

5 1.0 21/20 25.2 0 5.0

10. Access the Chest X-Ray radiology order

5 1.0 16/10 17.4 0 5.0

11. Enter a prescription for Hydrocodone to view DUR Warning for drug-allergy interaction for prescriber William Mayfield

5 1.0 45/30 79.4 0 4.8

12. Enter a prescription for Phenelzine to view DUR warning for drug-drug interaction for prescriber William Mayfield

5 1.0 31/30 33.0 0 4.8

13. Adjust the DUR preferences for William Mayfield

5 1.0 34/20 50.4 0 5.0

14. Enter a reported medication to Simvastatin 10 Mg

5 1.0 47/45 53

0 4.8

15. Change reported medication from Simvastatin 10 Mg to reported Simvastatin 20 Mg

5 1.0 43/35 45.8 0 4.8

16. Access simvastatin 20 mg reported medication

5 1.0 10/5 11.0 0 5.0

17. Record allergy to Carbamazepine with reaction of skin rash

5 1.0 22/20 27.2 0 5.0

Page 6 of 37

Task N Task Success


Errors Task ratings

# Mean (SD)


Mean (SD)

Mean (SD)

Mean (SD)

18. change allergy from Carbamazepine to Codeine

5 0.8 14/12 17.8 0.2 5.0

19. Access Codeine Allergy

5 1.0 5/5 6.2 0 5.0

20. Acknowledge Avandia Alert

5 1.0 15/15 13.4 0 5.0

21. Acknowledge Fasting Blood Glucose Alert

5 1.0 15/15 11 0 5.0

22. Open Therapeutic Reference Information for Diabetes Alert

5 1.0 14/10 29.6 0 5.0

23. Create a gender based alert in administration

5 0.8 33/24 56.25 0.2 5.0

24. Issue a prescription for Hydrochlorothiazide 50 Mg for provider William Mayfield

5 1.0 52/35 63.0 0 5.0

25. Reconcile patient’s active medication list

5 0.6 14/12 22.3 0.4 5.0

26. Reconcile patient’s active problem list

5 1.0 15/15 21.8 0 5.0

27. Reconcile patient’s allergy list

5 1.0 25/25 13.8 5 5.0

Page 7 of 37

The results from the System Usability Scale scored the subjective satisfaction with the system

based on performance with these tasks to be: 96%.

In addition to the performance data, the following qualitative observations were made:

1) Major findings a) 18. Change allergy from Carbamazepine to Codeine

b) 23. Create a gender based alert in administration

c) 25. Reconcile patient’s active medication list

2) Areas for improvement a) 2. Change the reported medication from Avandia 2 mg to Avandia 4 mg

b) 11. Enter a prescription for Hydrocodone to view DUR Warning for drug-allergy interaction for

prescriber William Mayfield

c) 24. Issue a prescription for Hydrochlorothiazide 50 Mg for provider William Mayfield

Page 8 of 37

INTRODUCTION

The EHRUT tested for this study was Cyfluent, Version 3.0. Designed to present medical

information to healthcare providers in Ambulatory settings, the EHRUT consists of an Electronic

Health Record. The usability testing attempted to represent realistic exercises and conditions.

The purpose of this study was to test and validate the usability of the current user interface, and

provide evidence of usability in the EHR Under Test (EHRUT). To this end, measures of

effectiveness, efficiency and user satisfaction, such as electronic prescribing were captured

during the usability testing.

METHOD

PARTICIPANTS

A total of 5 participants were tested on the EHRUT(s). Participants in the test were Physicians,

Nurses, Administrators and Billing specialists. Participants were recruited by Elaine Law,

Cyfluent, and were not compensated for their time. In addition, participants had no direct

connection to the development of or organization producing the EHRUT(s). Participants were

not from the testing or supplier organization. Participants were given the opportunity to have the

same orientation and level of training as the actual end users would have received.

Recruited participants had a mix of backgrounds and demographic characteristics conforming to

the recruitment screener. The following is a table of participants by characteristics, including

demographics, professional experience, computing experience and user needs for assistive

technology. Participant names were replaced with Participant IDs so that an individual’s data

cannot be tied back to individual identities.

Part ID

Gender

Occupation/

role

Professional Experience

Computer Experience

Product

Experience

Assistive Technology

Needs 1 User 1 Female Precertification Office Assistant

10 Years Experienced 6 months None

2 User 2 Female Asst Office Mgr. and Med. Coder

14 Years Experienced 6 months None

3 User 3 Female RN and Office Manager 20 Years Experienced 2 years None

4 User 4 Male Physician/Owner 35 Years Experienced 2 years None

4 User 5 Female Certified Ophthalmic Tech. 17 Years Experienced 2 years None

Five participants (matching the demographics in the section on Participants) were recruited and

five participated in the usability test. Zero participants failed to show for the study.

Page 9 of 37

Participants were scheduled for 60 minute sessions without time in between each session for

debrief by the administrator(s) and data logger(s), and to reset systems to proper test

conditions. A spreadsheet was used to keep track of the participant schedule, and included

each participant’s demographic characteristics as provided by Cyfluent.

STUDY DESIGN

Overall, the objective of this test was to uncover areas where the application performed well –

that is, effectively, efficiently, and with satisfaction – and areas where the application failed to

meet the needs of the participants. The data from this test may serve as a baseline for future

tests with an updated version of the same EHR and/or comparison with other EHRs provided

the same tasks are used. In short, this testing serves as both a means to record or benchmark

current usability, but also to identify areas where improvements must be made.

During the usability test, participants interacted with 1 EHR. Each participant used the system in

the same location, and was provided with the same instructions. The system was evaluated for

effectiveness, efficiency and satisfaction as defined by measures collected and analyzed for

each participant:

Number of tasks successfully completed within the allotted time without assistance

1. Time to complete the tasks

2. Number and types of errors

3. Path deviations

4. Participant’s verbalizations (comments)

5. Participant’s satisfaction ratings of the system

Additional information about the various measures can be found in Section 3.9 on Usability

Metrics.

TASKS

A number of tasks were constructed that would be realistic and representative of the kinds of

Page 10 of 37

activities a user might do with this EHR, including:

6.

7. Enter a REPORTED medication of AVANDIA 2 MG

8. Change the REPORTED medication from AVANDIA 2 MG to AVANDIA 4 MG via

Emdeon Rx

9. Access the AVANDIA 4 MG medication via Emdeon Rx


11. Enter a lab order of CREATININE 24 HOUR

12. Edit the lab order of CREATININE 24 HOURS and ADD RESULTS VALUE OF 8

13. Access the lab order of CREATININE 24 HOURS

14. Enter a radiology order of CHEST X-RAY via Assessment/Plan


16. Access the CHEST X-RAY radiology order

17. Enter a prescription for HYDROCODONE to view DUR WARNING for drug-allergy

interaction for prescriber William Mayfield

18. Enter a prescription for PHENELZINE to view DUR warning for drug-drug interaction for



20. Enter a REPORTED medication to SIMVASTATIN 10 MG

21. Change REPORTED medication from SIMVASTATIN 10 MG to REPORTED

SIMVASTATIN 20 MG

22. Access SIMVASTATIN 20 MG reported medication

23. Record ALLERGY to CARBAMAZEPINE with REACTION of SKIN RASH

24. Change ALLERGY from CARBAMAZEPINE to CODEINE

25. Access CODEINE Allergy




29. CREATE a gender based alert in administration

30. Issue a prescription for HYDROCHLOROTHIAZIDE 50 MG for provider WILLIAM

MAYFIELD.

Page 11 of 37

31. RECONCILE patient’s active MEDICATION list

32. RECONCILE patient’s active PROBLEM list

33. RECONCILE patient’s ALLERGY list

34.

Tasks were selected based on their frequency of use, criticality of function, and those that may

be most troublesome for users.

PROCEDURES

Upon arrival, participants were greeted; their identity was verified and matched with a name on

the participant schedule. To ensure that the test ran smoothly, one staff member participated in

this test and recorded for later data logging. The usability testing staff conducting the test was

experienced usability practitioners with five years of experience and thorough knowledge of the

system.

The administrator moderated the session including administering instructions and tasks. The

administrator also monitored task times, obtained post-task rating data, and took notes on

participant comments. A recording was used to capture the data on task success, path

deviations, number and type of errors, and comments.

Participants were instructed to perform the tasks (see specific instructions below):

1. As quickly as possible making as few errors and deviations as possible.

2. Without assistance; administrators were allowed to give immaterial guidance and

clarification on tasks, but not instructions on use.

For each task, the participants were given a written copy of the task. Task timing began once

the administrator finished reading the question. The task time was stopped once the participant

indicated they had successfully completed the task. Scoring is discussed below in Section 3.9.

Participants' demographic information, task success rate, time on task, errors, deviations and

verbal responses were recorded into a spreadsheet. Participants were thanked for their time.

TEST LOCATION

The test facility included a web conference, computer for the participant, and recording

computer for the administrator. Only the participant and administrator were in the test room. All

Page 12 of 37

observers and the data logger worked from a separate room where they could see the

participant’s screen and face shot, and listen to the audio of the session. To ensure that the

environment was comfortable for users, noise levels were kept to a minimum with the ambient

temperature within a normal range. All of the safety instruction and evacuation procedures were

valid, in place, and visible to the participants.

TEST ENVIRONMENT

The EHRUT would be typically be used in a healthcare office or facility. In this instance, the

testing was conducted in various facilities. For testing, the computer used various laptops

running various operating systems. The participants used a mouse and keyboard when

interacting with the EHRUT.

The EHRUT used various screen sizes, resolutions and color settings based on the participants’

equipment. The application was set up by Cyfluent. The application itself was running on the

internet using a test database on a wireless connection. Technically, the system performance

was representative to what actual users would experience in a field implementation.

PARTICIPANT INSTRUCTIONS

The administrator reads the following instructions aloud to the each participant:

Thank you for participating in this study. Your input is very important. Our session today will last

about 60 minutes. During that time you will use an instance of an electronic health record. I will

ask you to complete a few tasks using this system and answer some questions. You should

complete the tasks as quickly as possible making as few errors as possible. Please try to

complete the tasks on your own following the instructions very closely. Please note that we are

not testing you we are testing the system, therefore if you have difficulty all this means is that

something needs to be improved in the system. I will be here in case you need specific help, but

I am not able to instruct you or provide help in how to use the application.

Overall, we are interested in how easy (or how difficult) this system is to use, what in it would be

useful to you, and how we could improve it. I did not have any involvement in its creation, so

please be honest with your opinions. All of the information that you provide will be kept

confidential and your name will not be associated with your comments at any time. Should you

Page 13 of 37

feel it necessary you are able to withdraw at any time during the testing.

For each task, I will read the description to you and say “Begin.” At that point, please perform

the task and say “Done” once you believe you have successfully completed the task. I would

like to request that you not talk aloud or verbalize while you are doing the task. I will ask you

your impressions about the task once you are done.

Participants were then given 27 tasks to complete:

1. Enter a REPORTED medication of AVANDIA 2 MG

2. Change the REPORTED medication from AVANDIA 2 MG to AVANDIA 4 MG via

Emdeon Rx

3. Access the AVANDIA 4 MG medication via Emdeon Rx


5. Enter a lab order of CREATININE 24 HOUR

6. Edit the lab order of CREATININE 24 HOURS and ADD RESULTS VALUE OF 8

7. Access the lab order of CREATININE 24 HOURS

8. Enter a radiology order of CHEST X-RAY via Assessment/Plan


10. Access the CHEST X-RAY radiology order

11. Enter a prescription for HYDROCODONE to view DUR WARNING for drug-allergy

interaction for prescriber William Mayfield

12. Enter a prescription for PHENELZINE to view DUR warning for drug-drug interaction for



14. Enter a REPORTED medication to SIMVASTATIN 10 MG

15. Change REPORTED medication from SIMVASTATIN 10 MG to REPORTED

SIMVASTATIN 20 MG

16. Access SIMVASTATIN 20 MG reported medication

17. Record ALLERGY to CARBAMAZEPINE with REACTION of SKIN RASH

18. Change ALLERGY from CARBAMAZEPINE to CODEINE

19. Access CODEINE Allergy

Page 14 of 37




23. CREATE a gender based alert in administration

24. Issue a prescription for HYDROCHLOROTHIAZIDE 50 MG for provider WILLIAM

MAYFIELD.

25. RECONCILE patient’s active MEDICATION list

26. RECONCILE patient’s active PROBLEM list

27. RECONCILE patient’s ALLERGY list

USABILITY METRICS

According to the NIST Guide to the Processes Approach for Improving the Usability of

Electronic Health Records, EHRs should support a process that provides a high level of usability

for all users. The goal is for users to interact with the system effectively, efficiently, and with an

acceptable level of satisfaction. To this end, metrics for effectiveness, efficiency and user

satisfaction were captured during the usability testing.

The goals of the test were to assess:

1. Effectiveness of Cyfluent by measuring participant success rates and errors

2. Efficiency of Cyfluent by measuring the average task time and path deviations

3. Satisfaction with Cyfluent by measuring ease of use ratings

DATA SCORING

The following table (Table 1) details how tasks were scored, errors evaluated, and the time data

analyzed.

Measures Rationale and Scoring

Page 15 of 37

Effectiveness:

Task Success

A task was counted as a “Success” if the participant was able to achieve the correct outcome, without assistance, within the time allotted on a per task basis. The total number of successes were calculated for each task and then divided by the total number of times that task was attempted. The results are provided as a percentage. Task times were recorded for successes. Observed task times divided by the optimal time for each task is a measure of optimal efficiency. Optimal task performance time, as benchmarked by expert performance under realistic conditions, is recorded when constructing tasks. Target task times used for task times in the Moderator’s Guide must be operationally defined by taking multiple measures of optimal performance and multiplying by some factor that allows some time buffer because the participants are presumably not trained to expert performance. Thus, if expert, optimal performance on a task was seconds then allotted task time performance was seconds. This ratio should be aggregated across tasks and reported with mean and variance scores. Effectiveness:

Task Failures

If the participant abandoned the task, did not reach the correct answer or performed it incorrectly, or reached the end of the allotted time before successful completion, the task was counted as “Failures.” No task times were taken for errors. The total number of errors was calculated for each task and then divided by the total number of times that task was attempted. Not all deviations would be counted as errors.

This should also be expressed

as the mean number of failed tasks per participant. On a qualitative level, an enumeration of errors and error types should be collected.

Efficiency:

Task Deviations

The participant’s path (i.e., steps) through the application was recorded. Deviations occur if the participant, for example, went to a wrong screen, clicked on an incorrect menu item, followed an incorrect link, or interacted incorrectly with an on-screen control. This path was compared to the optimal path. The number of steps in the observed path is divided by the number of optimal steps to provide a ratio of path deviation. It is strongly recommended that task deviations be reported. Optimal paths (i.e., procedural steps) should be recorded when constructing tasks. Efficiency:

Task Time

Each task was timed from when the administrator said “Begin” until the participant said, “Done.” If he or she failed to say “Done,” the time was stopped when the participant stopped performing the task. Only task times for tasks that were successfully completed were included in the average task time analysis. Average time per task was calculated for each task. Variance measures (standard deviation and standard error) were also calculated.

Page 16 of 37

Satisfaction:

Task Rating

Participant’s subjective impression of the ease of use of the application was measured by administering both a simple post-task question as well as a post-session questionnaire. After each task, the participant was asked to rate “Overall, this task was:” on a scale of 1 (Very Difficult) to 5 (Very Easy). These data are averaged across participants. Common convention is that average ratings for systems judged easy to use should be 3.3 or above. To measure participants’ confidence in and likeability of the Cyfluent EHR overall, the testing team administered the System Usability Scale (SUS) post-test questionnaire.

Table 1 - Details of how observed data were scored.

RESULTS

DATA ANALYSIS AND REPORTING

The results of the usability test were calculated according to the methods specified in the

Usability Metrics section above. Participants who failed to follow session and task instructions

had their data excluded from the analyses. The usability testing results for the EHRUT are

detailed below (see Table2 ). The results should be seen in light of the objectives and goals

outlined in Section 3.2 Study Design. The data should yield actionable results that, if corrected,

yield material, positive impact on user performance.

Task N Task Success


Errors Task ratings

# Mean (SD)


Mean (SD)

Mean (SD)

Mean (SD)


5 1.0 40/40 58 0 4.8


5 1.0 51/40 62 0 4.4

30. Access the Avandia 4 mg medication

5 1.0 5/5 6.8 0 5.0


5 1.0 13/5 24 0 5.0

32. Enter a lab order of Creatinine 24 Hour

5 1.0 21/20 26.4 0 5.0

Page 17 of 37

Task N Task Success


Errors Task ratings

# Mean (SD)


Mean (SD)

Mean (SD)

Mean (SD)

33. Edit the lab order of Creatinine 24 hours and add results value of 8

5 1.0 25/20 33.8 0 5.0


5 1.0 12/10 15 0 4.8

35. Enter a radiology order of Chest X-Ray

5 1.0 21/20 17 0 5.0


5 1.0 21/20 25.2 0 5.0

37. Access the Chest X-Ray radiology order

5 1.0 16/10 17.4 0 5.0

38. Enter a prescription for Hydrocodone to view DUR Warning for drug-allergy interaction for prescriber William Mayfield

5 1.0 45/30 79.4 0 4.8

39. Enter a prescription for Phenelzine to view DUR warning for drug-drug interaction for prescriber William Mayfield

5 1.0 31/30 33.0 0 4.8


5 1.0 34/20 50.4 0 5.0


5 1.0 47/45 53

0 4.8


5 1.0 43/35 45.8 0 4.8

Page 18 of 37

Task N Task Success


Errors Task ratings

# Mean (SD)


Mean (SD)

Mean (SD)

Mean (SD)

43. Access simvastatin 20 mg reported medication

5 1.0 10/5 11.0 0 5.0

44. Record allergy to Carbamazepine with reaction of skin rash

5 1.0 22/20 27.2 0 5.0

45. change allergy from Carbamazepine to Codeine

5 0.8 14/12 17.8 0.2 5.0

46. Access Codeine Allergy

5 1.0 5/5 6.2 0 5.0


5 1.0 15/15 13.4 0 5.0


5 1.0 15/15 11 0 5.0


5 1.0 14/10 29.6 0 5.0


5 0.8 33/24 56.25 0.2 5.0


5 1.0 52/35 63.0 0 5.0


5 0.6 14/12 22.3 0.4 5.0

53. Reconcile patient’s active problem list

5 1.0 15/15 21.8 0 5.0

54. Reconcile patient’s allergy list

5 1.0 25/25 13.8 5 5.0

The results from the SUS (System Usability Scale) scored the subjective satisfaction with the

Page 19 of 37

system based on performance with these tasks to be: 96%. Broadly interpreted, scores under

60 represent systems with poor usability; scores over 80 would be considered above average.

DISCUSSION OF THE FINDINGS

EFFECTIVENESS

Based on the success, failure and path deviation data the following items were unable to be

completed by some participants.

18. Change allergy from Carbamazepine to Codeine



EFFICIENCY

Based on the observations of the task time and deviation data the following items took an

average of over 60 seconds to complete:


11. Enter a prescription for Hydrocodone to view DUR Warning for drug-allergy interaction for prescriber

William Mayfield


SATISFACTION

Based on the task ratings and SUS results data the following items received a less than perfect

rating. An average score of 4.8 for the entire system relates to a relative 96% user satisfaction

rating which is well above average. Broadly interpreted, scores under 60 represent systems with

poor usability; scores over 80 would be considered above average.

The following items had an average user rating of 4.4:


The following items had an average user rating of 4.8:


Page 20 of 37



William Mayfield

12. Enter a prescription for Phenelzine to view DUR warning for drug-drug interaction for prescriber William

Mayfield



MAJOR FINDINGS

There were a few key areas that were unable to be completed.

18. Change allergy from Carbamazepine to Codeine



AREAS FOR IMPROVEMENT

Based on the observations of the task time and deviation data the following items took an

average of over 60 seconds to complete:



William Mayfield


Page 21 of 37

APPENDICES

The following appendices include supplemental data for this usability test report. Following is a list

of the appendices provided:

1: Participant demographics 2: System Usability Scale Questionnaire

Page 22 of 37

Appendix 1: PARTICIPANT DEMOGRAPHICS

The report should contain a breakdown of the key participant demographics. A representative list is

shown below.

Following is a high-level overview of the participants in this study.

Gender

Men 1 Women 4 Total (participants) 5

Occupation/Role

RN/BSN 2 Physician 1 Admin Staff 2 Total (participants) 5

Years of Experience

Years experience 19 Facility Use of EHR All paper 0 Some paper, some electronic

0

All electronic 5 Total (participants) 5

Page 23 of 37

Appendix 2: SYSTEM USABILITY SCALE QUESTIONNAIRE

Orientation (5 minutes)

Thank you for participating in this study. Our session today will last 60 minutes.

During that time you will take a look at an electronic health record system.

I will ask you to complete a few tasks using this system and answer some questions. We

are interested in how easy (or how difficult) this system is to use, what in it would be

useful to you, and how we could improve it. You will be asked to complete these tasks on

your own trying to do them as quickly as possible with the fewest possible errors or

deviations. Do not do anything more than asked. If you get lost or have difficulty I cannot

answer help you with anything to do with the system itself. Please save your detailed

comments until the end of a task or the end of the session as a whole when we can

discuss freely.

The product you will be using today is Cyfluent, Version 3.0.

We are recording the audio and screenshots of our session today. All of the information

that you provide will be kept confidential and your name will not be associated with your

comments at any time.

Do you have any questions or concerns?

Preliminary Questions (5 minutes)

What is your job title / appointment?

How long have you been working in this role?

What are some of your main responsibilities?

Tell me about your experience with electronic health records.

Page 24 of 37

User User 1 User 2 User 3 User 4 User 5

Do you have any questions or concerns?

No

No No No No

What is your job title/appointment? Precertification Office Assistant

Assistant Office Manager and Medical Coder

Registered Nurse and Office Manager

Physician/Owner Certified Ophthalmic Technician

How long have you been working in this role?

10 Years

14 Years 20 Years 35 Years Since 1996 17

What are some of your main responsibilities?

Training Electronic Medical Record, precertification.

Surgical report coding, data entry into Cyfluent EHR, data entry into ProtoMED for billing services.

Besides doing patient care I also help with the billing, all the EOBs, referrals. I manage the office and staffing.

I run the whole show.

I work patients up for the doctors and handle all the EHR implementation and training.

Page 25 of 37


Tell me about your experience with electronic health records.

A little rocky to start, but now it’s going well.

This is my first experience with Cyfluent. We started in July.

We’ve been using an Electronic Health Record since August 2004 and we just upgraded our system.

Well I’ve actually had 3 different programs over the years. A company in Canada they said would integrate with any program I had. Because of proprietary reasons the billing would not open up the program for the Electronic Health Record without charging me about $20,000 to run an interface so that came and went. And then the second company I bought had a business that got picked up and I didn’t want to convert so I looked around and found Protomed and then they had a problem with the EMR people and actually got bought out so they brought up their own company Cyfluent and I signed on with them.

Currently using Cyfluent and have used one previously.

Page 26 of 37


This is the application you will be working with. Have you heard of it?

Yes Started it in July. Yes. Yes. Yes.

If so, tell me what you know about it It has everything that we need and it is set up now as our paper charts used to be.

I know who they are affiliated with for billing and really that’s all I know about it.

I know that it’s an electronic record and it’s pretty easy to use.

I’ve been using it in my own office for several years.

I’ve been currently using it.

Page 27 of 37


Please don’t click on anything just yet. What do you notice? What are you able to do here? Please be specific.

Right away you see the patient’s name and date of birth. It has everything that is very important to us… prescriptions, allergies, everything that is most important is on the top right side in patient alerts which are some of the most important things in our office which would tell us any alerts the patient might have.

The first thing I notice is I always go to the patient’s name and picture and my eyes are always drawn to the prescriptions first on the left side. The second thing I notice is I go to patient alerts.

I notice that it’s a summary of the patient’s chart. It has the patient’s name, birthdate, prescriptions, problems list, and social history. It has everything I need to get started.

Well I can go into prescriptions, allergies, problems, social history, family history, insurance, patient health history. I can look at it all and review everything that is there. I can go into documents and pull in documents that I’ve scanned in to the patient’s chart. Obviously, this is patient alerts. I can enter patient contacts. It shows my encounters past and current. It shows vitals and I can have my staff put that in and body measurements and of course that’s necessary for meaningful use which is so important for us physicians so we can get our 1% kicker and we don’t get dinged 2% by Medicare. There’s obviously a lot more here than I use on an everyday basis but that other doctors might use. We can change the position of these sections like

I am able to go into the patient’s complete health record medication, allergies, problems list, social history etc. and view encounter for the day.

Page 28 of 37


Enter a REPORTED medication of AVANDIA 2 MG

8 steps

8 steps 8 steps 8 steps 8 steps

Task Time 44 seconds 57 seconds 120 seconds 45 seconds 24 seconds

On a scale of 1 to 5 with 1 being very difficult and 5 being very easy how would you rate this task?

5

4 5 5 5

Change the REPORTED medication from AVANDIA 2 MG to AVANDIA 4 MG via Emdeon Rx

9 steps




5

4 5 4 4

Access the AVANDIA 4 MG medication via Emdeon Rx

1 step

1 step 1 step 1 step 1 step



5

5 5 5 5

Open the most recent encounter for patient

3 steps

3 steps 3 steps 3 steps 1 step


On a scale of 1 to 5 with 1 being very difficult and 5 being very easy how would you rate this task

5

5 5 5 5

Enter a lab order of CREATININE 24 HOUR

4 steps




5

5 5 5 5

Edit the lab order of CREATININE 24 HOURS and ADD RESULTS VALUE OF 8

4 steps


Task Time 20 seconds 59 Seconds 38 seconds 32 seconds 20 seconds

Page 29 of 37



5 5 5 5 5

Access the lab order of CREATININE 24 HOURS

2 steps 2 steps 2 steps 4 steps 2 steps



5 4 5 5 5

Enter a radiology order of CHEST X-RAY via Assessment/Plan

5 steps 5 steps 5 steps 1 step 5 steps



5 5 5 5 5

Edit a radiology order of Chest X-Ray and assign to Medical Assistant




5 5 5 5 5

Access the CHEST X-RAY radiology order




5 5 5 5 5

Enter a prescription for HYDROCODONE to view DUR WARNING for drug-allergy interaction for prescriber William Mayfield




5 4 5 5 5

Page 30 of 37


Enter a prescription for PHENELZINE to view DUR warning for drug-drug interaction for prescriber William Mayfield




5 5 5 5 4

Adjust the DUR preferences for William Mayfield




5 5 5 5 5

Enter a REPORTED medication to SIMVASTATIN 10 MG




5 4 5 5 5

Change REPORTED medication from SIMVASTATIN 10 MG to REPORTED SIMVASTATIN 20 MG




5 4 5 5 5

Access SIMVASTATIN 20 MG reported medication

1 step 1 step 3 steps 2 steps 3 steps



5 5 5 5 5

Record ALLERGY to CARBAMAZEPINE with REACTION of SKIN RASH



Page 31 of 37



5 5 5 5 5

Change ALLERGY from CARBAMAZEPINE to CODEINE

3 steps 3 steps 3 steps – fail 4 steps 4 steps

Task Time 17 seconds 9 seconds 23 seconds 33 steps 7 seconds


5

5 5 5 5

Access CODEINE Allergy 1 step

1 step 1 step 1 step 1 step



5

5 5 5 5

Acknowledge Avandia Alert 3 steps

3 steps 3 steps

3 steps 3 steps

Task Time 14 seconds 10 seconds 22 seconds 16 steps 5 seconds


5 5 5

5 5

Acknowledge Fasting Blood Glucose Alert

3 steps 3 steps 3 steps

3 steps 3 steps



5 5 5 5 5

Open Therapeutic Reference Information for Diabetes Alert




5 5 5

5 5

CREATE a gender based alert in administration


6 steps – FAIL 6 steps


Page 32 of 37



5 5 5 5 5

Issue a prescription for HYDROCHLOROTHIAZIDE 50 MG for provider WILLIAM MAYFIELD.




5 5 5 5 5

RECONCILE patient’s active MEDICATION list

4 steps 4 steps 2 steps – FAIL 6 steps – FAIL 6 steps



5 5 5 5 5

RECONCILE patient’s active PROBLEM list


3 steps 3 steps



5 5 5 5 5

RECONCILE patient’s ALLERGY list 3 steps 3 steps 3 steps 3 steps 3 steps



5 5 5 5 5

Overall Impressions/Comments: None No, I’m happy with it.

Only that the system is pretty easy to use, everything is laid out easy and it’s in a nice order.

No, not really No

The Language of Electronic Healthcare

3737 Columbia Pike

Arlington, VA 22204

www.cyfluent.com

877.266.6016

Are you Cyfluent?

Date: 12/9/2013 To: Drummond Group Re: Cyfluent MU2 Testing – Audit Logs

Cyfluent does not allow users direct access to audit logs – audit logs cannot be altered by end-users.

Direct access to all data including audit logs is prohibited outside of the protected Cyfluent Chart application. Access to the Cyfluent application is controlled via userid, password, and facilityID combination over an HTTPS connection. The only access end-users can have to the Cyfluent audit logs is from within the User Interface (UI), specifically ADMIN access for queries against the audit files. NO USER CAN ALTER AUDIT LOGS. NO USER CAN GAIN ACCESS TO THE NATIVE RECORDS.

Direct access to audit logs can be made by qualified engineers, but even in this case, the logs are protected and audited by triggers that will provide information regarding access to specific records.

Regards,

Lawrence R. Walsh President, Cyfluent Inc.

Arlington, VA Towson, MD San Antonio, TX St. Petersburg, FL Taipai, Taiwan Shanghai, China

Documents

ONC HIT Certification Program Test Results Summary for ... · 410-823-2201 Developer/Vendor Contact: Larry Walsh. Test Results Summary for 2014 Edition EHR Certification Version EHR-Test-144