DATA AND ANALYTICS PLATFORMS 2021

ADVANCED USER INSIGHTS: A FIRST LOOK AT DEEP ADOPTERS

DATA AND ANALYTICS

PLATFORMS 2021

Performance Report | September 2021

1

TABLE OF CONTENTS

Executive Insights

Vendor Insights

Customer Interview Details

CernerDimensional InsightEpicHealth Catalyst

Fully rated vendors

Limited data vendors

Vendors not rated

AlteryxArcadia.ioClearsenseInnovaccer

Information Builders

2

10

38

11141720

23262932

35

EXECUTIVE INSIGHTS

The need in healthcare for broader, deeper analytics has increased dramatically, and provider organizations are now looking for consolidated, end-to-end analytics platforms that offer a wide variety of capabilities. To validate what is possible with these solutions, KLAS gave vendors with broad analytics offerings the opportunity to identify their deepest platform adopters. We then interviewed three such organizations for each vendor and validated their adoption across the five pillars of a data and analytics platform (see framework below). To provide additional context as to how vendors perform, this report also includes customer experience data collected from the broader base of platform users.

DATA AND ANALYTICS PLATFORMS 2021

A FIRST LOOK AT DEEP ADOPTERS

The framework below outlines the five pillars of a data and analytics platform as well as the basic and advanced capabilities included within each pillar. See the full report for a detailed definition of each capability as well as a vendor-specific look at validated adoption.

Basic capabilities Advanced capabilities

Percent of deep adopters leveraging capability

(n=27 deep adopters)

21%–40% 1%–20% 41%–60% 61%–80% 81%–100%

Data ingestion Functionality for compiling data from disparate sources.

Clinical data

Financial and revenue cycle management data (including claims data)

Semi-structured or unstructured data

Operational data

Data management Process of organizing and managing data to ensure it is reliable, accessible, and timely.

Comprehensive ETL/ELT

Data stewardship workflows

Data integrity and quality

Master patient index (MPI)

Data lineage

Longitudinal patient view

Metadata, data catalog, and task

managementData life cycle management

Healthcare data models

Analytics Front-end, graphical, visual displays of various types of data (clinical, financial, operational) that help inform decision-making specific to an end user’s role.

Dashboard

All visualization tools native

to application

Self-service analytics

Reporting (distribution)

Embedded content

Automated analytics workflows

Prebuilt healthcare applications

Cohort analysis

Advanced analytics Includes at the basic level predictive analytics and at the advanced level machine learning/NLP (supervised and unsupervised learning); can include prescriptive outputs. Customer adoption and vendor development are both still early.

Predictive analytics

Natural language processing (NLP)

Automated model building and training

Supervised machine learning algorithms

Embedded outside data language

Prescriptive analytics

Unsupervised machine learning

Geospatial analytics

Underlying components Functional capabilities that allow healthcare organizations to secure, interact with, and operationalize data within the platform.

FHIR enabled

Security controls for access at data level

Security of patient data

Platform ROI calculationAPI enabled

Analytics process automation

Role-based security

Data masking, anonymization, de-identificationInteroperability

Operationalization of analytics

Security controls for access at application level

Data and Analytics Platform Framework

Pillar average

83%

79%

73%

32%

62%

Capability Validation from Deep Adopters

Cerner’s deep adopters validate adoption of 100% of the basic platform capabilities and 86% of advanced capabilities (not validated are supervised machine learning, NLP, and system ROI calculation). Customers highlight Cerner’s data-ingestion capabilities and services and describe them as a standout strength for the vendor. Deep adopters report deeper advanced analytics adoption than some other customer bases, with most leveraging predictive, prescriptive, and geospatial analytics.

100% of the basic capabilities have been adopted by at least one of Epic’s deep adopters; among the advanced

For each measured platform, three deep adopter organizations were asked to validate their use of the various data and analytics capabilities in the platform framework. These organizations were identified by their vendors as some of their most progressive and comprehensive platform users, and their early insights are intended to help peers understand what is possible at the cutting edge of this technology. Readers should note that these customers likely experience deeper adoption and partnership than a vendor’s broader customer base.

Cerner, Epic & Health Catalyst See Deepest Adoption among Established Analytics Solutions

capabilities, only NLP and system ROI calculation have not been validated. Though all deep adopters use Epic for predictive analytics, not all leverage Epic’s machine learning, partially due to difficulty integrating non-Epic data and to a lack of APIs to connect to other databases.

All three of Health Catalyst’s deep adopters leverage the vendor to aggregate clinical, financial, and operational data, and they report broad adoption of the advanced platform capabilities. Data management areas—including data stewardship workflows, data lineage, and data life cycle management—see lower adoption compared to other vendors. Two deep adopters say the solution’s prebuilt healthcare applications add great value.

Deep adopters of Dimensional Insight report particularly strong adoption of data aggregation, data management, and analytics capabilities. Adoption of advanced analytics is low compared to the

other established solutions—no deep adopters validate using supervised ML, unsupervised ML, or prescriptive analytics. Areas for improvement include more visualization and, for larger clients, the ability to handle bigger data sets.

Depth of adoption for Information Builders (who was recently acquired by TIBCO) is lower compared to that of other long-term players. Adoption of data management capabilities is deep, but adoption in the remaining pillars is below average, particularly in advanced analytics—no clients validate use of ML or NLP, and only one client reports use of predictive analytics. Outside of the three deep adopters, no additional customers responded, meaning Information Builders does not meet the data threshold required for inclusion in the “Performance Insights” section.

Depth of Adoption

Data ingestion

Data management

Analytics

Advanced analyticsUnderlying components

Average depth across pillars

Note: Depth of adoption measures how consistently the platform capabilities have been deployed across all interviewed deep adopters. Vendors may offer and may have deployed capabilities not validated by the deep adopters interviewed for this report.

† Vendor partners with third party for some data visualization and/or reporting.

Vendors ordered alphabetically

Average percent of capabilities within each pillar leveraged by deep adopters 21%–40% 1%–20% 41%–60% 61%–80% 81%–100%

Alteryx (n=3)

Arcadia.io (n=3)

Cerner (n=3)

Clearsense (n=3)

Dimensional Insight (n=3)

(n=3)Epic

(n=3)Health Catalyst

(n=3)Information Builders

(n=3)Innovaccer

†

†

†

†

Among Newcomers, Innovaccer Has Deepest AdoptionCompared to the more longstanding solutions above, four data platforms have more recently emerged as viable options. Innovaccer and Arcadia.io have expanded their data sources and capabilities beyond population health management (PHM) and value-based care (VBC) to create broader analytics offerings. Alteryx, a cross-industry analytics automation platform vendor, has increased their healthcare focus in recent years. Starting from a data archiving background, Clearsense is beefing up their capabilities to include analytics.

Deep adopters of Innovaccer report high adoption across all key pillars except advanced analytics. Compared to other customer bases, Innovaccer clients report deeper adoption of underlying components—all interviewed deep adopters are using FHIR, APIs, analytics process automation, operationalization of analytics, and security features. Deep adopters report adoption of 100% of the basic capabilities. They note that in addition to InData—the platform’s foundation—there are various PHM applications that help bring meaningful insights to the point of care, regardless of EMR.

Arcadia.io’s interviewed deep adopters—two large clinics and one ACO—report adoption of 100% of the basic capabilities. Across the pillars, use of advanced capabilities isn’t as deep, especially for machine learning. Customers highlight the ability to ingest data from all sources, especially clinical and claims sources.

Alteryx customers highlight several unique features, including the solution’s ability to automate workflows, perform many SQL functions, and enable the use of multiple programming languages

Performance Insights from Platform Users

Partnering, Value Consistently Generate High Satisfaction with Dimensional Insight; Cerner Has Great Potential but Overpromises

To provide helpful context to the validation data shared above, the sections below provide customer experience insights collected from organizations within each vendor’s broader base of platform users (including deep adopters).

Overall Performance vs. Depth of Adoption

Average depth of adoption (all pillars) (n=24)

50.0

60.0

70.0

80.0

90.0

100.0

50% 60% 70%30% 80%40% 90% 100%

Overall performance score (100-point scale) (n=143)

Epic

Health Catalyst

Arcadia.io

Alteryx

Dimensional Insight

Cerner

Clearsense

Market average

Innovaccer

Note: Information Builders not charted due to insufficient data.

Primarily ambulatory-based ACOs and CINs

Limited dataMedian bed size

501–1,000

1,001–1,500

1,501+

1–500

in the same platform. Only one of the three interviewed deep adopters reports leveraging the reporting and visualization capabilities; the other two feel the tools are not sufficiently mature.

Validated depth of adoption is lower among Clearsense’s deep adopters. The system’s ability to ingest, curate, and harmonize data from various sources is seen as a top feature. The reporting/dashboards and advanced analytics are still in development, though clients are optimistic about leveraging the ML technology more for prediction.

Among Newcomers, Alteryx Leads in High-Focus Areas of Functionality and OutcomesNot surprisingly, in emerging markets like data and analytics platforms, provider organizations are eager for solutions that can keep up with their evolving functionality needs and drive tangible outcomes. Alteryx does well in both areas, with customers specifically highlighting that the software allows the use of multiple programs and languages and can be used enterprise-wide. Several customers report achieving high value with Alteryx because the solution is scalable, eliminates the need for some third-party tools, and helps drive tangible outcomes by improving staff efficiency. Alteryx’s AI capabilities receive mixed reviews, and the reporting and dashboards don’t meet all needs.

The remaining three vendors with limited data do fairly well overall, but each has at least one highly dissatisfied customer, indicating the need for these growing companies to deliver more consistently across organizations.

Satisfied Clearsense customers describe the vendor’s personnel as smart, innovative, and trustworthy, saying their flexibility in meeting customers’ specific needs has driven positive outcomes. Clearsense is newer to the market but has partnered with some very large organizations to test and develop their capabilities. Organizations feel the vendor’s tools and process are still maturing and lack certain capabilities, such as data science, reporting, and dashboarding.

Some Arcadia clients highlight the vendor’s data experts as a value-add that drives outcomes: these experts are able to help solve data structure issues and help customers understand how to use the data. Despite the platform’s data integration capabilities, a few customers report some functionality concerns, including data quality issues, the cost to connect different platforms, and the data connectors taking longer than promised to build. While Arcadia provides visualization and reporting through their services, customers would like more robust capabilities to create these things themselves.

Innovaccer customers say the prebuilt applications and templates with healthcare content are of great value in driving outcomes. Data ingestion and automation of integration workflows are key strengths. A commonly cited weak spot is the need for more predictive analytics. Innovaccer is the only vendor with an offshore support model, and customer feedback is mixed.

Cerner | Despite continued functionality development (e.g., Snowflake and enhanced security features) and high energy among deep adopters, struggles to deliver consistently, and 40% of respondents report overall dissatisfaction. While some frustration stems from product quality, functionality, or support issues, the more common complaint is overpromising on timelines and features, leading 47% to say Cerner does not keep promises. There is some optimism that Cerner is starting to listen to customer concerns and make needed changes. Early users of Snowflake (a cloud database platform) are complimentary of the added speed and process power.

Dimensional Insight | Customer base composed mostly of midsize to small organizations; a handful of larger customers have been validated. Generates high loyalty and excitement with customer-centric approach; all respondents are satisfied or highly satisfied. Seen as a partner who keeps promises and seeks to understand customer needs. Integration is a strength. Solution is seen as a great value, especially for smaller organizations. Many customers would like better, more specific training to mitigate what can be a steep learning curve.

Epic | Customers consistently report positive experience and note that while functionality may be limited, Epic doesn’t oversell what the system can do. Not surprisingly, highlighted benefits include integrated analytics tools and system consolidation. Viewed as a partner, with support personnel described as smart, quick learners. Customers report receiving more features and models and are hopeful Epic will better integrate external data. Due to some immature tools, solution is not seen as up to par with some other offerings. A handful of customers note that Epic takes longer than expected to incorporate development requests.

Health Catalyst | Has large number of customers. Highlighted for strong focus on understanding customer needs to better achieve desired outcomes. The solution and Health Catalyst’s consultants drive results by integrating disparate data to produce actionable insights. A highly engaged partner that fosters a strong support culture and is seen as having a vested interest in customer success. Several interviewed customers are struggling overall—common concerns include cost, unmet timelines, and complex, disorganized implementations.

Drives Tangible Outcomes and Product Has Needed Functionality(1–9 scale) Vendors ordered by drives tangible outcomes

5.0 9.0

*Limited data

Note: Information Builders not charted due to insufficient data.

8.3Dimensional Insight (n=20) (n=23)

Epic (n=16) (n=17)

Health Catalyst (n=30) (n=30)

Cerner (n=16) (n=17)

8.4

Alteryx (n=6) (n=6)

Arcadia.io (n=8) (n=10)

Clearsense (n=6) 7.2*Insufficient data to chart drives tangible outcomes

Innovaccer (n=8) (n=9) 7.5* 7.6*

Market average for drives tangible outcomes

Market average for product has needed functionality

8.0

7.8

7.4

7.6

7.2

6.8

8.7*7.7*

7.6*7.3*

This material is copyrighted. Any organization gaining unauthorized access to this report will be liable to compensate KLAS for the full retail price. Please see the KLAS DATA USE POLICY for information regarding use of this report. © 2021 KLAS Enterprises, LLC. All Rights Reserved.

REPORT INFORMATION

The insights in this report are based on two types of customer insights: (1) capability validations from deep adopters, and (2) performance insights from platform users (including deep adopters).

Capability Validation from Deep Adopters

For each measured platform, three deep adopter organizations were asked to validate their adoption of the various data and analytics capabilities in the platform framework. These organizations were identified by their vendors as some of their most progressive and comprehensive platform users, and their early insights are intended to help peers understand what is possible at the cutting edge of this technology. Readers should note that these customers likely experience deeper adoption and partnership than a vendor’s broader customer base.

Performance Insights from Platform Users

This report also includes customer experience data collected from organizations within each vendor’s broader base of platform users (including the interviewed deep adopters). The feedback was collected over the last 12 months via KLAS’ standard quantitative evaluation for healthcare software and is intended to provide helpful context to the validation data from deep adopters.

KLAS’ standard quantitative evaluation for healthcare software is composed of 16 numeric ratings questions and 4 yes/no questions, all weighted equally. Combined, the ratings for these questions make up the overall performance score, which is measured on a 100-point scale. The questions are organized into six customer experience pillars—culture, loyalty, operations, product, relationship, and value.

Sample Sizes

Sample sizes displayed throughout this report (e.g., n=16) represent the total number of unique customer organizations interviewed for a given vendor or solution. However, it should be noted that to allow for the representation of differing perspectives within any one customer organization, samples may include surveys from different individuals at the same organization. Ratings from these individuals are aggregated in order to prevent any one organization’s feedback from disproportionately impacting a solution’s score. The table below shows the total number of unique organizations interviewed for each vendor or solution as well as the total number of individual respondents.

Some respondents choose not to answer particular questions, meaning the sample size for any given vendor or solution can change from question to question. When the number of unique organization responses for a particular question is less than 15, the score for that question is marked with an asterisk (*) or otherwise designated as “limited data.” If the sample size is less than 6, no score is shown. Note that when a vendor has a low number of reporting sites, the possibility exists for KLAS scores to change significantly as new surveys are collected.

About This Report

Customer Experience Pillars

Category

Standard software evaluation metrics

Quality of training

Quality of implementation

Ease of use

Operations

Money’s worth

Avoids nickel-and-diming

Drives tangible outcomes

Value

Would you buy again

Part of long-term plans

Forecasted satisfaction

Overall satisfaction

Likely to recommend

LoyaltyCulture

Proactive service

Keeps all promises

Product works as promoted

Product quality

Product has needed functionality

Supports integration goals

Delivery of new technology

Product

Quality of phone/web support

Executive involvement

Relationship

Customer Base Estimates

Alteryx

Arcadia.io

Cerner

Clearsense

Dimensional Insight

Epic

Health Catalyst


Innovaccer

Estimates exclude non-healthcare customer organizations

Estimated # of healthcare analytics customers

Large Midsize Small

Estimated # of platform users (including deep adopters)

31–40 21–30 11–20 6–10

Estimated # of deep platform adopters

7–10 5–7 3–5

Deep Adopter Interviews

Standard Evaluations†

# of unique organizations

# of individual respondents

Alteryx Analytic Process Automation (APA) Platform 3 6 6

Arcadia.io Data Platform 3 10 10

Cerner HealtheIntent Analytics 3 17 22

Clearsense Data Platform-as-a-Service 3 6 6

Dimensional Insight Diver Platform 3 23 28

Epic Cogito (Epic Only) 3 17 19

Health Catalyst Analytics Platform 3 32 41

Innovaccer Data Platform 3 9 11

Other Validated Vendors

Information Builders, a TIBCO Company, Omni-Health Data 3 3 3

Note: Some organizations may have rated more than one product.

† Respondents to the standard evaluation were randomly selected from each vendor’s pool of platform users (second line in graphic below).

To be measured in this segment, vendors must meet the following criteria:

• KLAS must have validated adoption of all basic capabilities in the data aggregation, data management, and analytics pillars. In addition, KLAS must have validated adoption of at least some capabilities in at least one of the remaining two pillars (advanced analytics and underlying components).

• KLAS must have validated at least six live customers using the solution in a platform capacity (i.e., customer has adopted at least some capabilities within each of the five pillars).

Criteria for Measurement in Data and Analytics Platforms Segment

AnalystSam Eaquinto

[email protected]

WriterElizabeth Pew

[email protected]

Reader ResponsibilityKLAS data and reports are a compilation of research gathered from websites, healthcare industry reports, interviews with healthcare, payer, and employer organization executives and managers, and interviews with vendor and consultant organizations. Data gathered from these sources includes strong opinions (which should not be interpreted as actual facts) reflecting the emotion of exceptional success and, at times, failure. The information is intended solely as a catalyst for a more meaningful and effective investigation on your organization’s part and is not intended, nor should it be used, to

Copyright Infringement WarningThis report and its contents are copyright-protected works and are intended solely for your organization. Any other organization, consultant, investment company, or vendor enabling or obtaining unauthorized access to this report will be liable for all damages associated with copyright infringement, which may include the full price of the report and/or attorney fees. For information regarding your specific obligations, please refer to klasresearch.com/data-use-policy.

NotePerformance scores may change significantly when additional organizations are interviewed, especially when the existing sample size is limited, as in an emerging market with a small number of live clients.

365 S. Garden Grove Lane, Suite 300Pleasant Grove, UT 84062

Ph: (800) 920-4109

Cover image: ©insta_photos / Adobe Stock

Our MissionImproving the world’s healthcare through collaboration, insights, and transparency.

For more information about KLAS, please visit our website: www.KLASresearch.com

AuthorRyan Pretnik

[email protected]

AuthorBradley Hunter

[email protected]

AuthorLois Krotz

[email protected]

Project ManagerMary Bentley

[email protected]

DesignerNatalie Jamison

[email protected]

replace your organization’s due diligence.

KLAS data and reports represent the combined candid opinions of actual people from healthcare, payer, and employer organizations regarding how their vendors, products, and/or services perform against their organization’s objectives and expectations. The findings presented are not meant to be conclusive data for an entire client base. Significant variables—including a respondent’s role within their organization as well as the organization’s type (rural, teaching, specialty, etc.), size, objectives, depth/breadth of software use, software version, and system infrastructure/network—impact opinions and preclude an exact apples-to-apples comparison or a finely tuned statistical analysis.

KLAS makes significant effort to identify all organizations within a vendor’s customer base so that KLAS scores are based on a representative random sample. However, since not all vendors share complete customer lists and some customers decline to participate, KLAS cannot claim a random representative sample for each solution. Therefore, while KLAS scores should be interpreted as KLAS’s best effort to quantify the customer experience for each solution measured, they may contain both quantifiable and unidentifiable variation.

We encourage our clients, friends, and partners using KLAS research data to take into account these variables as they include KLAS data with their own due diligence. For frequently asked questions about KLAS methodology, please refer to klasresearch.com/faq.

mailto:sam.eaquinto%40KLASresearch.com?subject=

mailto:elizabeth.pew%40KLASresearch.com?subject=

http://klasresearch.com/data-use-policy

http://www.KLASresearch.com

mailto:ryan.pretnik%40KLASresearch.com?subject=

mailto:bradley.hunter%40KLASresearch.com?subject=

mailto:lois.krotz%40KLASresearch.com?subject=

mailto:mary.bentley%40KLASresearch.com?subject=

mailto:natalie.jamison%40KLASresearch.com?subject=

http://klasresearch.com/faq

VENDORINSIGHTS

11

VENDOR INSIGHTS

Figure 1

Fully Rated VendorsVendors listed alphabetically

Cerner HealtheIntent Analytics: 76.9

Customer Satisfaction Summary

C- B+ C+ C+ B- C-

Cerner HealtheIntent Customer Experience Pillars (n=17)

The overarching HealtheIntent platform is composed of multiple separate applications, including HealtheIntent Analytics, which is intended for more comprehensive analytics outside of population health management. Due to inconsistent delivery, customer satisfaction with Cerner is below the market average—overall, 40% of respondents are dissatisfied, i.e., they report an overall score below 70 (out of 100). While these customers report some frustration with things like product quality, functionality, or support, most feel the product is capable. The more common complaint is that Cerner has overpromised on delivery timelines and product features, leading almost half (47%) of respondents to say Cerner does not keep their promises. A chief analytics officer explained, “I don’t have a problem with Cerner’s vision. I have a problem with the fact that Cerner promises a January delivery and then we don’t see a prototype until July or August. . . . Cerner often overpromises the delivery time frame, but in general, Cerner keeps their promises in regard to functionality.” Additionally, a few customers describe the system as expensive (functionality beyond the basic capabilities costs extra) and feel the solution doesn’t live up to their expectations.

On a positive note, there is some optimism that Cerner is starting to listen to customer concerns and make needed changes. A CIO reported, “Cerner is improving and being more proactive. They have heard from their customers and have a lot more involvement from their team members in terms of working with customers and helping with training. If something is not working, then they change things. Cerner has been doing very well over the last several months. One problem was that when the HealtheIntent product came out, it was not completely baked, and things have been changing continuously. At the time, nobody from Cerner provided proper training or followed up. Now, Cerner knows to do that because HealtheIntent is a really good tool to have for data warehousing. I looked at it a few years ago, and it was the best solution at that time. Nobody else has a similar tool that could integrate so well with our Cerner clinical system. We don’t have to bring all the data from different sources.”

Cerner Customer Experience Pillars (n=17)

12

VENDOR INSIGHTS

95%

85%

53%

53%Avoids Nickel-and-Diming

Keeps All Promises

Part of Long-Term Plans

Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=17)Cerner HealtheIntent—Standard Yes/No Indicators

Cerner HealtheIntentMarket Average

Limited Data

Cerner

7.67.2

7.4

7.07.4

7.56.86.8

6.4

6.86.96.9

6.67.5

7.16.5

Forecasted Overall SatisfactionOverall Satisfaction

Likely to Recommend

Money's WorthDrives Tangible Outcomes

Supports Integration GoalsProduct Has Needed Functionality

Overall Product QualityDelivery of New Technology

Quality of TrainingQuality of Implementation

Ease of Use

Quality of Phone/Web SupportExecutive Involvement

Product Works as PromotedProactive Service

4.0 9.0

1–9 scale (n=17)Cerner HealtheIntent—Standard Numeric Indicators


Limited DataCustomer Experience Pillars

Culture

Relationships

Operations

Product

Value

Loyalty

95%

85%

53%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=17)Cerner HealtheIntent—Standard Yes/No Indicators


Limited Data

Figure 2

Figure 3

Cerner—Standard Numeric Indicators

Cerner

Cerner—Standard Yes/No Indicators

13

VENDOR INSIGHTS

Capability Validation

Deep Adopters Only100% of the basic functionality in the platform framework has been adopted by at least one of Cerner’s three interviewed deep adopters. Of the advanced functionality in the framework, 86% has been adopted, with the exceptions being NLP, system ROI calculation, and the ML platform. Cerner’s deep adopters report deeper advanced analytics adoption than some other vendors’ customer bases, with most leveraging predictive, prescriptive, and geospatial analytics.

Additional Capability Insights from All Platform Users (Deep Adopters & Standard Platform Users)Customers are complimentary of Cerner’s data-ingestion capabilities and services and describe them as one of the vendor’s standout strengths. Customers also highlight security improvements and the ability to connect to third-party tools. Cerner has started to roll out Snowflake (a cloud database platform), and early users have positive feedback regarding the added speed and processing power.

Figure 4 Cerner—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics†

Advanced analytics

Underlying components

75%

81%

71%

54%

67%

Overall average:

Comprehensive ETL/ELT 67%

Dashboard 100%

Predictive analytics 100%

FHIR enabled 33%

Clinical data 100%

Master patient index (MPI) 100%

Reporting (distribution) 100%

Supervised machine learning algorithms 0%

API enabled 100%

Healthcare data models 100%

Cohort analysis 67%

Geospatial analytics 67%

Security controls for access at application level 100%

Operationalization of analytics 67%

Financial and revenue cycle management data (including claims data) 67%

Metadata, data catalog, and task management 67%

Prebuilt healthcare applications 67%

Unsupervised machine learning 67%

Interoperability 67%

Data stewardship workflows 33%

All visualization tools native to application 33%

Natural language processing (NLP) 0%

Security controls for access at data level 100%

Platform ROI calculation 0%

Operational data 67%

Data integrity and quality 100%

Self-service analytics 100%

Automated model building and training 33%

Security of patient data 67%

Data lineage 100%

Embedded content 33%

Embedded outside data language 67%

Analytics process automation 67%

Data masking, anonymization, de-identification

†Vendor partners with third-party for some data visualization and/or reporting.

33%

Semi-structured or unstructured data 67%

Longitudinal patient view 100%

Automated analytics workflows 67%

Prescriptive analytics 100%

Role-based security 100%

Data life cycle management 67%

14

VENDOR INSIGHTS

Dimensional Insight Diver Platform: 93.1


Figure 5

A+ A A A A A+

Dimensional Insight Customer Experience Pillars (n=23)

Dimensional Insight’s customer-centric approach generates high loyalty and excitement among their customer base, which is composed mostly of midsize and small organizations. Most respondents (75%) are highly satisfied overall, and none report overall dissatisfaction. Dimensional Insight is seen as a vendor who partners with customers by keeping promises, seeking to understand customer needs, and being highly responsive to questions or concerns. A CIO noted, “I wish that every vendor were like Dimensional Insight. In my book, they are the best analytical and technology partner I could have. The people that I work with from the vendor always keep their promises. The product actually works the way the vendor promised. I have never run into a situation where they said the product would work a certain way and then it didn’t.” Customers, especially smaller organizations, report strong value, noting that the solution gives them capabilities they wouldn’t be able to afford otherwise.

Amidst the high praise, customers do highlight some areas for improvement. Some feel the solution’s data visualizations are starting to look outdated. Additionally, the system can be complex to learn, and many customers say they would like Dimensional Insight to provide better, more specific training and to either share more best practices themselves or facilitate the sharing of best practices between customers. A COO described some of the shortcomings with Dimensional Insight’s current training: “In terms of training, Dimensional Insight should use some very specific use cases that clients have and that are essential to their efforts in particular. Oftentimes, the initial training is not specific enough. The training shows the functionality, but I don’t think the information stays with users. The knowledge dissipates because it is not tied to something that users are going to replicate. Dimensional Insight needs to pick a few items that are core metrics for a business and use those as vehicles to train. That is what we ended up doing on the back end with some of the work that we have done. The important thing is how a vendor engages users and brings value in a clear, concise way rather than showing a piece of functionality once during a training program.”

15

VENDOR INSIGHTS

100%

100%

89%

92%

Avoids Nickel-and-Diming

Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=23)Dimensional Insight—Standard Yes/No Indicators

Dimensional InsightMarket Average

8.58.3

8.7

8.48.3

8.38.4

8.28.1

8.18.28.3

8.48.5

8.58.2


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=23)Dimensional Insight—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

100%

100%

89%

92%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=23)Dimensional Insight—Standard Yes/No Indicators


Figure 6

Figure 7

16

VENDOR INSIGHTS


Deep Adopters OnlyDeep adopters report strong adoption, especially in the data aggregation, data management, and analytics pillars. Adoption of advanced analytics is low compared to the other fully rated platforms in this report—no deep adopters validate using supervised ML, unsupervised ML, or prescriptive analytics.

Additional Capability Insights from All Platform Users (Deep Adopters & Standard Platform Users)Data aggregation is often hailed as one of Diver Platform’s biggest strengths. The solution is able to ingest real-time data from just about any source, enabling prompt decision-making, and end users are empowered to slice and dice the data, perform self-service analytics, and use the ETL tool. Areas for improvement include more visualization and, for larger clients, the ability to handle bigger data sets.

Figure 8 Dimensional Insight—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics

Advanced analytics


100%

96%

88%

17%

73%

Overall average:


Dashboard 100%


FHIR enabled 0%

Clinical data 100%




API enabled 0%


Cohort analysis 67%



















Data lineage 100%




Data masking, anonymization, de-identification 100%







17

VENDOR INSIGHTS

Epic Cogito (Epic Only): 90.1


Figure 9

A- A B B+ A A-

Epic Cogito (Epic Only) Customer Experience Pillars (n=17)

Epic customers consistently report a positive experience, with no interviewed customers reporting overall dissatisfaction. Customers note that while functionality may be limited, Epic doesn’t oversell what the system can do, and the solution is stable. Epic is viewed as a partner, and their support representatives are described as smart and quick learners. Customers are optimistic about recent improvements (they feel they are seeing more features and models), and they are hopeful Epic will be able to better integrate external data. An IT manager explained, “Things are getting better. The system really does work well. What doesn’t work is pretty limited. We have some hopes for the future to make it easier to integrate external data. The product is very stable.”

Amidst this optimism, customers note that the Epic solution has some immature tools and is not up to par with some other offerings. The user experience is not ideal, and the solution can be difficult to use. The ability to use APIs to connect to other databases and build apps is not sufficient, and it can be difficult to incorporate data from non-Epic sources. An IT manager described some of the gaps: “There are several things missing from the platform. It is pretty good at doing custom models or models that specifically involve clinical data. But we are starting to get to a point where we are doing models that involve supply chain data or other data that lives outside of Epic’s platform, and the platform isn’t very friendly to that data. Being able to incorporate nonclinical data would be helpful; then, we could use the Epic platform as a one-stop shop for advanced analytics. The platform is lacking there. Part of the issue is that it is fairly new. Epic is growing and building out the platform. I don’t know whether they have specific things on the road map. We have always gone with Epic, and we probably have 1% of our data outside Epic solutions.”

Epic Customer Experience Pillars (n=17)

18

VENDOR INSIGHTS

8.6

8.18.0

8.07.6

7.7

7.87.8

7.4

8.58.3

8.0

7.97.7

7.8

7.8


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=17)Epic Cogito (Epic Only)—Standard Numeric Indicators

Epic Cogito (Epic Only)Market Average


Culture

Relationships

Operations

Product

Value

Loyalty

100%

100%

97%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=17)Epic Cogito (Epic Only)—Standard Yes/No Indicators


Limited Data

100%

100%

97%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=17)Epic Cogito (Epic Only)—Standard Yes/No Indicators


Limited Data

Figure 10

Figure 11

Epic—Standard Numeric Indicators

Epic—Standard Yes/No Indicators

Epic

Epic

19

VENDOR INSIGHTS


Deep Adopters OnlyEpic’s deep adopters report using 100% of the basic functionality in the platform framework and 19 of the 21 advanced functionalities. Interviewed deep adopters have not yet adopted NLP or system ROI calculation. Even though all deep adopters use Epic for predictive analytics, not all leverage Epic’s machine learning; some other advanced analytics areas—e.g., automated ML and geospatial analytics—have even lower adoption.

Additional Capability Insights from All Platform Users (Deep Adopters & Standard Platform Users)Not surprisingly, benefits highlighted by Epic customers include having integrated analytics tools and being able to consolidate to fewer systems.

Figure 12 Epic—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics

Advanced analytics


75%

78%

88%

50%

73%

Overall average:


Dashboard 100%


FHIR enabled 33%

Clinical data 100%




API enabled 67%


Cohort analysis 67%



















Data lineage 100%











20

VENDOR INSIGHTS

Health Catalyst Analytics Platform: 86.2


Figure 13

B+ A- B- B A- B+

Health Catalyst Customer Experience Pillars (n=32)

Health Catalyst has a large number of customers, and most of those interviewed for this research are satisfied or highly satisfied. These customers highlight the vendor’s strong focus on understanding customer needs and helping organizations achieve their desired outcomes. They say the solution drives results by integrating disparate data and producing actionable insights. An executive director noted, “Health Catalyst is better than any of their competitors at driving outcomes. They really understand the needs of health systems, including care delivery, data and analytics, and support. The product provides actionable insights. It brings in multiple forms of disparate data and makes sense of that data.” Health Catalyst is known for being a highly engaged partner and fostering a strong support culture; as a result, customers feel the vendor has a vested interest in their success. Health Catalyst is also seen as a leader in delivering new technology. The same executive director stated, “Health Catalyst is also very aggressive about adding new functionality or acquisitions to help them keep pace with ever-changing technology and the healthcare industry.”

Despite the success of most clients, five interviewed Health Catalyst customers are struggling. A common complaint among these less satisfied customers is that implementations are complex and often take longer than expected. Additionally, some describe the solution as expensive, report a slow speed to value, or say the cost of the consulting services left them feeling nickel-and-dimed: “The consulting fees are more than we had hoped for. Once the system is up and running, then the costs for our Health Catalyst solutions should flow together. But the data operating system has been the painful part when it comes to the cost. In the sales pitch Health Catalyst made for the solution, they made it sound like things were mapped and using the platform would just require a minimal effort on our part because we pay the consulting fees. The reality is that they didn’t understand a lot of our financial data and data streams, so it took a lot more of my team’s effort to support that, and then the consulting hours increased. With the subscription license and many of the things that Health Catalyst is using for consulting hours, I would have expected to just be covered by our subscription license” (VP). Customer feedback on Health Catalyst’s training suggests it is hit or miss—some respondents would like more systematic ongoing training and documentation. A couple of non-hospital organizations report that Health Catalyst is not a good fit given that the vendor’s expertise is focused more on health systems.

21

VENDOR INSIGHTS

93%

87%

90%

87%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=32)Health Catalyst—Standard Yes/No Indicators

Health CatalystMarket Average

7.9

7.4

8.18.3

8.07.8

8.1

7.47.8

7.77.2

7.5

7.47.1

7.97.6


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=32)Health Catalyst—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

93%

87%

90%

87%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=32)Health Catalyst—Standard Yes/No Indicators


Figure 14

Figure 15

22

VENDOR INSIGHTS


Deep Adopters OnlyHealth Catalyst’s deep adopters report some of the broadest, deepest functionality adoption compared to other customer bases in this report. All three interviewed deep adopters are aggregating clinical, financial, and operational data. Data management areas—including data stewardship workflows, data lineage, and data life cycle management—see comparatively lower adoption.

Figure 16 Health Catalyst—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics†

Advanced analytics


92%

63%

88%

46%

73%

Overall average:


Dashboard 100%


FHIR enabled 33%

Clinical data 100%




API enabled 67%


Cohort analysis 100%








Interoperability 0%











Data lineage 33%












23

VENDOR INSIGHTS

Limited Data VendorsVendors listed alphabetically

Alteryx Analytic Process Automation (APA) Platform: 90.9*


Figure 17

A* A+* B* A-* B+* A-*

Alteryx Customer Experience Pillars (n=6)

A cross-industry vendor, Alteryx is widely used in conjunction with solutions such as Tableau or Microsoft Power BI. While healthcare is not a primary marketing focus for Alteryx, the solution is available to healthcare organizations. Alteryx is well known for their data management, data transformation, and data connectors, and the vendor is quickly moving into AI. Customers feel the product is high quality and easy for healthcare users to leverage. Several customers highlight value as a strength, saying the solution is very scalable, eliminates the need for some third-party tools, and helps drive tangible outcomes by improving staff efficiency. A senior data scientist reported, “Alteryx provides a tool to allow data scientists to do everything in-house. We brought development in-house, and we stopped paying a lot of money to third parties. We hit the economy of scale internally. That is a big difference compared to other vendors. . . . The product is cost effective. But we saw the greatest return with our analytics staff members that just knew Excel and were only trained on Excel. They were spending 30 hours a week doing data preparation in Excel, and they now only spend 2 hours a week. That is transformational. Just two weeks of training on the Alteryx system saves 28 hours of time per person in that position.”

Feedback regarding Alteryx’s developing AI capabilities is mixed—while customers appreciate that the data science language is embedded in the workflow, some note that the AI interface could be more intuitive to use. Alteryx’s solution can be used as a self-service tool to create dashboards or manipulate data before it is displayed in a dashboard; however, customers say Alteryx has less dashboard and reporting functionality than niche visualization and reporting tools do.

*Limited data

24

VENDOR INSIGHTS

100%

100%

100%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=6)Alteryx—Standard Yes/No Indicators

AlteryxMarket Average

Limited Data

8.38.58.5

8.08.7

8.27.7

8.08.2

8.0

8.3

8.3


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=6)Alteryx—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

100%

100%

100%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=6)Alteryx—Standard Yes/No Indicators


Limited Data

Figure 18

Figure 19

25

VENDOR INSIGHTS


Deep Adopters OnlyOne of Alteryx’s interviewed deep adopters validates leveraging most of the basic and advanced functionality in the platform framework. Customers validate that outside languages, such as R or Python, can be embedded in the workflow or accessed via an API link. Customers highlight several unique features, including the solution’s ability to automate workflows, perform many SQL functions, and enable the use of multiple programming languages in the same platform.

Figure 20 Alteryx—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics†

Advanced analytics


92%

63%

63%

42%

42%

Overall average:


Dashboard 33%


FHIR enabled 33%

Clinical data 100%




API enabled 67%





















Data lineage 67%












26

VENDOR INSIGHTS

Arcadia.io Data Platform: 86.3*


Figure 21

B-* A* B-* B* A-* A*

Arcadia.io Customer Experience Pillars (n=10)

In addition to their value-based care applications, Arcadia has been building a data platform layer that is scalable across organizations of different types and sizes. Customers highlight the Arcadia solution as user friendly and easy to navigate. A senior VP described using the solution to both process and analyze data: “Processing and analytics are very different. When I think of data processing, I think about why we chose Arcadia to ingest, normalize, validate, and group our data. That was one of the key decision factors. That is a functionality that we decided to buy instead of build because all of the data sources are ingested, normalized, validated, and grouped together. Analytics is completely different. Analytics functionality involves both front-end user interfaces and self-serve analytics, but then there is also a back-end database we use as well, so we are using both.” Overall, customers have a positive experience with the support, and some specifically highlight the value that Arcadia’s data experts bring in helping solve data structure issues and helping customers understand how to use the data in the platform.

Despite the solution’s data integration capabilities, a few customers note some issues, including data quality issues, the cost to connect different platforms, and the data connectors taking longer than promised to build. While Arcadia provides visualization and reporting through their services, customers would like more robust capabilities to create these things themselves. Clients note that the company is growing and that this sometimes results in resource issues and insufficient training. However, they are optimistic about recent personnel changes and Arcadia’s commitment to being a partner. A project manager reported, “My speculation is that Arcadia is a little resource strapped. They have had some good sales this year, and the process for issue resolution is broken. It tends to rely on email, and we have told them that stretches it out. We need to talk to the resources who are actually working on the problem so that we can have back-and-forth communication in real time as opposed to sending an email with an answer, waiting a week to get a response, and then going down the path.”

*Limited data

27

VENDOR INSIGHTS

100%

100%

89%

80%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=10)Arcadia.io—Standard Yes/No Indicators

Arcadia.ioMarket Average

Limited Data

8.47.9

8.1

8.07.7

8.1

8.0

7.6

7.37.3

7.87.5

6.0

8.1

7.56.9


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=10)Arcadia.io—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

100%

100%

89%

80%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=10)Arcadia.io—Standard Yes/No Indicators


Limited Data

Figure 22

Figure 23

28

VENDOR INSIGHTS


Deep Adopters OnlyInterviewed deep adopters report using 100% of the basic platform capabilities, and all three are aggregating clinical, financial, and operational data. Use of advanced functionality isn’t as deep, especially for machine learning functionalities. Two deep adopters note that the dashboard capabilities could be strengthened.

Additional Capability Insights from All Platform Users (Deep Adopters & Standard Platform Users)Customers highlight the ability to ingest data from all sources, especially clinical and claims sources.

Figure 24 Arcadia.io—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics

Advanced analytics


92%

70%

63%

33%

67%

Overall average:


Dashboard 100%


FHIR enabled 33%

Clinical data 100%




API enabled 33%


Cohort analysis 67%



















Data lineage 33%











29

VENDOR INSIGHTS

Clearsense Data Platform-as-a-Service: 85.8*


Figure 25

B* B+* B-* B* A* A*

Clearsense Customer Experience Pillars (n=6)

Many customers begin their relationship with Clearsense via the vendor’s data archiving work. The majority of interviewed analytics customers are satisfied or highly satisfied, with just one respondent reporting dissatisfaction.The more satisfied customers highlight the vendor’s strong relationships, culture of support, executive touch points, and adaptability in meeting customers’ specific needs. “Things have been great with Clearsense,” reported an IT director. “We have had them working on some use cases, and they have done really well. The customer relationship is fantastic. The product has worked really well. Generally speaking, it has been great to watch the vendor grow, learn, gain experience, and help us solve some challenges. Overall, our experience has been very positive. I have regular meetings with vendor executives. I like their frequent contact with us. The vendor’s project managers run regular meetings with us too. When we have challenges or a significant obstacle that we are trying to overcome, the vendor’s approach to managing those things is rapid and on point. Clearsense is a small company, but they make us feel like we are working with a very large company. That treatment is well received by us.” Clearsense personnel are described as smart and innovative. Clients trust the vendor and note that even though Clearsense is experiencing growing pains, they listen, learn from their mistakes, and then make improvements.

The Clearsense solution is newer to the market, and some customers feel it is not yet a full analytics platform. They cite immature tools and processes, with the most commonly requested development being more machine learning and NLP capabilities. A director stated, “One of the things that we are working with Clearsense on is a little bit of a better platform for our data scientists. While Clearsense’s system can take the genomics data, it looks like we would have to extract the genomics data and put it somewhere else to be able to use it. The data science tools are not available to us up there. That would be a feature that we would probably use if Clearsense could create a data scientists’ platform.” Clients also note some immaturity and growing pains within the company itself—customers would like the vendor to develop better training and a clearer pricing structure.

*Limited data

30

VENDOR INSIGHTS

100%

83%

83%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=6)Clearsense—Standard Yes/No Indicators

ClearsenseMarket Average

Limited Data

7.8

7.7

8.2

7.37.7

7.27.5

7.3

7.0


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=6)Clearsense—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

100%

83%

83%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=6)Clearsense—Standard Yes/No Indicators


Limited Data

Figure 26

Figure 27

31

VENDOR INSIGHTS


Deep Adopters OnlyDeep adopters report lower breadth and depth of adoption compared to customers of other vendors in this report. The system’s ability to ingest, curate, and harmonize data from various sources is seen as a top feature. The reporting/dashboards and advanced analytics are still in development, though clients are optimistic about leveraging the ML technology more for prediction. Clearsense has partnered with several large health systems for product development, though the deep adopters interviewed for this report have all started with Clearsense at the department level.

Figure 28 Clearsense—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics

Advanced analytics


58%

78%

46%

17%

27%

Overall average:


Dashboard 67%


FHIR enabled 0%

Clinical data 67%




API enabled 33%










Interoperability 0%











Data lineage 100%

Embedded content 0%










32

VENDOR INSIGHTS

Innovaccer Data Platform: 87.7*


The majority of interviewed Innovaccer customers are satisfied or highly satisfied, with just one respondent reporting dissatisfaction. Many customers describe the company’s executives as engaged and involved, and they say there is a good support system in place for problem resolution. They also highlight the company’s flexibility in meeting customer needs, as this director of analytics explained: “Our organization moves very quickly in terms of things that we like to push boundaries on. Innovaccer’s ability to partner and adjust their platform and workstreams to really help meet our operational needs is one of the big benefits that we have seen. We have looked at other vendors that are a little more structured, and Innovaccer is much quicker and more agile in terms of being able to meet our needs effectively. They also are very good partners because they want to allow us as much empowerment as we want. . . . They are able to turn the keys over for us to be able to do what we need to do, and they are more than willing to try to help offload things that we don’t need to have on our plates.” Besides developing the specific PHM applications that are part of the end-user workflow, Innovaccer has partnered with Microsoft (Azure) to build a data platform layer.

Feedback on Innovaccer’s offshore resources is mixed—some customers like how fast Innovaccer responds given the time difference, but others feel it is hard to get support during the daytime and would like 24/7 access to support individuals. A VP noted, “The vendor is not great at resolving support tickets. The help desk is in India, and we need a 24/7 solution. Innovaccer needs a presence in the United States. They need to blend their rates between the offshore and onshore team so that technical solution engineers are available to us during day hours in the United States.” Also, some feel that Innovaccer hasn’t staffed enough to support their company growth. A common weak spot reported by several customers is that they would like predictive analytics capabilities to be more fully integrated into the platform. Additionally, some report concerns regarding data quality, data integration, and reports and dashboards.

Figure 29

B+* A-* B* A-* A+* B+*

Innovaccer Customer Experience Pillars (n=9)

*Limited data

33

VENDOR INSIGHTS

91%

90%

91%

80%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=9)Innovaccer—Standard Yes/No Indicators

InnovaccerMarket Average

Limited Data

8.17.6

8.18.3

7.77.5

8.58.7

8.18.4

7.97.7

7.9

7.77.5

6.7


Likely to Recommend





Ease of Use



4.0 9.0

1–9 scale (n=9)Innovaccer—Standard Numeric Indicators



Culture

Relationships

Operations

Product

Value

Loyalty

91%

90%

91%

80%


Keeps All Promises


Would You Buy Again

0% 100%

Percent of respondents who answered yes (n=9)Innovaccer—Standard Yes/No Indicators


Limited Data

Figure 30

Figure 31

34

VENDOR INSIGHTS

Figure 32 Innovaccer—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics†

Advanced analytics


100%

93%

83%

13%

88%

Overall average:


Dashboard 100%


FHIR enabled 100%

Clinical data 100%




API enabled 100%





















Data lineage 100%












Deep Adopters OnlyInnovaccer’s deep adopters leverage 100% of the basic functionality areas and report high adoption of all key pillars except for advanced analytics—no interviewed deep adopters report adoption of machine learning, NLP, or API links for other data science languages. All three deep adopters are aggregating clinical, financial, and operational data, including semi-structured and unstructured data. Compared to other customer groups, Innovaccer customers have the deepest adoption of underlying components; all deep adopters leverage FHIR, API, analytics process automation, operationalization of analytics, and security features.

Additional Capability Insights from All Platform Users (Deep Adopters & Standard Platform Users)Customers note that in addition to InData—the platform’s foundation—there are various population health management applications that help bring meaningful insights to the point of care, regardless of EMR.


35

VENDOR INSIGHTS

Vendors Not RatedInformation Builders, a TIBCO Company Omni-HealthData: Insufficient DataCustomer Satisfaction Summary

Acquired by TIBCO in late 2020, Information Builders is a long-term player in healthcare analytics and offers the fairly comprehensive WebFOCUS product line as well as a newer, healthcare-specific solution, Omni-HealthData. The latter has approximately 6–8 customers, but it is not frequently considered and has seen few new wins.

While KLAS was able to interview three deep adopters of Omni-HealthData, no other customers responded, so performance ratings for Information Builders cannot be shown (they do not meet the required threshold of six unique interviewed organizations). However, it can be noted that satisfaction among the three deep adopters was variable, with two reporting overall satisfaction and one reporting strong dissatisfaction. Overall, customer satisfaction has remained stable through the early stages of the acquisition by TIBCO, and clients report they are taking a wait-and-see approach. A CIO reported, “I am hoping that TIBCO gets pretty active with their customers. TIBCO seems to have an approach of wanting to integrate the two companies rapidly, and I appreciate that. I would rather just rip off the Band-Aid, merge the teams, change email systems, and change HR systems. TIBCO is doing all that now. The downside is that the acquisition is very disruptive because it kind of jerks everyone around in a lot of different directions very quickly. I haven’t seen much from the TIBCO team yet. I am told that TIBCO’s people want to get out and meet some of their customers. I am told that they want to learn what we do and those types of things, but they aren’t there yet. They seem to know healthcare. I don’t know that they know what goes on in a provider organization or in some of the other spaces we are involved in.”


Deep Adopters OnlyDeep adopters of Omni-HealthData report lower depth of adoption than customers of other long-term players. Adoption of the data management capabilities is deep, but adoption in the remaining pillars is below average, particularly in advanced analytics—no clients validate use of ML or NLP, and only one client reported use of predictive analytics.

Figure 33 Information Builders—Depth of Adoption (n=3 deep adopters)

Data ingestion

Data management

Analytics

Advanced analytics


67%

93%

71%

21%

52%

Overall average:


Dashboard 100%


FHIR enabled 0%

Clinical data 100%




API enabled 67%


Cohort analysis 67%



















Data lineage 100%











36

VENDOR INSIGHTS

Definitions of Platform Capabilities

Data Ingestion

Clinical data Ingestion of EMR, telehealth, remote patient monitoring, and SDOH data, among other types.

Financial and revenue cycle management data (including claims data) Ingestion of ADP, RCM, and claims data, among other types.

Operational data Ingestion of ERP, supply chain, and other logistical data (e.g., weather), among other types.

Semi-structured or unstructured data All other types of data, e.g., open notes, audio files, images, blob, and audio/transcriptions.

Data Management

Comprehensive ETL/ELT Back-end SQL technology/techniques that allow IT or analysts to manipulate the data to create understanding and structure. Common elements must link the data sources together.

Master patient index (MPI) The master patient index identifies patients across separate clinical, financial, and administrative systems and is needed for information exchange to consolidate the patient list.

Metadata, data catalog, and task management

Descriptive data that gives information about other data in order to offer a unified understanding across the organization.

Data integrity and quality Ensuring overall usability of a data set so it can be easily processed and analyzed.

Longitudinal patient view Longitudinal patient records that incorporate information over time and across systems to provide a holistic view of a patient’s medical history.

Healthcare data models Predefined models based on a use case—for example, clinical, claims, operational, and cost.

Data stewardship workflows The formalization of accountability for data management and data-related resources.

Data lineage History of the journey data takes over time, from its creation through its transformation.

Data life cycle management Automated processes that organize data into policy-based tiers across the entire life of the data, from inception to deletion.

Analytics

Dashboard Single screen in which various critical pieces of information are placed in the form of panels.

Reporting (distribution) Ability to receive scheduled or ad hoc reports.

Prebuilt healthcare applications Prebuilt healthcare content that users can utilize (nearly) out of the box.

Self-service analytics Interactive visual exploration, data analysis and drill-down, and creation of parameters and reports/dashboards.

Automated analytics workflows Building and embedding of automated analytics workflows that feed and support visual analytics reporting platforms like Excel, Tableau, Power BI, and Qlik.

Cohort analysis Ability to group patients into similar categories based on relevant healthcare markers; users can manipulate data within a registry beyond creating a registry.

All visualization tools native to application Solution does not require third-party visualization tools.

Embedded content Ability to embed content in EMR or other solutions.

37

VENDOR INSIGHTS

Advanced Analytics

Predictive analytics Make predictions about unknown future events. Basic functionality includes techniques from data mining, statistics, modeling, etc. Advanced functionality includes machine learning to analyze current data to predict future events.

Supervised machine learning algorithms Use of labeled data sets to train algorithms to classify data or predict outcomes accurately.

Unsupervised machine learning Type of algorithm that allows the model to work on its own to discover previously undetected patterns and information.

Automated model building and training Automation of the building and training stages of the ML process.

Prescriptive analytics Ability to find the best course of action in a scenario given the available data.

Geospatial analyticsGeospatial analytics gathers, manipulates, and displays geographic information system (GIS) data and imagery, including GPS and satellite photographs. Used to create geographic models and data visualizations for more accurate modeling and predictions of trends, adding more context in the form of information about timing and location.

Natural language processing (NLP) Ability to understand, process, and analyze natural language (speech or text).

Embedded outside data language API link to embed outside data language (such as Python/R) for creating machine learning models

Underlying Components

FHIR enabled Ability to interface with FHIR APIs.

API enabled Ability to interface with standard APIs; can include general data/compute/terminology API.

Interoperability Ingestion of data sources outside the healthcare organization.

Security of patient data System can restrict patient data and who can access it (e.g., HIPAA compliant).

Role-based security Restrictions as to which roles can access particular data sets.

Security controls for access at application level Application-level control of access levels and which users can interact with the data.

Security controls for access at data level Control of access to data at the database level.

Analytics process automation Automation of analytics processes and reporting workflows via prebuilt drag-and-drop tools and apps to improve analyst and data scientist productivity.

Operationalization of analytics Ensuring users know how to leverage data.

Platform ROI calculation Providing the methodology to calculate the platform’s ROI.

Data masking, anonymization, de-identification Process used to shield confidential data or prevent someone’s personal identity from being revealed.

CUSTOMERINTERVIEW

DETAILS

39

CUSTOMER INTER VIEW DETAILS

Questions Asked in Supplemental EvaluationWhat specific functionality from this platform has made it easier for your organization to do data processing and analytics?

What specific functionality is lacking from this platform that would make it easier for your organization to do data processing and analytics?

Based on your own perception and definition, does your vendor provide a master data management (MDM) solution?

Are you using data ingestion capabilities from this vendor?

Are you using data management capabilities from this vendor?

Are you using analytics capabilities from this vendor?

Are you using advanced analytics capabilities from this vendor?

Are you using underlying data and analytics components from this vendor?

Additional comments

40


What specific functionality from this platform has made it easier for your organization to do data processing and analytics?Alteryx

We are happy with Alteryx. The main reason we purchased the Alteryx solution was it does a lot of SQL functions without requiring that we teach a bunch of people SQL. Folks use the solution as a tool to give out to other departments so that they can automate a lot of their workflows. Departments have been able to ingest whatever they want, bring it over into our enterprise data warehouse or the data lab, and then build whatever they want. That has been very valuable for us.

This software is unique because it allows multiple programming languages to be used in the same platform, and that is something we can’t get in other software programs. The solution was one of the first analytic tools that allowed us to use multiple programming languages in one space. What was novel with that is also the click-and-point interface. I can get Python or R to drive click-and-point functionality. We can also do SQL by connecting to a database SQL and then actually writing SQL from that. For our company, that allowed the Excel users to get off Excel and do basic analytics in the Alteryx system. It even allowed the data scientists to completely drive Python programming. Python is what opens users up to machine learning. The data science language is embedded in the workflow. That was what made the Alteryx software so unique. It immediately stood out from any other analytic products on the market. Alteryx found a way to integrate all of that seamlessly. What is neat is the scale. We are in a big data environment. A vitals table, which has just blood pressures, has a billion rows for just 12 months, and the Alteryx software has no problem hunting that, either in memory or at a server level. There is also no other desktop-level software I can think of that can crunch a billion rows with no problem and analyze that data. That is part of what immediately drove us to the Alteryx software. It could handle enterprise-wide functionality, and it had server-level products and automation that pairs with it.

For me and my team, it was very easy to learn to use Alteryx’s system. We are not SQL heavy; I can write a basic select statement, but I am not advanced in SQL. I don’t try to claim that I can write SQL, but I can use the Alteryx system very easily. It is a great platform for people who are very data driven and analytical and are not hardcore SQL writers, but using the solution is more advanced than trying to do things in Microsoft Excel or Microsoft Access. The system is really good for people who don’t want to take the time to learn how to write SQL.

Arcadia.io

Processing and analytics are very different. When I think of data processing, I think about why we chose Arcadia to ingest, normalize, validate, and group our data. That was one of the key decision factors. That is a functionality that we decided to buy instead of build because all of the data sources are ingested, normalized, validated, and grouped together. Analytics is completely different. Analytics functionality involves both front-end user interfaces and self-serve analytics, but then there is also a back-end database we use as well, so we are using both.

With Arcadia Analytics, we have the claims and clinical data together in one place. This is the first time that we have been able to see the data together. That is the biggest thing.

The supplemental data files are a good place to start to make things easier for data processing and analytics. We ingest data from EMRs into our data warehouse, and then we have a team that works with Arcadia.io to generate the necessary reporting to meet our contractual obligations. We are also working to improve reporting. As we get more and more practices connected to the Arcadia.io tool, the process will be streamlined. Currently, the system analytics team has to query several different EHRs to gather the data, and even then, the data only includes members who are seeing employed providers. Our supplemental data files aren’t as robust as they could be. Leveraging the data in Arcadia.io’s platform is going to allow us to have a clearer picture of our true performance, and we can include that in our supplemental files. That is really exciting. As a health system, we are requiring all primary care practices to be connected to the Arcadia.io tool. The capability is just going to grow and improve.

41


At the end of the day, Arcadia.io’s tool provides visibility. We have the data, and the tool works collaboratively with Arcadia.io’s foundation tool to enable us to leverage other inputs other than data from our internal EMR or employed medical groups. Doing that is very easy. We can easily access and manipulate the data. The interface is really functional on the front end. If we had built something internally, it wouldn’t have been as easy to use.

Cerner

HealtheAnalytics sits on top of the HealtheIntent data, which works off of AWS, but it is Vertica based. One key win is that we can import our own data and create our own data sets. We can take some of the big data tables that are provided to us and optimize that data to run specific reports. We can create APIs to export large amounts of data out of there for different purposes. We do not have to buy Tableau licenses for everyone; we just need some for our developers. Also, we have our own integrated reporting platform where we want everyone to go for a one-stop shop. We can integrate those reports into that platform via a single sign-on software. The amount of storage in the big data platform is a bonus. We get value-added data through Cerner. There are things that we don’t have to purchase that are included. We get all of the functionality from another Cerner product with HealtheEDW Advanced as well as a lot of enhanced security. Role-level security is available too. Also, we can connect to the system using any third-party tools. We don’t have to use just what is coming through on another Cerner product. That product covers query and data set tools. We can apply all kinds of different levels of security via the platform on Snowflake. Once someone has used HealtheEDW Advanced, they can use statistical packages like SAS, Python, or R, as well as other programs like Excel with HealtheEDW Advanced. It is a fantastic tool that works like Microsoft’s SQL Server Management Studio, but it has a lot more functionality and is easier to extract large data sets from. The processing power of the Snowflake platform is fast. We are trying to create automated jobs to save a ton of hours for extracting data sets. We are also finding that we do not need to take the data out, put it somewhere else, and then have our code done somewhere else for the normal models that we run. HealtheEDW Advanced enables us to just leave the data sitting there. When we ran predictive models previously, we had to move the data out because we could not connect with the statistical software. Now we can do that, and the process runs well. The system is saving hours of time when we have to process our modeling.

All our data is already in HealtheIntent, and it comes from our clinical and financial systems. That makes things easier for us. We have some basic reports, and we can create other reports from that data.

HealtheEDW, in conjunction with HealtheAnalytics, is more than just an enterprise data warehouse. It does more than just write logic in the data. It is actually a recording repository. Cerner has analytic tools coupled in the space, so we don’t have to go to multiple environments to do a complete report with data validation and quality checks. Everything is integrated into the same platform, so having a reporting repository that is directly connected to our back-end data has really helped us move to a self-service reporting approach. The process isn’t disjointed anymore, and we don’t have to have a dedicated resource who makes sure the data and reports move and refresh in normal functions. We have used a lot of our infrastructure that already existed, so we haven’t had to start up a team to worry about access or other basic IT functions. One thing is around the standardization of data ingestion with an enterprise data warehouse. We don’t have the flexibility and freedom to throw data in there. Outsourcing to Cerner is part of the mapping and ingestion process, so we have to give them a use case that exists in their world so that they can map it. We can’t just throw things in there because then we could get to a point where we don’t know how to maintain things because we wouldn’t have processes. We really have to sink through everything up front and have standardized concepts across many sources. There is a lot of initial legwork, but standardization has immense downstream benefits, especially for reporting because everybody uses the same variables. We have a standard source for definitions. That really forces us to be more organized and standardized with the data that we are using. We do the mapping for certain kinds of data collaboratively with Cerner. That is done in their transfer tool. They have very specific commands that we use so that when we transfer a file from our environment to Cerner’s, the software reads the commands and tells the system how to process things. There are a couple of different layers within the Cerner environment that the software processes for full data ingestion before we see the data in HealtheEDW. Cerner actually does the checks and quality control for data ingestion because the data lives in their environment. We don’t actually have visibility into that process, so we rely on them.

42


Clearsense

We have a bunch of different tools today that manage credentialing and other processes and different areas that receive provider information. Clearsense is doing an assessment of all the different business lines that have provider data, and we are looking at other demographic and data elements, which we will collect from the source systems. The elements come into Clearsense’s provider data hub, which pushes down changes to many of the corporate applications that we have today. We also use the Clearsense tool to push all the provider information and updates through our digital front door application, our EMR, our claims application, and our revenue cycle application. We want to make sure we use the tool as our one source for provider information. There are many sources that have provider data, but the sources don’t match, or they have the wrong address or TIN. There are a lot of things that go into keeping providers up to date. We will use Clearsense’s tool as a hub for dozens of source systems so that we can keep the provider updates clean.

The use of the tools and the rapidity by which we can curate the data so that a group of users can only see their data and not others’ data has made things easier for us. There is rapid deployment. The tools and the techniques that we are using up there make things very easy. We are providing this for self-service analytics. The data is all going to be inside the data lake without the restrictions that we would have provisioning a sequel server because we could never provision something big enough. We are always going to run into limitations. The data is all there, and then there is just a final step of curation to filter out just what those various groups need. Our heart and vascular institute is looking to provide services to cancer patients because people in cancer treatment that have heart conditions frequently get worse. With this, we have the ability to get the heart and vascular institute the basic data that it needs, but then they tell us that there are cancer patients that coincide with ours and give us their cancer information. It is more rapid to provision that by using another model.

The vendor provides the platform, like the data lake and SQL servers to harmonize the data.

Dimensional Insight

It took us a few years before we got people to buy into using Diver Platform. Now we have multiple dashboards that we have been able to build from to help people do their everyday work and keep track of things. Some of those dashboards are very specific and pull from different systems we have access to. That is one of the biggest strengths of the system; we can get data from just about any source that we use within the health system. The most difficult thing was getting buy-in. But once we overcame that, we weren’t having any problems with people getting information from the system. The system has eliminated errors because the data is being pulled electronically. As long as people are validating their data before a report goes live, it is accurate, whereas when humans are involved, there is room for error. The system has saved me hours and hours of pulling reports and collecting data. Before, I was collecting some readmission information by hand. That would take me anywhere from two to five days. Now, I just pull the information by changing the date. We have someone who uses Diver Platform who says that a report takes a while to come back. That person’s idea of taking a while is 15–30 seconds. The person expects the report to generate immediately. And most often, that is what happens. Sometimes the system takes 15 seconds to pull the information. But Diver Platform has been a time-saver for a lot of people. The system has allowed us to act on our data because we have been getting data in real time as opposed to two or three months down the road when the state or the federal government dishes out the information to us. That puts us ahead of the game. We can be a little more proactive in addressing opportunities for improvement rather than severely retroactive. I don’t have to wait for the data warehouse to create us things like a reporting table. Sometimes our team is like a second layer of ETL. The main ETL is done in the data warehouse, and if there is a project that needs a little bit more ETL, then we do that in Diver Platform.

We have empowered end-user departments to use the ETL tool. It is very simple so that end users can do basic ETL without issues and produce their own reports independently. It is easy enough so people who know basic Excel functions can also do some data manipulation the way they want to do it with their tools. Dimensional Insight has Measure Factory now, and that makes things really nice. We empower a lot of end users to slice and dice data with Measure Factory.

43


Epic

Caboodle’s data warehouse makes reporting a lot easier and faster in performance and in writing queries out of the SQL databases. We have been successful at adopting that. It was a change. The team did a really good job with seeing the value of Caboodle. It is a very fast database, so when we send in a query, it can usually come back in seconds instead of minute or hours. That helps us. My team members can check their work, see problems, and run things again. Caboodle has made it a lot easier for my team to make sure that we are providing good data and don’t have to specialize as much. Epic trains someone on Caboodle, and when we lose that person, it is easier for someone to come behind that person if the code is written in Caboodle or the person works from Caboodle. So Caboodle has most of the things that we report on all the time. Slicer Dicer has been a great tool. We encourage people to use it. When a request comes to us, we tell people it is a simple one and that they can do it themselves. We send it back, and that has saved almost a full person’s effort in a year. People can get their data that they want because they don’t want to wait for us. The platform is unified now. Previously, we had another vendor, and there were 10 different products that we had to integrate to get a single report. Now that the Epic system has all of those capabilities, we can retire those legacy products, and everything comes from one data source. All the information is in one location. Epic is doing a decent job of keeping everything running.

A couple of years ago, we were looking at getting into predictive, prescriptive, and advanced analytics. We evaluated a couple of different areas. Epic’s program was one area, and Microsoft’s Azure platform was another. We also thought about just building something in-house. But Epic’s program hit 90% of the boxes that we wanted to check. We have built a pretty good predictive analytics program, which we have used during COVID-19. We used it in a big virtual health center. The program has put us on the map and has been easy to implement. It advances us a lot in terms of analytics. My team does all of their reporting out of Clarity and Caboodle. Those are great on the reporting side. There isn’t one specific functionality that I feel is great for everything. Whether Clarity or Caboodle is better depends on what we are reporting. The software isn’t as fully built out as we need it to be at times depending on the subject, but at other times, the software is great for reporting.

Health Catalyst

The sweet spot that started the whole project was data transformation and ingestion. We also launched a different platform. The last piece for me was implementing a highly capable service delivery model and having deep talented data scientists that could work with clinical customers to drive analysis toward decision and change. Health Catalyst’s system has made the applications easier. Readmission Explorer, Population Builder, and those types of prebuilt applications have been helpful. When we are on multiple EMRs, the integration is pretty easy with Health Catalyst’s software. I think that is their bread and butter. It is easy to pull data into their operating platform now. They have spent a lot of time, energy, and effort on updating their data operating system (DOS) since they transitioned their DOS platform several years ago. That has made the ingestion a lot easier. Now that Health Catalyst has organized the software into something they call DOS Marts, the hierarchy of development is much faster and easier to do.

The marks in Health Catalyst Data Operating System are really useful. They allow us to repeatedly reuse components without having to build them from scratch. The system has definitely improved the efficiency and consistency of what we present. We have made use of some of Health Catalyst’s new functions, which bring in files from different formats and ingest the data from other applications. We are starting to enjoy those functions. The other thing that has been helpful is the labor-management tool, which came off the shelf. We were able to plug that in and start playing. Obviously, months’ worth of work was involved, but the work was around connecting the business processes to the application in terms of how we represent those. There is always work involved, but it is really nice to have a tool that is already built. We already have a user interface and design. Having those things accelerates the process because we aren’t spending a lot of time figuring out what things look like and where functionality belongs. We have been able to take advantage of what Health Catalyst has built from an analytic standpoint.

44



Information Builders has taken some steps in terms of master data management. At the highest level, there is a data-integration capability, but that is broken down into multiple functions. That capability is helping us extract, normalize, and transform data. The platform has some basic data quality functions. It creates the conformed data model that we use for a lot of our reporting. As a generic reporting tool, the Information Builders platform helps us, but we construct all of our own applications to do the analyses that we want to do. Otherwise, mastery is just standardized practice. Information Builders has a very clever way of integrating and matching data. They have different clinical domains. We could take any data source and quickly map it to the domain to be loaded into the Information Builders platform, which takes care of the rest of the transformation work. The platform gets the data into a conformed data model. That is reasonably unique. There isn’t a heavy ETL lift on our part. Information Builders’ data-mastering, ETL tool has proven itself to be robust and flexible. We use it to synthesize data sources. The vendor’s data-mastery tools and integration services have a lot of unique capabilities. There are two major functions of the system. One is to have a centralized data warehouse that brings everything together so that we can match that data to a particular patient or physician. We didn’t have that capability before. We wanted to bring everything to one place because we were finding ourselves doing more and more analytics that needed to draw in all of those data sources. Secondly, Information Builders has a product called webFOCUS, which is a presentation layer, and the intent is to draw from all of that data and bring everything together so that our customers who are logged in through a portal see not only a person’s longitudinal patient record but also the person’s SDOH screening, risk score, and other data. Our goal is to bring all of those data sources together and present them differently than we had been. We were in data silos. Omni-HealthData was the only product that we looked at that we really strongly felt could bring that data together.

Innovaccer

Innovaccer has the InGraph module, and that is the main tool we have been using. It is an application that sits on top of InData, the vendor’s data warehouse. We have also been using the InCare module, the InReport module, the InConnect module, and the InNote module at our physician practices.

The key capabilities are in the point-of-care piece. We have a great infrastructure in place, but we have to figure out how we bring meaningful insights to the point of care regardless of EMR so that the system is embedded and integrated within the workflow. We have what is essentially a third screen that seamlessly works behind the scenes and automatically recognizes the patient that is being viewed in the EMR. The system brings up the patient’s information right there so that it is at the fingertips of the care team members. We also present acute visit history, which is like a 12-month snapshot of the acute visit history. We can quickly see where, when, and why the events happened. We also have a specialty visit card, and that helps with the coordination and collaboration of past visits with specialists and future scheduled visits to understand the other members of the care team that are helping to support the patient and family. That is where we are integrating the specialty clinical notes as well and having direct access to the notes to help support the efficiencies with which the collaboration can happen. We are doing risk coding gaps and making the coding gaps available at the point of care. The other functionality that we are about to add is enabling social determinants of health at the point of care. We are integrating a system into the Innovaccer platform, and we are enabling the ability to initiate a referral and then help, track, or manage partnerships or relationships to benefit organizations in the community. We are hoping and looking to start to have the infrastructure in place to help evaluate the impact that providing social services have on healthcare outcomes. We are excited to help continue to build the infrastructure to help align further community investment in that type of service and potentially even use capitated funds and Medicaid premiums that we have access to through our capitated arrangements to help fund and support the advancement of the services. On the analytics side, it has been critical that Innovaccer be a good partner to us. Innovaccer’s system does all of the standard quality analytics and standard cost-utilization analytics that we would expect. Innovaccer is developing dashboards and reports to help evaluate the effectiveness of user adoption and adoption of platform capabilities. Innovaccer sees where value is being captured and

45


realized. They build the analytics to evaluate the value and effectiveness and to help continue to inform and educate end users as to how we get the most value out of what we are doing.

The first piece in the platform that has helped us is obviously automating a lot of the integration workflows. We have done a lot of lifting around that. There is the technical integration and the actual acquisition of data, which needs to be cleansed and normalized for the reporting side. The other big thing is along the lines of building out efficiencies with the maintenance of data and the reporting structures. We have a lot of reporting hierarchies within our physician communities and various slices that go in, so we have been able to offload the maintenance and structures in a fairly automated way. That way, we can get more into the analytics aspect.

46


What specific functionality is lacking from this platform that would make it easier for your organization to do data processing and analytics?Alteryx

It would help if Alteryx Connect had more prebuilt functions to pull data from other sources. A lot of those things were manually creating polls for the metadata from different environments. From the Alteryx server side and the designer side, the solution pretty much meets our functionality needs. I know they have a data science version that is out there, but it is more for a citizen data scientist, not a full-blown data scientist. Other platforms are free, so there is no reason to buy the Alteryx version.

This software is so comprehensive that even for a health system our size, there is very little functionality left that we would consider a critical need. The solution solved almost all of our critical basic functionality and even most of our advanced functionality needs. But there are two areas in advanced analytics that the Alteryx system is probably a little weak in. It doesn’t do great with artificial intelligence development, and the neural network development is a little subpar. That is their click-and-point interface and macros within the software. I could drive the system completely with Python, so that doesn’t matter; that is fine. But Alteryx’s system also works with click-and-point interfacing, and Alteryx doesn’t have a lot of great click-and-point interface tools that can then get into that unsupervised machine learning or AI development. We would have to exclusively use Python because the system doesn’t have some of the other AI-type programming languages and platforms that are really good for AI that may not connect to the Alteryx system. The vendor is working on that, so I know that somewhere in the future of their product line, that will come.

Right off the top of my head, nothing is jumping out at me that the vendor is lacking. The ability to create PDFs in the Alteryx system is not as easy as it is in some of the other tools. The Alteryx tool is more for data processing and less for report creating.

Arcadia.io

On the processing side, having the ability to do API connections allows us to more efficiently bring data in or push it out to other systems. That is something that is lacking today. On the analytics side, something that could make it easier is just building out more meaningful dashboards that allow users like me to get in there and not have to put in a request to a report writing team to pull a bunch of data out of the back end. The breadth of reporting available on the front end could be improved.

The vendor provides us with a database where they return all the data to us. Things would probably be easier and more beneficial if the vendor did incremental loads instead of sending us the entire database every week.

Arcadia.io has a lot of capabilities that we don’t even know about. When we start asking questions, we realize that the system can do certain things and that we just don’t have the features turned on or that there is a module that we haven’t paid for. Arcadia.io continues to grow and improve their platform. We have to stay up to date with all the things that they have to offer. There are a few platforms from Arcadia.io that we are going to be looking at in the near future. We are looking at Arcadia Vista, which helps integrate all the aspects in the module and packages information more easily for front-end users. Arcadia.io also has a desktop model that we don’t use, but it is very intriguing. The model sits on top of EHRs and gives real-time alerts to providers in the charts. That model could potentially decrease the need for providers to go to the Arcadia.io tool separately because they would get the alerts while in the charts. That is really exciting. We use Arcadia.io’s communicator tool, which allows us to send reminder messages to patients. Right now, the functionality is built so that any messages have to be built by the data production team. That can take up to two weeks. We have suggested that Arcadia.io create functionality that gives users the ability to create their own scripts and deploy those how they see fit. That functionality would really enhance the use of the tool. There is data integration, and how things are mapped is an expected challenge because data is coming in from so many different sources. But Arcadia.io is very on top of things. There isn’t a lot of interoperability yet, and we don’t have a connection to everyone. That is the issue. The functionality is there, but it is expensive. We definitely have barriers because of the cost to connect all the different platforms. There is an initial cost of having the internal teams have a knowledge base of and training on the tools and limitations. But the Arcadia.io tool has allowed us to get data from multiple sources into one location, and that has been wonderful.

47


Cerner

The thing that we thought was missing was solved by moving to HealtheEDW Advanced. We were missing greater security and the ability to connect to third-party tools. I think the vendor calls the system advanced because it gives us more capabilities for those people that are looking for them in their departments.

Sometimes when we are bringing in a lot of data, the system is slow. If we are bringing in one or two months of data, the system is okay, but if we are bringing in one year of data, the system is a little slow. The vendor needs to make it faster. They are working on that.

The concept of outsourcing data ingestion is definitely a weird one. We relinquish the control of seeing a certain dollar amount in our raw data. With HealtheEDW, we are seeing half that amount. From an analyst’s perspective, we would backtrack into the data ingestion steps of what went wrong and where there were changes. When the raw data does not match the output, we have to go to Cerner, say that we have no visibility into any of the steps, and ask whether they can do things for us. We are really learning their language of how they do things and helping them build processes to give us more visibility without us having to maintain things. There is a collaborative space that we are still working on with Cerner. Our relationship with Cerner is one that definitely needs oversight over time. In terms of what could be improved, I would have said geocoding in the past, but Cerner has found a geocoding partner, so now all of our data can be geocoded to better categorize information and the available national algorithms. Cerner is putting robustness in one space to help enhance our analytics without putting the maintenance burden on us internally. Cerner really owns and is moving forward in the space.

Clearsense

Clearsense’s pricing model for their capabilities isn’t fully baked. If they are going to sell the tool as a package, they need to look at a three-year ROI and a three-year cost build-out. We want an implementation fee, a data or licensing charge, and a monthly fee for whatever we sign up for. The hardest thing for us to negotiate is the pricing of the platform. Clearsense needs to take the cost out of some things so that they can compete in the market. Clearsense has great technology and brilliant people who work for them. They manage things that nobody has ever seen, and they are super confident. But Clearsense needs to have levels of pricing. Right now, there is just the full burden. Clearsense needs to figure out how to fund their start-up costs and reimburse themselves and not pass those costs on to their clients.

The tools that Clearsense provides are a little immature. I wouldn’t even say there are bugs, but we have issues with going somewhere and clicking this check box and things like that. Those are little things where we need to know about those tools that are done differently in other tools, especially when we are doing our data loading and our ETL. Using the tools that Clearsense provides was not as easy a transition as we thought. We keep running into those nuances and little glitches with the tools, and a maturation of the tools we use will help in this particular product offering with the data hub. Clearsense’s support is top notch; they do not leave us hanging. They give us what we need in order to keep going. One of the things that we are working with Clearsense on is a little bit of a better platform for our data scientists. While Clearsense’s system can take the genomics data, it looks like we would have to extract the genomics data and put it somewhere else to be able to use it. The data science tools are not available to us up there. That would be a feature that we would probably use if Clearsense could create a data scientists’ platform.

I don’t think the vendor has any expertise with NLP. The overall data science pipeline is not very well defined. If we want to have a machine learning pipeline or anything with implementing the clean decision support system or anything like that, I don’t think the vendor has the capability.

Dimensional Insight

We figure out a way to work out pretty much anything we request. The system has graphing and dashboard capabilities. One thing I would like to have is more graphing capabilities. I like my graphs to be the colors of our organization. But there are a multitude of colors and types of graphs, presentations, and tables available. I just wish that I had access to more Excel-type functionality when I am

48


building a graph. The graph can still be built; I just have to think about it a little differently. That is a weakness, but at the same time, it isn’t, because it is the case with any system.

Dimensional Insight needs to figure out how to handle bigger datasets somehow. Right now, the system is not working; it just can’t handle it. With the performance that comes with it, Dimensional Insight is going to need to focus on that so we can use it more often, especially for healthcare, because the data sets are so gigantic.

We use every aspect of things where we even empower end users to generate emails automatically to the physician groups with quality data. I don’t have anything that comes to mind in terms of specific functionality that is lacking from the platform that would make it easier for our organization to do data processing and analytics.

Epic

We are always looking for more speed out of Caboodle. When we do upgrades, the backfilling process can be slow. We are interested in getting third-party information more easily into Caboodle. We would like to be given the social demographic information with a click and not have any of the work that we have to do. We are hoping Epic can make some common data connections that people are interested in from the Bureau of Labor Statistics and anything they can do to make things easier with the APIs. There is a promise in Caboodle that companies will eventually be able to use APIs to connect to our database and build apps. We are really looking for that to become a reality because we are all doing the work over and over. We are not sharing code. That just feels very wasteful. So the promise of Caboodle is that vendors can eventually tap into that and pull the data. I am looking forward to Epic doing more there. We write every extract from scratch, and it is just frustrating sometimes, especially with big companies that have a lot of clients that use Epic.

Let’s say we have a table that supplies information to the Radar system. In order to duplicate a dashboard in that system, we have to duplicate five different components first. It is a difficult process to get a simple dashboard. We hate that process. Part of the issue may be that the program was written in the late seventies and was based on old databases. Everything becomes harder downstream. We should be able to connect the data tables, format them, and be good to go. Behind the scenes, things aren’t very simple or straightforward. Epic needs to take a step back and look at things from a higher level. The Caboodle system is being updated fast and furiously. The vendor is trying to get some of their systems to a higher level. Right now, there is only a single thread of execution in Caboodle. The product runs nightly, and sometimes, depending on the amount of data that needs to be brought over, the system can run for hours. When we need something for our senior management or when a report needs to be done, we can’t run it because the system is already running one execution, and we can only do one thing at a time. That is a big issue. The vendor is hoping to fix it soon, but I don’t know when. In some Epic systems, users have the ability to drag, drop, and create their own reports. In the Slicer Dicer system, users don’t have that ability. We have spent a lot of time and effort looking at the Epic universe. The Slicer Dicer system is not where it needs to be yet, but it is moving up; a lot of progress is happening. What is missing from the system currently are charting capabilities. We can put paragraphs in a tabular format, but a lot of the charting capabilities are limited.

There are so many different things that Caboodle is lacking. There are so many different avenues to get data from Epic, and that is one of the reasons that our organization has done really well. We have data everywhere. But it can be very difficult to home in on where data should be coming from. As a large organization with multiple people reporting, we run the risk of people having the same requirements, trying to build the same reports, and pulling from different places. There are almost too many options, but the options aren’t narrow enough. We kind of fight with Epic about Word. A lot of their information lives in Epic, so they want users to sign into Hyperspace for all reporting needs. But I don’t think our CEO has ever signed into Hyperspace. We spend a lot of time manipulating Epic’s solutions into a format that our executive teams would like to read. Epic doesn’t necessarily provide that format. They need to have functionality that makes things available to people outside of Hyperspace. Sometimes Hyperspace isn’t as user friendly as Epic thinks it is.

49


There are several things missing from the platform. It is pretty good at doing custom models or models that specifically involve clinical data. But we are starting to get to a point where we are doing models that involve supply chain data or other data that lives outside of Epic’s platform, and the platform isn’t very friendly to that data. Being able to incorporate nonclinical data would be helpful; then, we could use the Epic platform as a one-stop shop for advanced analytics. The platform is lacking there. Part of the issue is that it is fairly new. Epic is growing and building out the platform. I don’t know whether they have specific things on the road map. We have always gone with Epic, and we probably have 1% of outside data.

Health Catalyst

I don’t always know how many issues are due to the vendor’s lack of functionality and how many issues are due to our lack of spending time with the vendor. There are opportunities for the vendor in things like the operational management end of healthcare. Labor management is one thing I would elevate. The vendor groups up with quality and clinical data as their sweet spot. I think a further move toward back-office functions of analytics and looking beyond the revenue cycle would be good. I could see the vendor maturing some of those capabilities that we buy off the shelf from other vendors.

I can’t think of anything we are missing. I know Health Catalyst is working on data cataloging, but they could probably stand to develop more there. Their catalog functionality is a bit weak, but that is a minor thing.

There are always opportunities in terms of improving processing efficiency on the data warehouse side. We have issues sometimes. The processes end up interfering with each other, and things fail to load, so we can’t deliver the reports that the teams need in the morning. That issue doesn’t go on as regularly as it did. I think the problem was that as Health Catalyst and their data warehouse were growing, the vendor needed to become more sophisticated in how they managed data on the back end for balancing the timing of different jobs so that they didn’t step on each other. There is an ongoing challenge there, but the situation is improving. There is no question of that. As a customer, we have always challenged Health Catalyst on some fronts, and that isn’t a bad thing. They respond well to the challenge; they actually turn things around and typically deliver what we need. We are pushing the envelope and bringing more things to Health Catalyst’s attention so that they can work on those. Us doing that improves the platform for everybody.


It would take a long discussion to go over the functionality that the Information Builders platform is lacking. I don’t mean that as a criticism of the vendor but as a criticism of the maturity of healthcare functionality and analytics in general. More support for AI would be helpful. Information Builders has plug-ins and things of that nature, so we aren’t necessarily constrained by the vendor, but they aren’t exactly helping us climb the maturity curve in terms of building and deploying AI models. That is something we have on our list. We are looking forward to some advanced capabilities, but we will see. Information Builders does have a cloud-based strategy, but we aren’t there yet. In terms of standardized metrics, Information Builders has a long way to go. If I looked at all the possible metrics, there could be thousands of things, and Information Builders doesn’t have thousands of things yet.

I find Information Builders’ reporting environment to be very cumbersome and quirky. The vendor prides themselves on having a compatible solution and supporting many legacy customers without changing things, but that has resulted in multiple different ways of doing things. The solution is very challenging for new users, and it lags far behind products like Tableau’s or RBI Medical’s in terms of use and overall capacity.

So far, one thing that is missing is built-in AI. Another thing that is missing is FHIR integration, but that doesn’t really relate to analytics. The system is new enough that we feel that those capabilities should have been built into the product.

Innovaccer

The system doesn’t lack anything that we would expect from a normal population health vendor. It would be nice if more vendors began to add more actuarial capabilities to their systems in terms of forecasting overall cost experience relative to actual reconciliation under different programs. That issue is a little nuanced though. It is probably more of a service than technology because the reconciliation from payer to payer is different. The feature is not a must-have. It would be an added bonus if the vendor could figure out how to do some modeling based on the major components of risk and attribution as well as cost trends.

50


In the space that we are focused on, Innovaccer’s system is not an enterprise data system yet. We have a lot of claims data, but we aren’t doing the revenue cycle management piece with Innovaccer. We are also not doing staffing and HR, which would fall into the operational category. We are really focused on clinical and financial data.

Innovaccer needs to put in predictive analytic capabilities that are more fully ingrained. I see the capabilities, but they aren’t quite there yet. One area that we are homing in on is how we can effectively identify high-risk patients. We can look at historical data and come up with key drivers, but that isn’t really how we identify rising risks. In order to do that, we have to do more in the machine learning area. What is missing is integration within our build to effectively facilitate machine learning. We can get things to fit into a specific parameter, and we can take in the parameters and put those into the application. The application can do everything that it needs to do, but the process would be much cleaner if Innovaccer had everything contained in one area. That would give us more flexibility around what carries more value for us in our market.

51


Based on your own perception and definition, does your vendor provide a master data management (MDM) solution?Alteryx

I wouldn’t say the solution does master data management. But at least with Alteryx Connect, it does gives us a central depository where we can store information on data that we have mastered. We have mastered a 360-degree view of a consumer, and we have a master provider database. We store all the information about that in Alteryx Connect so that if people have questions about where some data source comes from, they can look there really easily.

There are multiple parts of the Alteryx software that accomplish functions of master data management in our organization. Alteryx Designer is the main desktop product that we are developing in. That is where we can do all of the data engineering to do data quality, transformation, and standardization. But there is another Alteryx product called Alteryx Connect. Alteryx Connect is a full data management solution that has data lineage, data linkages, and data catalog functionality. It has search functions where it will connect to all our databases. If I am searching for the patient gender, the system will tell me all the databases patient gender is sitting in as a field. All of that function exists in Alteryx Connect. Between the two Alteryx products, they have a full end-to-end master data management solution.

The product has the capability to do master data management; we just aren’t utilizing it in that way at this point.

Arcadia.io

We didn’t consider this vendor because we didn’t buy them for that purpose based on our definition of master data management. When I think of master data management, I think of a source for a vendor that is going to integrate all of my data sources across our organization, and that is not what Arcadia is for. We are using it for clinical and population health as well as value-based care analytics, so it does integrate some data sources, but not all.

Master data management (MDM) is the single source of truth for all of our downstream third-party systems, which includes Arcadia Analytics. It isn’t at the top level of MDM and doesn’t have the level of detail that we need in an MDM system for our provider and practice information. We do handle MDM internally.

Cerner

Cleanup has to occur someplace. But our experience is that we are doing some of that and Cerner is doing some of it because they are creating EMPIs. I would say that other clients probably use more of the system than we do.

The product is a true enterprise data warehouse. It has the means to ingest different data sources from across a couple of different fields. There isn’t just claims and clinical data. We can take our other data sets and incorporate them nicely and cleanly in the warehouse, which is much easier to maintain. We have to have standardized processes. Since HealtheEDW is an enterprise data warehouse and built off of Cerner’s software, it is built into part of our EMR and the technical infrastructure that we already had in place. We use our internal IDs to log in to the warehouse. It isn’t a foreign platform that people have to remember to go to. It isn’t quite as ingrained into the EMR as we would like, but HealtheEDW is still a central place that everybody can commonly refer to and use. It doesn’t have the nuances of a new enterprise data warehouse, but HealtheEDW does provide flexibility and brings things together easily.

Clearsense

We are not using the system as a master data management solution. The vendor has begun a master data management practice, and I have personal relationships with folks in Clearsense. I know that they have embarked on a master data management project and are offering that as a cloud service. It seems to be pretty successful, but we are not using them for that particular service yet.

The solution ingests the data, and it goes to harmonization and refreshes the data sharing with the solutions we have.

52


Dimensional Insight

Dimensional Insight does have some tools for data governance now. We are in the middle of implementing certain data sets for it. I am not particularly involved with it at the moment, but I do know that it is coming for some tools. From what I have seen, it looks cool. I just haven’t used the tool myself.

Dimensional Insight has a number of products now that they have created that create master data sets. From there, we create the data hubs. That was something we did not have 10 years ago with them, but now it covers not only our clinical data but also our financial and general ledger data. That has helped us a lot with having everybody on the golden record.

Epic

Our history makes us consider Epic’s software as a master data management (MDM) solution because we are a centralized data and analytics team; we have to be centralized. If we were decentralized, there would be a lot of databases and warehouses all over the place. But IT has had centralized business intelligence and data analytics for many years, and we did not build many data warehouses before Epic had one. So if Caboodle was an organization’s first big data warehouse, they are probably thinking of Epic’s software as their MDM. We really look to Epic’s solutions right now because, luckily, we were centralized. We did have a data warehouse before Epic came. We weren’t waiting for them. But we really converted to Epic’s system very quickly when we saw it. They have some good best practices for MDM. Their data dictionaries are very good. They are an example of what we would like to replicate for our other data warehouses.

We recently moved over to data governance. We haven’t had the ability to create a master patient record with the Epic system. We use other tools and have come up with our own unique way of generating them. I am sure the system has those capabilities, but we haven’t been able to get there yet.

Everything depends on what we are talking about. Let’s say we are talking about physician data. One would think that we would have all of the physicians’ information in the same place in the Epic platform, but that isn’t the case. We have disparate systems that track physician data, and different organizations are in charge of different things. We can’t get everyone to come to the table and be standard. Epic would say something different from our credentialing office. We have undergone a project, and we have purchased a separate master system for data management that we are using for the project. I have been very happy with that system. Epic’s platform is great and can be used as a master system for data management when users have data that lives in the platform, but that isn’t the case for everything that we need.

Health Catalyst

The reason I consider the system to be a master data management solution is that it takes data sets from across the organization and ultimately houses them for use for the single purpose of driving operational excellence for the company. I see that as different than how other systems serve us for clinical care or our revenue cycle. The system is the ultimate one-stop shop; it houses all the critical data we need to review to make decisions.

Health Catalyst’s DOS platform definitely functions as a master data management system with the metadata and everything else. We don’t work with anybody else.

There is a third-party tool that Health Catalyst uses in a lot of their master data management (MDM) work, and we definitely use Health Catalyst Data Operating System for that work. We have been pushing Health Catalyst about MDM and pointing out where they can do things better. We spend a lot of time on MDM.


The one healthcare-specific thing that Information Builders delivers out of the box is a terminology-management service. The most valuable thing to include is the standardization of code use and specific metrics. Information Builders provides master data management, and that is necessary. If we bought a competing product, we wouldn’t get that. Master data management is unique because it isn’t ubiquitous across all vendors yet.

53


We did our homework by calling around to some of Information Builders’ customers, and we heard that master data management (MDM) was one of Information Builders’ strengths. One organization we talked to said that Omni-HealthData was one of the strongest MDM platforms that they had. We also talked to one of our hospital partners who said the same thing. Our experience is showing they provide MDM. We use another MDM product, and we are finding some really good correlations, but we are pretty pleased with Omni-HealthData’s capabilities.

Innovaccer

The system is not just ingesting claims like a normal population health system would. The system is also ingesting clinical information, C-CDA documents, ADT files, and other data from our affiliated practices as well as our employed and owned assets. The system is ingesting data from outside sources as well, including HIE data, nonsolicited lab data, biometric information from some employers we work with, and SDOH data feeds. The system is also beginning to ingest scheduling data and some practice billing information. We haven’t historically used the system for those things, but with the rollout of another module, those things are becoming more common. The system is really ingesting all the population health data we need from soup to nuts. There is a collaborative partnership. Innovaccer does all the technical configurations and executes how we need to manage our data and hierarchy effectively for our analytics. We set the parameters and guideposts, and Innovaccer can figure out their solution and architecture and make those things work within the current environment. The big delineator with Innovaccer compared to other vendors is that other vendors fit organizations into the vendors’ structures and parameters, but Innovaccer conforms their workflows to meet how our data is formatted and efficiently moves things into the structures that we need.

54


Are you using data ingestion capabilities from this vendor?Alteryx

From a data connector perspective, most of what we do are connections from open database connectivity, and we are able to connect to pretty much any one of those systems. On the clinical and financial revenue cycle side, all of those departments are using the Alteryx software and pulling either from their database or from our enterprise data warehouse that might have replication in their data. We are not doing a lot of unstructured data, but we do a lot of semistructured data. We have used the solution to parse data from an unstructured format to do some sentiment analysis, although sentiment analysis did not happen inside of the Alteryx system. But we used the Alteryx software to learn the data.

Data ingestion contains some of the strengths of Alteryx’s system. One of the strengths of the software is being able to mix disparate data and disparate types of databases. We can be pulling in data from SQL but then also pulling data from another product, and then we can go grab data, and everything comes together in seconds, depending on how much data we have. The software is pretty robust and is very functionally efficient. The software doesn’t limit the speed; the server or the computer we are running the software on is what limits the speed. We are in the cloud already. We have Microsoft Azure. That system has a lot of tools in it and a lot of different functions. We use Microsoft Azure as data storage for different containers of data. Right now, I am not using the data science tools in Microsoft Azure because a lot of them were based on transactional costs, and that gets expensive. I don’t have to pay for that in the Alteryx system. There is no charge for how much data I am crunching in the Alteryx system at any scale. Because of that, we don’t really use Microsoft’s machine learning services or their tools. The Alteryx solution is doing everything else; it is doing the data engineering, moving the data, putting the data in its final resting spot, analyzing it, and reporting on it. The system is truly the middle layer of the infrastructure.

Data ingestion in the system is great. It is really easy to pull in all the different file types. It is easy to see what we are pulling in and then clean and manipulate the files from there. In general, the analytics and reporting are great. The system is easy to use for what we utilize it for.

Arcadia.io

If we include payer adjudicated claims files in financial, then we do have data connectors. In healthcare, if we are not listing payer-provided medical and pharmacy claims data as a separate connector or data source, we have a big gap there. I don’t incorporate anything out of my finance or revenue cycle system except for what is in the clinical. When we say clinical EHR data, to me that includes everything in our EHR; practice management, revenue cycle, and clinical data is all going into Arcadia’s system from our EHR.

We are definitely using the tool on the clinical side. We are also using it on the operational side. Arcadia.io relies on us to provide the right data in terms of practices, hierarchies, and physicians who are termed. With the financial and revenue cycle, we are diving into that data as well as the advanced functionality of the semi-structured or unstructured data. That data lives on the back end, and we are starting to take deeper dives into it. We definitely have a lot of room for growth there. There isn’t as much growth on the financial and revenue cycle data side. We aren’t reconciling any unbalanced there in terms of total revenue. We look at the financial cost, but the Arcadia.io tool reprices the claims. The tool doesn’t correlate directly with our financial side.

Cerner

We do not have bed data in the system. Research is a different beast. I do not consider research data any different than data that would be in the system. We do not have census, supply chain, clinical engineering, or HR things in the system. We mostly have structured data in the product, but we do have some kind of unstructured data.

We aren’t using the metadata, the data catalog, or the task management. Those concepts don’t quite fit into how we are using the platform today. We are loosely using the management piece for the data life cycle. Data stewardship workflows are an area of opportunity with the ongoing work.

55


Clearsense

For clinical notes, we have several billion pieces of data on the Clearsense platform. Some of the data is in the Observational Medical Outcomes Partnership (OMOP) format, and some of it is raw, but we have all our EMR data coming over to the OMOP platform. Financial revenue cycle data is already in the same clinical workflow data, so it is not separate. I would say we do have unstructured data that is not being explored yet. The vendor is trying to handle semistructured or unstructured data. We do not have unstructured data to structure. We will be using newer capabilities to use them for any machine learning algorithms, but it is a really long way to go. That is why we need an NLP platform.

Dimensional Insight

We mostly use structured data. Dimensional Insight does well with data ingestion until the datasets just get too large. I am talking about billions of rows. I think anything over 2 billion rows is already an issue in there.

Epic

We are not ingesting strictly structured data, but we produce it inside Epic’s system. A clinical note is unstructured or might be semistructured with some data elements attached to it. So there is nothing from outside, but our users do generate both types of data.

We are using a claims data connector. We can integrate outside data. That capability requires a huge learning curve, but the data integrates well.

Health Catalyst

We are using the system for claims pretty heavily. We will use it for both semistructured and unstructured data. Structured data is more helpful and usable. There is less value in unstructured, but if it is relevant, then we will try to use it.

We have been working on the unstructured text in the data mart with the vendor. The project has been relatively slow.


We aren’t using the Information Builders system for structured or unstructured data. We bring in some notes and merit things, but we haven’t gone down the NLP path yet.

We have established some dual coding capabilities that Information Builders offers as add-on modules. They make use of a third-party product.

We are already using Omni-HealthData for claims and SDOH data.

Innovaccer

[No comments provided by respondents.]

56


Are you using data management capabilities from this vendor?Alteryx

We are not using the Alteryx software for our master patient index. We do the metadata catalog task management and that side of things. As far as comprehensive ETL goes, we give that out to other departments to do their ETL work, but the Alteryx solution is not our main ETL tool. We use it to clean data up, especially when we are pulling from outside sources. We do data transformation and might make change to things in one system, and we do that data integrity and pull it together. We are not actually doing a lot of data checks to see what the sell rate is on something like that. We can do that; we just don’t. For the enterprise data warehouse, we have a different vendor’s system. But this allows us to have people extract, transform, and load things and then do any sort of data manipulation things they want to do. We used to have people who would build a dashboard in SQL, and then we would end up taking the SQL apart and rebuilding it and the ETL language so that it was more reusable. This eliminates that step because we just load it to the Alteryx server here, and it runs after our enterprise data warehouse for us. For the advanced functionality, we have a longitudinal patient view, but it is built outside of our Alteryx environment. We do data manipulation off that view in the Alteryx system where we are building diagrams and things like that and using the Alteryx software to manipulate the data into the format that we need to run our dashboard. We don’t build anything into a healthcare data model on the Alteryx side of things. We mark who the data stewardships are in Alteryx Connect, and we have actually connected those workflows over to another tool we have. So if somebody requests access, the system links over to another tool and goes through that just because there is a different security apparatus for that one. In Alteryx Connect, we show the data lineage, where things are coming from, and how data has been manipulated. People can click through to their hearts’ content going backward and forward. That is helpful. We only get so far with metadata. A lot of that it is still manual input at a certain point where we are putting in the information behind that data lineage that makes it make sense. As far as data life cycle management goes, we haven’t really used the software from that perspective. What the Alteryx solution does is big. There are a couple of nuances around task management that other products do. There are some service-side operations that are very useful to large companies, and the Alteryx system doesn’t do those things. We are using some aspects of another product for some of the things that the Alteryx system doesn’t do. It will do some type of quality and master data management reporting from servers. We have so many servers in the company, and another product is deployed on all of them, so we can see into all of them. With the Alteryx software, we have to connect to the database we want to work with. The other product is automatically in those servers and monitoring performance in real time. When we see something wrong in the other product’s monitoring, then the Alteryx solution is our tool to go fix it. That is the way we are interplaying those products. Some automated task management things around data quality and monitoring are not built into the Alteryx software.

Arcadia.io

Arcadia already does back-end SQL technology techniques to manipulate data. We don’t have to do it. Arcadia certainly gives us data dictionaries and data definitions about everything we see, so we definitely have a metadata catalogue. The vendor takes data integrity seriously and they are doing what they can to make sure that the data is of high quality when we get it. There is a longitudinal patient view in the system, they have a patient chart that brings in all the data longitudinally. Inside Arcadia, they have clear accountability for the different data sources, so the vendor has data stewardship. But that is not something we do. I would assume that Arcadia has linear data stored somewhere, but I have never quite asked how they take the raw claims data and transform it to get it to how it appears to us in the back end.

57


We are able to track all the measures for the different payers and their value-based programs. We are able to build what we want to track on the front end. From a care management perspective, we are able to run and leverage reports from all patients that we are tracking. We can share those reports with payers to get credit for some of the services that we are providing. We definitely use the metadata in terms of figuring out all the data inputs and risk-score levels. Risk stratification is something that we are looking at, and the Arcadia.io tool definitely gives us that view. We also have the longitudinal patient view. We are able to track patients down when they come in and look at any subsequent care.

Cerner

Cerner has the technology for master patient indexing.

We have metadata. The vendor has that because we set up that process and run it. The platform is new, but we are establishing it. The functionality is there, but there are different ways that it can be used. The vendor also has the functionality where we can do our own healthcare data models. The vendor has a wiki site where all the information is stored by clients. Everything has dates, but because the platform is a big data platform, we aren’t limited to just a few years. We may not limit a certain report to a few years, but all the data is there back to a certain year, and it just keeps building. If we wanted to cut off data at a certain point, the vendor would do that for us. However, there is no need to do that. That is why the information can all still sit there.

Clearsense

We don’t use the MPI piece today because we aren’t working on the patient side. We are working on the NPI side. In terms of data cataloging and task management, we are using the Clearsense platform in conjunction with another platform for metadata and data cataloging. The combination of those two platforms is a nice marriage. We use Clearsense’s healthcare data models. Data stewardship and data lineage workflows are done in the two platforms. We definitely use the data-life-cycle management for research data and for the provider data hub.

Dimensional Insight

The data management is okay until there is too much data. Right now, we are finding out which data sets we have where we have to deal with that already. We have been dealing with this ever since we moved to a new EMR. At the very beginning, there were a lot of issues, like just trying to figure out how to get the data in the system. We really don’t have too many issues now; maybe once a month we see network issues or a full server. That is nothing that is really out of the norm with the size of data that we use.

Epic

We haven’t deployed Epic’s metadata and data catalog as much as we would like. That is not because we don’t like it; that is because we have to come up with a plan for it. I am not sure about data stewardship workflows.

Health Catalyst

The MPI is tricky because we use another vendor’s system as the primary driver that builds the EDW, but not everything is patient related. When something is not patient related, we use the data warehouse. We don’t use the vendor’s system for data lifecycle management, but that is maybe more our choice than the system’s lack of functionality. We are not doing anything for data stewardship right now, but it is on our road map. Data life cycle management is on our road map as well.

I don’t think Health Catalyst offers something for managing the data life cycle. Getting that is on our to-do list, but unfortunately, there are so many things to do before that. Managing the data life cycle is absolutely interesting. That is a big part of information management, but we haven’t really begun to do anything in the space.

58



We are just starting to look into ETL approaches. We do a lot of ETL work by conforming data to a data model, but we just lift and shift data forces. We aren’t really using ETL as people would normally interpret ETL, which is doing complex transformations on the fly in a certain layer. We are taking a look at doing that, though. Talking about metadata could be a conversation about three or four different things. We have metadata, but we don’t get a good data dictionary from Information Builders. We are going to do an upgrade in the future, and my understanding is that we will have some metadata things. We lightly use Information Builders’ platform for data integrity and data quality. We also do light data stewardship. There are a lot of areas that we aren’t using but that I wish we could get.

We are almost live with the Master Patient Index feature, so I guess I probably have to say that we aren’t using the system for that. A month from now, we will be.

Innovaccer

The system offers data lineage for some of our MSSP reporting. The system can track where the data source came from, and that information is time stamped.

Metadata is there. The vendor does a lot of standardization and normalization for us. That is how we are evaluating information across the types of data. Data integrity and quality are critical. We have a lot of controls in place not only in the process of implementing the data feeds but also in monitoring the data feeds and quality of data. That is great. We also have data lineage. We can always trace things back to the raw data with the vendor.

We are fairly new to the platform, so we haven’t gotten to end-of-life things yet. Ultimately, we will get there, but we haven’t had to encompass that yet.

59


Are you using analytics capabilities from this vendor?Alteryx

We do not use the Alteryx system for our dashboards, but it does all the data manipulation prior to the dashboard. We have done some visuals in it, but nothing that we have published out to the enterprise. We might do this for some sort of analysis where we do some graphical components so that we can understand the data, but we are not really doing that in the Alteryx system. From a reporting distribution, we have used their Gallery tool where we will put certain elements in the tool for people to grab reports and change the dates on them. That basically eliminates the need for us to throw out an Excel file. The solution allows people to get their own pieces from that side. We haven’t used any prebuilt healthcare applications. Our main piece is really the self-service analytics piece. I have many Alteryx Designer systems across the organization, and we have trained people on those. We have access through our data systems to do that, and that has been a big boon for our organization. The only problem is when people get data, they just ask more questions, but that is life. The automated analytic workflows are actually one of the main keys for us being able to take the workflow and automate it in a server where that is a nightly job. That is a big win from our perspective because we don’t have to maintain the workflow, and we can just automate it. If something breaks, we hand the system back to the vendor, and they fix it. Or if the issue is something people want to modify, and that is what normally comes when people want to add something to the dashboard, they can still work with their workflow. And when the vendor updates their dashboard, we just update the nightly job, and that is a day’s worth of work versus three or four weeks of rewriting, and that is a big thing. We have done some cohort analysis in the Alteryx system. We look at people who use different services that we have and how they performed versus a doppelganger that didn’t take those services. We use some of the visual tools related to the application, just not a lot. Most of the time, if there is something that needs to be answered really quickly, the service is great. If there is something that people are going to want to see, then we build it out in a different system. We have not used the solution to embed anything into an EMR. There has been embedded content, but not in our EMR. I stay away from that because it is too much of a fight. There is a benefit with the cost and advanced analytics. In Alteryx’s system, we can do custom algorithm development and embed it into the EMR. No other systems can do that. Doctors want to see a risk-stratification algorithm in their EMR. That is the doctors’ first choice. Their second choice is in a reporting environment. Alteryx’s system has answers for what a lot of other systems don’t do. The dashboard capability is one thing that is lacking. Alteryx’s system has a traditional reporting functionality built in. We can build reports in the system, but the functionality is nothing compared to what we see in Tableau’s system, Power BI, Qlik’s system, or MicroStrategy’s system. Those are true dashboard visualization tools, and they have a lot more functionality from a dashboarding and reporting perspective than Alteryx’s system. That is why we use another product as our visualization layer instead of Alteryx’s system.

Arcadia.io

If healthcare content includes the prebuilt reporting for things like member rosters, patient registry, and all of that, then we are doing that with Arcadia. We have self-serve analytics through the Vista dashboard and a few others. When it comes to visualization tools, I don’t think Arcadia partners. For the new Vista dashboard that they just rolled out, I don’t know who they are partnering with, but I believe they told us they built the technology themselves. I am not aware of any other big-name visualization tool that they are working with. We are not embedding Arcadia content anywhere today.

60


We really relied on Arcadia.io to build the dashboards and walk us through everything. We have scorecards that are being created in a certain program, and we have really relied heavily on Arcadia.io to do these things for us. The functionality definitely exists for us to create our own things, but we aren’t maximizing it. That is where we are lacking and starting to dive into. We don’t do much with embedding any content outside of the EMR. The advanced functionalities for development teams and capabilities are something that we are starting to look into. There isn’t too much on the dashboard, reporting, or prebuilt healthcare applications. The self-service analytics is the main key that gives us the ability to quickly manage data without relying on a vendor to pull reports. I am not sure whether the desktop model is bidirectional where the EHR could pull information from the tool. The model is more of a visual reminder tool. Although we do build our own reports from the back end, there is a balance between what is prebuilt and what we pull in from the back end for transitional care reports and utilization dashboards. The tool is still in its infancy in terms of its true capability, but it is being used.

Cerner

If we needed to do anything with cohort analysis, we can do it ourselves. We can only use the visualization tools that the vendor provides for HealtheAnalytics. With HealtheEDW Advanced, the visualization tools are not native. For example, we can hook up Power BI to the system.

We are largely using analytics across the board. We aren’t using the ability to embed content into the EHR, though. Right now, we very much have a one-way street. We can take as much information from the EHR as we want, but there is a gap in terms of our ability to push information back to the EHR in a meaningful way after we get some analytics.

Clearsense

We are using the dashboards only for research. We don’t use the reporting distribution piece today, but that isn’t to say that we won’t. Automated analytics workflows will come in phase two. We use Power BI as our visualization tool, so we force the data into that tool. We don’t have a round-trip data flow into the EMR today; the data just flows one way.

We are working on dashboarding and reports. Those are not very well used so far. The vendor has a third-party provider for our cohort builder that has an application where we can do self-analytics without any knowledge of programming. I do not think we have the ability to embed content into our EMR. It is a two-way process.

Dimensional Insight

We don’t embed content in the EMR. We use all the visualization tools, but when I need a quick turnaround, I pull the data and load it into Excel because I have a lot of templates already built in Excel. That doesn’t imply against Dimensional Insight; it is totally on me. The reporting can still be done in Diver Platform, and it has been.

Epic

We have done prebuilt analytics for certain things. Automating the process for data that goes out to monitors is one thing we have done, especially around COVID-19. We do some alerts based on analytics inside Epic’s system. So if a metric is running high or low, there are a few pieces where we push alerts on that. We do cohort analyses. All the visualization tools are native. The platform is like a toolkit for us. We use all the Epic visual tools, and then sometimes we go into another vendor’s system because that allows us to do the visualizations and custom reporting that we want to do. We have not embedded content in our EMR. We have done proofs of concept on that, and we probably will in the future. We do want to use Epic’s solution as our master catalog in the coming years, and that means bringing our other vendor’s lists into that.

61


I don’t think we are there yet for the automation of analytics. I would like to do more things like that. We have done a good job of operationalizing. We build the analytics. We have done some good things with the deployment and access. We mainly work on service lines, like the emergency department, inpatient care, and finance. When we deploy things to our other vendor, for example, we have very specific approvers that let people in to see that data. We have done a good job of deploying the content, making people aware of the content that is out there, and making it accessible for them. They can’t always build it for themselves, but once we work and deploy things, we can use the software to track metrics; it does get into their operational workflows pretty well. We are not doing anything with data masking, anonymization, or automation right now. In some cases, we even do heads-up displays with dashboards that are on monitors, especially around COVID-19. We are really getting excited about mobile versions because some of the dashboards that are heads-up displays should really be on our executives’ phones. We should make mobile versions of whatever dashboards they are going to want to see when they are standing in line at the coffee shop in our hospital. Epic is actually getting into that in our next version, so we are excited to see whether we can put our dashboards into the palm of our hands so that the hospital president can see that there are 35 people waiting in the ED when he or she is about to get a coffee and go down there. For mobile versions, the key is having things in real time. People do not want to see something that is a month old or a week old. They want to see something that tells them where they need to go next. We are pretty excited about that. We use Power BI and some of Epic’s data tools. Outside of the custom dashboarding reports that we build in Clarity and Caboodle, if we are building anything custom that isn’t on the Radar dashboard and doesn’t need real-time data, we typically build it in Power BI.

Health Catalyst

We don’t use native visualization tools from Health Catalyst. We only have visualization tools in their prebuilt applications.


Information Builders has some kind of integration from a different company for geo-mapping, and we use that integration. It is seamless to us. There are some really good tools, but they aren’t preassembled in a way that we have been able to take advantage of. For example, we get HL7 messages from hospitals. That standard has been around long enough that I would have thought that Information Builders would have already had prebuilt templates showing what data is going to come in, where it is going to go, and how it is going to be mastered. In my experience, prebuilt templates are pretty common. We had a system full of templates that we tweaked and adjusted. We didn’t have to create anything from scratch. When we bought the system, we envisioned a process where physicians log in to the portal and pull up a patient’s full longitudinal record; we would see everywhere the patient had ever been with prefilled dashboard views. But the audit trail is not yet fully baked, and the user security or roles aren’t there yet. I would say that the system is prebuilt. Information Builders says we have an audit trail; we just can’t get to it because it is more made for a DBA person. But I need to be able to log in and pull an audit report when someone asks me to. I need to know when someone looks at a record they aren’t supposed to see. There are pieces that don’t exist yet. If they did, I would have said that the application was completely out of the box.

Innovaccer

We do cohort analysis for a bunch of different disease states. We have the ability to embed content through the InNote application. We have self-service analytics, but that was previously a pain point. Now, things are much better. We were on a different platform and switched, so there were some challenges with the transition. Now, the accessibility and reporting capabilities that are available to us in terms of self-service analytics are much improved, and we are excited about that piece.

62


We are trying to make sure that we have all of the pieces in place to do an ROI calculation. We are extremely committed to the importance of knowing the value that is being captured by both us and the vendor.

Innovaccer has some prebuilt healthcare applications, but we don’t use those as much. We use cohort analyses to a degree, but we do a lot more ad hoc things. Right now, Innovaccer is migrating to another BI tool, and that is a big improvement. The current tool is one of our biggest complaints with Innovaccer. We don’t use their platform to embed content into our EMR or other solutions, but there is that capability.

63


Are you using advanced analytics capabilities from this vendor?Alteryx

We have not utilized any of Alteryx’s really advanced functionality, except for the geospatial analysis. We have done a lot on that. And we also have used the API link to outside data language. Those two pieces have been helpful. We haven’t done the advanced model building. We typically do prescriptive analytics outside in a different tool. We also do NPL outside in another tool. Alteryx has that, but it is more on the guardrail, citizen data scientist side, and we just haven’t really pushed that side. The data science language is embedded in the workflow. We are using everything on the list except natural language processing. That is a brand-new function that Alteryx just came out within their intelligence suite. That is in its infancy compared to what we can do in Python. A lot of times, Alteryx’s system is compared to things based on what users can click and point to in the menus, and people completely forget that once users can work with a programming language inside a software solution, everything the program makes is opened up. NLP is oftentimes built with Python. Alteryx uses Python in the system, but there isn’t a great point-and-click interface for that. The geospatial functionality is actually very powerful because we can layer statistics into all of the data geospatial visualizations. In the healthcare space, we drive time analytics, such as how far the patient is driving to our facilities, how far all of our patients are from our key service locations, or whether we are missing services by geography based on both drive times and distance from the patients’ residences to our office. All of that type of information is on Alteryx’s system.

Our plan is to eventually use the system for advanced analytics, but we just haven’t had the opportunity to use it in that way yet.

Arcadia.io

We aren’t doing prescriptive analytics with Arcadia. We are not at that level of sophistication.

We are trying to get more regular access to the data experts from Arcadia.io. We meet frequently with them, and we often get to points where we could really use the help and insight of an Arcadia.io data expert in terms of how the data is structured in the database or how to best map the tables to get the reporting that we are trying to develop. Arcadia.io is working with us. In terms of the process and automation of advanced analytics, some things are prebuilt, and we just update those things. With the front-end dashboards, we can just assign those to an individual, and running them is pretty easy because the reports can be saved. The operationalization of analytics relies on the owner. We take on and learn the tool, and we start developing standard reports as well as cadences when we run the reports.

Cerner

If the vendor’s registry setup would be considered prescriptive, then we would use them for prescriptive analytics. They have a HEDIS certification. There isn’t an API link to embed outside data languages. To me, an API link would be used to take data in or out of the data platform. The tools that we can use in the HealtheAnalytics platform are query, data mart, and beta build tools. With HealtheEDW Advanced, we don’t need to use APIs anymore. We can set up jobs and then run them through another system. We can use any other visualization or statistical tools like Excel, Microsoft’s SQL Server Management Studio, or whatever we want. We can hook up anything.

We haven’t used a lot of predictive analytics, although the capabilities are there. We only look at those analytics in the IT department.

64


We are just starting to get into the predictive analytics space. We are not using machine learning. We are using automated model building and training. We are using geospatial analytics in very cool ways.

Clearsense

Our goal is to use predictive analytics 100%. That is why we want Clearsense. We definitely use their ML technology. We are building out some of the capabilities now. We recently got all the data from disparate data sources into the Clearsense platform, so we are starting to build out and use the capabilities. We aren’t using unsupervised ML yet, but I am sure that as the platform matures, we will. Automated model building and training will be something that we build out and mature. We are early in the life cycle. We do use Python and R, and we have HAPI FHIR. We currently have our sepsis grant work, our atrial fibrillation study, and our COVID-19 data on the platform.

Hypertension, chronic disease, and CKD were not good because we need to have some background in medical science to do such things. If we do not have this, programming will not be sufficient to do anything. We cannot build a cohort without knowing what we are doing. We haven’t actually reached the point to use unsupervised machine learning. We are still building appropriate data sets where we could use them for building models. They build very basic models that will not tell the risk factor or tell whether a patient will be getting a disease after 5 years or 10 years. This is something we already know about a patient, and we are telling through models whether that person has a disease or not. I am basically completely re-creating. We did not even touch those things because I did not find those models useful.

Dimensional Insight

The analytics and reporting capabilities are good; they just need to look nicer. Dimensional Insight’s tool for self-service BI is really, really good.

We are currently not using an API link to embed outside data language. I know it is there, but that is something we will explore more in the future.

Epic

We have used some of Epic’s predictive analytics models. We have not gone into some of their more advanced ones that would involve supervised or unsupervised learning. I am not sure geospatial analytics can be as simple as heat maps over COVID-19 vaccines and showing where people have gotten them if that is geospatial. That is just pretty simple geocoding. We are getting into that with Slicer Dicer.

Health Catalyst

Our embedding of outside data language is very small.

We have done a lot of work with predictive analytics. We are also using the AI component for supervised machine learning, some unsupervised machine learning, automated model building, and training. Prescriptive analytics depends on the definition, but we are generating worklists that target and stratify patients. We could call that prescriptive analytics. We aren’t up with NLP yet because we just got the unstructured data. I imagine within the next 6–12 months, we will have some stories to tell, but right now, we are just getting started.


I think the system has machine learning and predictive analytics. We are just not to the point in our implementation where we can turn those functions on. I don’t believe the system has AI, but I don’t know for sure.

65


Innovaccer

We haven’t used SPSS, Python, R, or any other commercial statistical tools. We typically export the data and then dump it into our own statistical modeling, so we don’t use an API. The system does have API capabilities; we just haven’t used them.

Innovaccer has a lot of capabilities; we just don’t use those capabilities. The issue is not on the vendor’s side.

66


Are you using underlying data and analytics components from this vendor?Alteryx

When it comes to the underlying components, we haven’t done anything with the FHIR API piece we have. The interoperability has been fine. We used the software for pulling in census tract data and comparing the census tract versus who we are actually giving the vaccine to see whether we are skewed in terms of race, ethnicity, or age group. We have done that part without much trouble. Security is a big animal for us. We do not use the Alteryx software at all to handle any of our security. That is mainly handled at the database level. We have used advanced functionality of analytics process automation, and it powers a good portion of our dashboards that are out there. That piece has been very helpful. It did take us a bit of time to train people on the tool, but it was still a heck of a lot faster than training people in SQL or any other program. People seem to pick up the Alteryx system really quickly. One of the good things about it is most people are using the basic functionality. But the fact that it can be a lot more complex is very helpful. It is hard to do an ROI when it comes to analytics because now the questions have doubled or tripled. I look at whether people are utilizing the tool and are getting the information that they want from utilizing the tool. From that perspective, I don’t have any complaints. I think if I tried to pull the solution out of the organization, that wouldn’t be a pretty picture without a very similar replacement coming in because people are really utilizing the system. When we looked at our analyst feed, the ROI typically says that if we are able to speed someone up, then we need fewer FTEs. We also seem to be hiring analysts, so it is kind of hard to say we are losing people. But the solution probably speeds up our analysts by at least 20%–30% from that perspective.

I don’t think that we use any security features in Alteryx’s system. We are doing security through another product. I actually like how Alteryx’s system handles security by interacting with another security program. For example, when we pull up a SQL server environment through Alteryx’s system, it only shows the databases that we have access to in the Active Directory. That is great. If I were doing the work in the SQL server management studio, I would be seeing every single database, and then I would have to select the database before the system tells me that I don’t have access privileges. That is a big difference from other systems. We don’t want to go through thousands of lines of tables to find the two tables that we have access to. I like how Alteryx’s system interacts with security software.

For data masking, the system has been very simple to use. We had a big project where we had to mask some data recently, and it was very simple to do within the Alteryx system.

Arcadia.io

Arcadia has security controls. They are separate user credentials required for the back-end analytics platform. At the front-end user interface level, we can limit access based on the roles as well as the population within the platform. With accessing the raw data on the back end, we can limit access as well because it is a different process to get access. Arcadia works very closely with us to make sure that we know how to use the data in the platform. We don’t do data masking or anonymization in Arcadia. That is already done in the raw data before it goes into Arcadia.

We are using security. That is a requirement for any vendor. From a security perspective, we are able to monitor who is accessing the tool and who is looking at patient-level data. We have an employee plan in the tool, and we are able to block users from seeing members that are colleagues. The platform is very secure and allows us to share PHI at a practice level. We always struggled to share that data with the independent providers. Arcadia.io’s tool is a big opportunity.

Cerner

The whole platform is about trying to combine multiple data sources into one place.

67


Clearsense

APIs will happen between now and the end of the year. The security roles are in play right now, and we are developing those with Clearsense. We will have role-based security using their capabilities, which will be integrated with Microsoft’s Azure software. For data masking and anonymization, we will use Clearsense’s Observational Medical Outcomes Partnership (OMOP) capabilities. We have raw data on the platform as well; depending on the study or grant, people want either OMOP data or raw data. We do as much data masking as we can. There is a mix right now, but we will use Clearsense’s platform for the monetization of data. We will partner with other folks as well to ensure we are sending appropriately secured data for different studies and partnerships. The vendor doesn’t have security protocols at the data level. We have actually told them that if we want to give access to someone in a SQL database, that person will have to view all of the databases. What I requested was that if that user logs in to the database, they should not have a view of the other databases. We have a data set that we could use to actually look at the columns and see what we need to make this user view.

Dimensional Insight

We don’t use the deidentification function often, but we do use it.

Epic

We have FHIR enabled, and that could be considered an API. Our patients can tap into our system; that is part of the regulations. The Epic EMR does have FHIR and APIs enabled. We use interoperability in the application but not so much in data analytics yet. Any data that comes into Epic’s system that gets reconciled into the chart from interoperability will show up in my databases. So if a clinician sees an outside medication and reconciles it into the patients’ charts, that will come into my data and analytics sphere. It is very controlled and centralized.

Health Catalyst

Data masking is inherent in some of the data use.

Health Catalyst talks about their software being FHIR and API enabled, but we aren’t using any of those things. I am not sure that our version of the product is ready for FHIR yet. We are maybe a version or two away from that. We aren’t doing any FHIR work at this time in the data warehouse.


The Information Builders software is FHIR enabled, but we aren’t using that capability yet in our data warehouse. We haven’t tried to use it. Information Builders is a big integration company, and they have all kinds of APIs. They have other clients, like HIEs, who use that capability. Whenever we need to mask data, we do that through our own means. Information Builders doesn’t automatically generate a model that deidentifies data.

Innovaccer

Innovaccer does their own platform ROI calculations. They don’t have a tool I can use that spits out data, but they quantify the system’s value occasionally by saying how many dollars we have saved by using the system for certain things.

68


Additional commentsAlteryx

We have internal domain expertise, so we use Alteryx’s system as a tool to provide content. Alteryx’s system is great at finding data that was previously hidden and making it visible. Alteryx’s system has assisted machine learning as a functionality in the intelligence suite, and that is an add-on to the designer license. The functionality takes a nonstatistician or a non-data scientist through the steps of how to build even a simplistic machine-learning model through a tutorial interface. That is huge. We actually trained analysts that weren’t statisticians, and they now do assisted machine learning. The only nuance is what the system won’t do, which is not telling someone how to interpret the model. We provide the education and internal training so that even someone who doesn’t know mathematical theory or statistics can do some of the work in a simple way. There is a standardized education workflow to learn the system, train on the system, and use it to support our company. Analytics process automation has two parts. One is Alteryx Server, which is a product that does workflow automation and scheduling in the same way we would in a SQL server. The other part is how Alteryx’s system provides analytics process automation. Because the workflow is visual, we can take steps between the different programmatic functions that exist within the tool set. The tools are a bunch of icons, but the icons are really just macros of the programming language. Alteryx built most of the software off of R programming on the back end. Alteryx’s system is future forward. The system allows the end user to see and modify the source code, and that is something most platforms don’t do. With process automation, we have triggers, alerts, and emails when an automatic workflow process goes wrong within a step of a process. If there is an error because of an out-of-reference range, the system flags the error and automatically stops the writing process to wait for us to manually fix the error. That is great for an automatic workflow. We can just continue through if we don’t have an error. We can customize the alerts too. Alteryx is exploding in the industry, but that may not be visible to most people yet. When Alteryx grew and developed all of the technology, they were a small company. Then they realized that they had a gold mine of a product when all of the other industries were taking off. Healthcare is always the last industry to take off. Alteryx couldn’t speak the language and didn’t have the subject matter expertise that a data scientist would need to specialize in healthcare. Alteryx built a whole healthcare vertical two years ago. They recruited a whole division to serve the healthcare industry. Within the division, Alteryx nested experts recruited from EHR companies so that the vendor would have the domain knowledge and could interface with doctors, business leaders, and end users using the system. Because the software solution was deployed behind the vendor’s firewall, the system was immediately HIPAA compliant. Alteryx’s solution is one of the key software products that we can use that hits all of the privacy regulations. Alteryx has been growing quickly. They are probably struggling to keep enough sales representatives to keep up with the healthcare side of things. Alteryx provides a tool to allow the data scientists to do the work in-house. We brought development in-house, and we stopped paying a lot of money to third parties. We hit the economy of scale internally. Other vendors say that we don’t have the talent or skill set to build the infrastructure, so they do the work for us. Alteryx is different. Other organizations are trying to get a prebuilt or automated product because they don’t have the skill sets to run it. As the organization grow and start investing in analytics, they start bringing in tools like Alteryx’s system and start seeing synergies and overlaps between products. Where we have gotten some of the best return on engagement from our staff wasn’t necessarily with the advanced programmers even though they were all happy with Alteryx’s system. The best return was with our analytics staff members that just knew Excel. They were spending 30 hours a week in data preparation in Excel; now they only spend two hours a week. That is transformational. Just two weeks of training on Alteryx’s system now saves 28 hours of time per person in the position.

Arcadia.io

We have to set things up by department practices. Because of our structure, we have a hierarchy built where we can look at things from a system-wide view or a regional view. Within the tool, we can also drill down into the provider level. Right now, most people using the tool are administrators, people on the quality team, people on the care management team, leaders, and regional medical directors.

69


We have started rolling the Arcadia.io tool out to provider practices because we can limit access. We can give people access to just their TIN. The tool is in the ambulatory space, and we are able to see all the inpatient claims and outpatient claims for the payers. On the front end, we are using Arcadia.io’s tool to prepare, cleanse, and enrich data. That is the nature of the tool. It digests all the data and packages it for us. There are some EHRs that allow providers to manually satisfy measures by just putting in a date of service, even though in reality, there isn’t anything to justify that date of service. In those cases, the measures don’t show as satisfied in the Arcadia.io tool. The vendor is NCQA certified, so they have to go through a ton of checks and balances in order to have the measures deployed. Arcadia.io definitely meets the requirements. Looking at the data in their system really gives us a true understanding of real data. We are diving into the back end a bit more deeply.

Cerner

[No additional comments provided by respondents.]

Clearsense

We had started an attempt to create a data warehouse using a very well-established formal data warehouse, and that failed. The reason that failed is that healthcare data is often not in the most perfect form. What we were looking for was not just the ability to store a data warehouse up there but also the ability to work with data that may not be perfectly formed, semistructured, or unstructured. We haven’t gone into that area yet, but we have the ability to make the data available to those who have the skills to work with it while simultaneously giving us the ability to curate it such that we can use it in a self-service analytics fashion. That has played out well. Right now, the system is departmental. The purpose of the system was to give various departments access to the data that they care about in such a way that one person’s query will not bring the server to the ground like it would if we were to have a SQL server. I want to say that it is departmental, but eventually, it is going to be enterprise in that a growing number of departmental analytics folks are going to be using it.

Dimensional Insight

Diver Platform has the potential to use advanced analytics. We just don’t at the moment. Our team specifically is more focused on the business flow and the business side and on getting everything that they need. As a hospital system, I don’t think that our team is ready for that. But we do have another group that is supposed to be focused on that. I believe that team is using things like SaaS and probably Python to try to do that. I am not sure how accurate they are.

Epic


Health Catalyst

We are getting JSON files from a non-SQL database that is supporting one of the vendors that we work with. Those files are new for us, and the capability is also new to our version of Health Catalyst Data Operating System. In the past, we had to take the files, convert them to XML files, and ingest them that way. There were multiple steps to the process; it wasn’t as efficient as it could have been.



70


Innovaccer

We are doing a lot of things with Innovaccer because we have seen results from them. We will continue to use the system for the foreseeable future because it has some capabilities that our other vendors’ systems don’t have. A lot of those capabilities have to do with point-of-care abilities for physicians. The system also has some unique risk-coding capabilities that we haven’t seen from other vendors’ systems. Innovaccer is pretty progressive in terms of reporting. The system obviously has a bunch of canned reports, just like every other vendor’s system in the space. But we are a bit needy, and we ask Innovaccer to do a lot of custom reports for us with different visualizations. We have frequent meetings with the vendor where we talk about the reports and other things we need. Our team has appreciated Innovaccer’s overall approach to analytics, reporting, scorecards, and dashboard development in that we have the capability to serve ourselves. One of the benefits of the company is that Innovaccer is both domestically and internationally based. They work what feels like 6 days a week and 14–15 hours a day because of the differences in where people live. The sun hardly ever sets on the company, and when we put in a request, it is generally turned around really quickly. Innovaccer adopted a new project style probably within the last 18 months. It is like an Agile workflow. The vendor does sprints to get things done and delivered to us really quickly. If we don’t like the vendor’s work, we can turn it back around and they can do another sprint to get it back to us. There are a lot of iterative processes to get exactly where we would like to be as opposed to the traditional project management workflow where we would spend 3–4 weeks building something that is not to our liking, and then we have to undo and redo a bunch of things. The sprint work has been helpful for us. We are doing a lot of things around care coordination and care management to determine how impactful we are. Everyone knows that the top X percent drives the majority of population costs and utilization. But by going a level deeper, we have become a more sophisticated and value-based shop. We want to not only focus on patients that bring us the highest yield but also know what programs and care managers are delivering the best value. Certain people may be better at certain things than other people. I can use analytics to show things that are statistically significant, and we can make management decisions based on those analytics. We just have to be resource conscious and do the things that will get us the biggest bang for our buck.

Documents

DATA AND ANALYTICS PLATFORMS 2021