Results of a qualitative survey on the application of ... · 3/26/2019 · Results of qualitative survey on the application of CSV in EU SMEs of the Medical Device Industry | Marius

Results of a qualitative survey on the application of Computer System Validation in European SMEs of the Medical Device Industry

First findings of an ongoing research project

The Future of Entrepreneurship: Policy and Practice8-9 April 2019, Paris, France

Marius Schönberger

Prof. Dr. Tatjana Vasiljeva

Results of qualitative survey on the application of CSV in EU SMEs of the Medical Device Industry | Marius Schönberger & Tatjana Vasiljeva 11.04.2019 2

Agenda

1. Introduction

2. Description of the problem

3. Related work

4. Research steps and methodology

5. Research findings

6. Limitation and outlook


Introduction

Software has become a significant economic factor

Software has thus become a significanteconomic factor not only in business,

science and technology, but also in the healthcare sector.

Medical device manufacturers

Software is necessaryto build devices that help patients lead

better lives.

With the benefits of software come the risk

of defects and bugs.

Software products are increasinglybecoming the central component of

complex electronic devices that controlor support technical or business

processes.

Source: Schönberger & Vasiljeva, 2018


Introduction

Software failures in the medical device industry

Source: Fu, 2011; AAMI, 2016

From 2002 to 2010, medical devices based on software resulted in over 537 recalls affecting more than 1.5

million devices.

11.3% caused by software failures

Medical devices are often subject to a high number of errors with

potentially catastrophic effects on the patients.

Medical errors are involved in more than 250,000 deaths each year in the US andaccount for nearly one in ten deaths.


Description of the problem

The need for computer system validation (1/2)

With the current technological possibilities of the software development …

… a considerable increase in the complexity of the devices and …

… major challenges for the reliability, patient safety and

security emerges.

The computer system validation are major means of avoiding defects and resultant recalls and is a requirement of the quality system for medical device manufacturers

Source: Fu, 2011; Alemzadeh et al., 2013; Bhusnure et al., 2015; FDA, 2002


Description of the problem

The need for computer system validation (2/2)

Source: FDA, 2002; Hrgarek, 2008

The implementation of computer system validation is necessary due to …

Legal regulations Economic, social and technological aspects

Problem: The regulations only determine that a computer system validation has to

be carried out, the exact scope of the validation as well as a structured approach

are not specified.

Challenge: All medical device manufacturers shall determine what needs to be validated and how muchvalidation is enough to ensure that regulatory requirements are met.


Related work

Approaches to Computer System Validation (1/2)

Author(s) Year Title Type Focused sector

Bendale et al. 2011 Computer software validation in pharmaceuticals Journal Pharmaceuticals

Bhusnure et al. 2015 Computer validation and ethical security measures for pharmaceutical data processing Journal Pharmaceuticals

Charan and Vishal Gupta 2016 GAMP 5: A quality risk management approach to computer system validation Journal Medical Device Industry

Esch et al. 2007 Good Laboratory practice (GLP) – Guidelines for the Validation of Computerised Systems Journal Laboratories

European Commission 2015 EU guidelines for good manufacturing practice for medicinal products for human and veterinary use EU Guideline Medical Device Industry

FDA 2002 General principles of software validation; Final guidance for industry and FDA staff US Guideline Medical Device Industry

Hrgarek 2008 A management approach to software validation requirements Proceeding Medical Device Industry

Huber 2005 Qualification and validation of software and computer systems in laboratories Monography Laboratories

McDowall 2016 Welcome to the brave new world of CSV? Journal Pharmaceuticals

Tracy and Nash 2002 A validation approach for laboratory information management systems Journal Laboratories

von Culin 2011 New approach to system validation Journal Medical Device Industry

Yogesh et al. 2015 Computer system validation: A review Journal Pharmaceuticals

Source: Schönberger, 2018


Related work

Approaches to Computer System Validation (2/2)



Related work

Conclusions from the related work

No references to Small and medium enterprises 1

No differences to the general approach2

No description of recommendations for action3



Research steps and methodology

Research questions

What is the status quo in European SMEs with regard to the implementation and application of the CSV?

RQ 1

RQ 2Do the results of the study reveal commonalities and differences in the implementation

and application of CSV?

Does the implementation and application of CSV have an impact on the firm performance of SMEs?

RQ 3



Conceptual model

Source: Schönberger and Vasiljeva, 2019



Development of measures for the questionnaireConstructs

DimensionsMeasures and reference to literature Items

IT assets (tangible)

Flexible IT infrastructure (FIT)Adopted from Van de Wetering et al., 2017; Garrison et al., 2015; Rockmann et al., 2015; Liu et al., 2013; Chen andWu, 2011; Kim et al., 2011; Prasad et al., 2010; Saraf et al., 2007; Ray et al., 2005; Byrd and Turner, 2001; Byrd andTurner, 2000.

10

Business applications (BAP)Adopted from Chen and Tsou, 2012; Chen and Wu, 2011; Kim et al., 2011; Pavlou and El Sawy, 2006; Chung et al., 2003; Byrd and Turner, 2000.

6

IT capability (intangible)

Human IT resources (HIT)Adopted from Chen and Wu, 2011; Kim et al., 2011; Chung et al., 2003; Byrd and Turner, 2001; Byrd and Turner, 2000.

6

IT management capability (ITM) Adopted from Chen et al., 2015; Kim et al., 2011; Bharadwaj et al., 1999. 6

Computer System Validation (CSV)

CSV approach (CAP) Newly developed based on indications of Elser, 2016; Dehghan-Dehnavi et al., 2015. 6

Risk assessment (RAS)Adopted from Dehghan-Dehnavi et al., 2015; Zou et al., 2009.Newly developed based on indications of McCaffery et al., 2010; Burton et al., 2006.

7

Firm performance

Product quality (PRQ) Adopted from Eikebrokk and Iden, 2017; Van de Wertering et al., 2017; Liu et al., 2013; Chen and Tsou, 2012. 2

Number of recalls (NOR)Adopted from Van de Wetering et al., 2017.Newly developed based on indications of Santos and Brito, 2012.

2

Competitive advantage (CAD) Adopted from Eikebrokk and Iden, 2017; Queiroz et al., 2017. 2


Research findings

Findings of the pre-test of the questionnaire (1/2)

Similarities

Characteristics Case study 1 Case study 2 Case study 3 Case study 4 Case study 5

Medical products Hearing protectionInjectors for imaging

methodsNutritional pumps Equipment for endoscopy Compression stockings

Respondents 1 CEO and 1 IT manager 1 CIO and 1 IT administrator 1 IT manager1 CEO and

2 IT administrators1 CIO

Staff 20 employees 146 employees 80 employees 72 employees 65 employees

Self-assessmentof IT skills

good very good very good good good

Employees are responsible for the maintenance of ICT within

the companies

Sufficient experience about the ICT used within the

companies could be recorded

All companies are located in the German-speaking area and

share cultural similarities

0

1

2

3

4

FIT01

FIT02

FIT03

FIT04

FIT05

FIT06

FIT07

FIT08

FIT09

FIT10

Diagrammtitel

Case study 1 Case study 2 Case study 3 Case study 4 Case study 5


Research findings

Findings of the pre-test of the questionnaire (2/2)

Results for Flexible IT Infrastructure (Selection)

Dimensions Case study 1 Case study 2 Case study 3 Case study 4 Case study 5

FIT01 2 4 3 3 4

FIT02 3 4 2 3 3

FIT03 1 3 3 4 2

FIT04 2 4 3 4 4

FIT05 2 3 4 3 3

FIT06 1 3 2 3 4

FIT07 4 4 3 4 3

FIT08 1 3 2 3 3

FIT09 2 4 2 2 3

FIT10 2 3 3 2 4

Note: All items use five-point Likert scales anchored at 1 (strongly disagree) and 5 (strongly agree).


Research findings

Evaluation of the case studies (Summary)

The work with ICT is absolutely necessary in all enterprises1

Overall, neither positive nor negative changes in the firm's performance could be registered6

Only a few enterprises stated that their IT infrastructure allows for rapid changes2

Only one enterprise has a risk-based CSV approach which is complaint to current regulations5

Overall, the IT personnel has a good knowledge about their IT systems and business applications3

Almost all enterprises lack in the ability to manage their IT resources to deliver business value4


Research findings

Conclusions (based on the case studies)

What is the status quo in European SMEs with regard to the implementation and application of the CSV?

RQ 1

RQ 2Do the results of the study reveal commonalities and differences in the implementation

and application of CSV?

Does the implementation and application of CSV have an impact on the firm performance of SMEs?

RQ 3

The majority of enterprises do not have CSV approaches. A risk analysis of the IT systems being used is also predominantly lacking.

Since only one company has a documented CSV process, this question cannot be answered at this time.

In all enterprises participating in the pretest, no significant changes in firm performance could be identified.


Limitations and outlook

Limitations of the research

The research does not addressthe validation of software in

medical products.

The legal regulations mentioned within the research are not

examined more closely for their necessity or correctness.

The research focusses on SMEsin the medical device industryand, therefore, does not claim

to be universally valid.

Such examinations are rather located in the field of software

engineering.

These regulations are rather regarded as general laws

according to which companies in the medical device industry

have to comply.

The application in largecompanies as well as in otherindustries, e.g. the pharmacy

industry, must first be checked.


Limitations and outlook

Further research steps and overall research methodology

Phase Process step Method and approach

Problemidentification State of research

Survey of medical devicemanufactures

Data collection and validation of the results

Expertinterviews

Impact of the computer system validation on the firm performance of small and medium enterprisesin the medical device industry

Development of a risk-basedmodel for CSV

Development and implementation

Evaluation of the resultsExpert

interviews

Systematic literature review

Questionnaire technique / pretest

Guided interview

CSV approaches / methodsRisk management approaches / methods

Conditions of international standards

Guided interview

Results of a qualitative survey on the application of Computer System Validation in European SMEs of the Medical Device Industry

First findings of an ongoing research project

The Future of Entrepreneurship: Policy and Practice8-9 April 2019, Paris, France

Marius Schönberger

Prof. Dr. Tatjana Vasiljeva


References

AAMI (2016). Medical errors involved in nearly 1 in 10 U.S. deaths, study says. Association for the Advancement of Medical Instrumentation, 10 May 2016, http://www.aami.org/newsviews/newsdetail.aspx?ItemNumber=3612, accessed 17 February 2019.

Alemzadeh, H., Ravishankar, K. I., Kalbarczyk, Z., & Raman, J. (2013). Analysis of Safety-Critical Computer Failures in Medical Devices. IEEE Security and Privacy Magazine, 11(4), 14-26.

Bendale, A., Patel, N., Damahe, D. P., Narkhede, S. B., Jadhav, A. G., & Vidyasagar, G. (2011). Computer software validation in pharmaceuticals. Asian Journal of Pharmaceutical Sciences and Clinical Research, 1(2), 27-39.

Bharadwaj, A. S., Sambamurthy, V., & Zmud, R. W. (1999). IT capabilities: theoretical perspectives and empirical operationalization. Proceedings of the 20th International Conference on Information Systems (ICIS), Charlotte, NC, United States, 1999, 378-385.

Bhusnure, O. G., Kendre, S. P., Kaudewar, D. R., Bankar, I. N., Gholve, S. B., & Giram, P. S. (2015). Computer validation and ethical security measures for pharmaceutical data processing. World Journal of Pharmaceutical Research, 5(1), 1092-1134.

Burton, J., McCaffery, F., & Richardson, I. (2006). A risk management capability model for use in medical device companies. Proceedings of the 2006 international workshop on Software quality. Shanghai, China, May 21-21, 2006, 3-8.

Byrd, T. A., & Turner, D. E. (2000). Measuring the Flexibility of Information Technology Infra-tructure: Exploratory Analysis of a Construct. Journal of Management Information Systems, 17(1), 167-208.

Byrd, T. A., & Turner, D. E. (2001). An exploratory examination of the relationship between flexible IT infrastructure and competitive advantage. Information & Management, 39(1), 41-52.

Charan, H. Y. and Vishal Gupta, N. (2016) ‘Gamp 5: A quality risk management approach to computer system validation,’ International Journal of Pharmaceutical Sciences Review and Research, 36(1), 195-198.

Chen, J. S., & Tsou, H. T. (2012). Performance effects of IT capability, service process innovation, and the mediating role of customer service. Journal of Engineering and Technology Management, 29(1), 71-94.

Chen, Y. C., & Wu, J. H. (2011). IT management capability and its impact on the performance of a CIO. Information & Management, 48(4-5), 145-156. Chung, S. H., Rainer Jr, R. K., & Lewis, B. R. (2003). The impact of information technology infrastructure flexibility on strategic alignment and application implementations. The

Communications of the Association for Information Systems, 11(11), 191-206. Dehghan-Dehnavi, M., Dinarvand, R., Khoshayand, M., Kia, F., Maghazei, M., & Dorkoosh, F. A. (2015). Evaluation of Validation Plans and Validation Activities in Iranian

Pharmaceutical Industries. Journal of Pharmacoeconomics and Pharmaceutical Management, 1(1/2), 32-38. Eikebrokk, T. R., & Iden, J. (2017). Strategising IT service management through ITIL implementation: model and empirical test. Total Quality Management & Business Excellence,

28(3-4), 238-265. Elser, C. (2016). Validation master plans: Progress of implementation within the pharmaceutical industry (Doctoral dissertation, University of Southern California). Esch, P. M., Donzé, G., Eschbach, B., Hassler, S., Hutter, L., Saxer, H. P., Timm, U., & Zühlke, R. (2007). Good Laboratory Practice (GLP)–guidelines for the validation of computerised

systems. The Quality Assurance Journal, 11(3‐4), 208-220.


References

European Commission (2015). EU Guidelines for Good Manufacturing Practice for Medical Products for Human and Veterinary Use, Annex 15: Qualification and Validation. Brussels, 30 March 2015.

FDA (2002). General Principles of Software Validation. Final Guidance for Industry and FDA Staff. Rockville, 11 January 2002. Fu, K. (2011). Trustworthy medical device software. Public Health Effectiveness of the FDA 510(k) Clearance Process: Measuring Postmarket Performance and Other Select Topics,

Workshop Report, Washington, D.C., July 2011. IOM (Institute of Medicine), National Academies Press, 97-118. Garrison, G., Wakefield, R. L., & Kim, S. (2015). The effects of IT capabilities and delivery model on cloud computing success and firm performance for cloud supported processes

and operations. International Journal of Information Management, 35(4), 377-393. Hrgarek, N. (2008). A management approach to software validation requirements. Proceedings of the Central European Conference on Information & Intelligent Systems (CECIIS)

2008. Varaždin, Croatia, September 24-26, 219-226. Huber, L. (2005). Qualification and validation of software and computer systems in laboratories. Part 3. Installation and operational qualification. In: De Bièvre, P., & Günzler, H.

(eds.). Validation in Chemical Measurement, Berlin, Heidelberg: Springer, 140-144. Kim, G., Shin, B., Kim, K. K., & Lee, H. G. (2011). IT capabilities, process-oriented dynamic capabilities, and firm financial performance. Journal of the Association for Information

Systems, 12(7), 487-517. Liu, H., Ke, W., Wei, K. K., & Hua, Z. (2013). The impact of IT capabilities on firm performance: The mediating roles of absorptive capacity and supply chain agility. Decision Support

Systems, 54(3), 1452-1462. Mc Caffery, F., Burton, J., & Richardson, I. (2010). Risk management capability model for the development of medical device software. Software Quality Journal, 1(18), 81-107. McDowall, R. D. (2016). Welcome to the Brave New World of CSV?. Advances in Pharmaceutical Analysis, 29(2), 93-96. Pavlou, P. A., & El Sawy, O. A. (2006). From IT leveraging competence to competitive advantage in turbulent environments: The case of new product development. Information

Systems Research, 17(3), 198-227. Prasad, A., Heales, J., & Green, P. (2010). A capabilities-based approach to obtaining a deeper understanding of information technology governance effectiveness: Evidence from

IT steering committees. International Journal of Accounting Information Systems, 11(3), 214-232. Queiroz, M., Tallon, P. P., Sharma, R., & Coltman, T. (2017). The role of IT application orchestration capability in improving agility and performance. The Journal of Strategic

Information Systems, 27(2018), 4-21. Ray, G., Muhanna, W. A., & Barney, J. B. (2005). Information technology and the performance of the customer service process: A resource-based analysis. MIS quarterly, 29(4),

625-652. Rockmann, R., Weeger, A., & Gewald, H. (2015). IT Capabilities and Organizational Utilization of Public Cloud Computing. Proceedings of the 23rd European Conference on

Information Systems (ECIS), Münster, Germany, May 26-29, 2015, 1-19. Santos, J. B., & Brito, L. A. L. (2012). Toward a subjective measurement model for firm performance. Brazilian Administration Review, 9, 95-117. Saraf, N., Langdon, C. S., & Gosain, S. (2007). IS Application Capabilities and Relational Value in Interfirm Partnerships. Information Systems Research, 18(3), 320-339. Schönberger, M. (2018). Comparison and evaluation of procedures for the computer system validation in the European medical device industry. Proceedings of the 31st

International Business Information Management Conference (IBIMIA), 2018, April 25-26, Milano, Italy.


References

Schönberger, M., & Vasiljeva, T. (2018). Impact of the computer system validation on the firm performance of small and medium enterprises in the medical device industry. Proceedings of the 31st International Business Information Management Conference (IBIMIA), 2018, April 25-26, Milano, Italy.

Schönberger, M., & Vasiljeva, T. (2019). Determinants and impacts of the computer system validation on firm performance: A literature review. Proceedings of the 12th Annual Scientific Baltic Business Management Conference (ASBBMC), 2019, February 21-23, Riga, Latvia.

Tracy, D. S. and Nash, R. A. (2002). A Validation Approach for Laboratory Information Management Systems. Journal of Validation Technology, 9(1), 6-14. Van de Wetering, R., Mikalef, P., & Pateli, A. (2017). A strategic alignment model for IT flexibility and dynamic capabilities: Towards an assessment tool. Proceedings of the 25th

European Conference on Information Systems (ECIS), Guimaraes, Portugal, June 5-10, 2017, 1468-1485. Von Culin, R. (2011). New Approach to System Validation. Applied Clinical Trials, 20(2), 32-37. Yogesh, P., Kamlesh, M., Mohini, B., Phad, R., Ismail, S., & Shivam, L. (2015). Computer system validation: a review. World Journal of Pharmaceutical Research, 4(9), 444-454. Zou, P. X., Chen, Y., & Chan, T. Y. (2009). Understanding and improving your risk management capability: Assessment model for construction organizations. Journal of Construction

Engineering and Management, 136(8), 854-863.

Documents

Results of a qualitative survey on the application of ... · 3/26/2019 · Results of qualitative survey on the application of CSV in EU SMEs of the Medical Device Industry | Marius