Significance of Software Metrics in

ERP ProjectsS. Parthasarathy, and N. Anbazhagan

Abstract--Software engineering metrics are units ofmeasurement that are used to characterize the softwareengineering products and processes. Enterprise ResourcePlanning (ERP) is a generic term for integrated systems. ERPprojects are a subclass of software projects often accompanied bylarge-scale organizational changes. One of the major reasons

found in the literature for the failure of ERP projects is the poormanagement of software processes. In this paper, we propose a

Software Metrics Plan (SNP) containing different softwaremetrics to manage software processes during ERPimplementation. Two hypotheses have been formulated andtested using statistical techniques to validate the SNP. Thestatistical analysis of the collected data from an ERP projectsupports the two hypotheses, leading to the conclusion that thesoftware metrics are momentous in ERP projects.

Index Terms-- Enterprise resource planning (ERP), softwaremetrics, software process improvement

I. INTRODUCTION

USINESS information system is an area of the greatestsignificance in any business enterprise today. Enterprise

Resource Planning (ERP) projects are a growing segment ofthis vital area. Software engineering metrics are units ofmeasurement used to characterize the software engineeringproducts and processes [1]. ERP is a generic term forintegrated systems. ERP projects are a subclass of softwareprojects and often accompanied by large-scale organizationalchanges [2]. The research about the software process hasacquired great importance in the last few years due to thegrowing interest of software companies in the improvement oftheir quality. Enterprise Resource Planning (ERP) projects are

very complex products, and this fact is directly linked to theirdevelopment and maintenance.

Enterprise Resource Planning (ERP) is primarily an

enterprise-wide system, which encompasses corporatemission, objectives, attitudes, beliefs, values, operating style,and people who make the organization. ERP covers thetechniques and concepts employed for the integratedmanagement of businesses as a whole, from the viewpoint ofthe effective use of management resources, to improve the

S. Parthasarathy is with the Department of Computer Science &Engineering, Thiagarajar College of Engineering, Madurai, Tamil Nadu,INDIA (e-mail: partha s79 oyahoo.com).

N. Anbazhagan is with the Department of Mathematics, ThiagarajarCollege of Engineering, Madurai, Tamil Nadu, INDIA (e-mail:anbazhagn n oyahoo.co.in). 1-4244-0370-7/06/$20.00 2006 IEEE

efficiency of an enterprise. Software projects generally exhibitvariable returns to scale, and the output from ERP projects ismultivariate. ERP projects are a subclass of software projects[2]. Key issues in ERP projects are the larger project durationdue to schedule slippage, poor matching between thecompany's requirements and the ERP system and less supportfor maintenance from the ERP vendor [3] [4] [5]. If usedproperly, software engineering metrics allow us to (i)quantitatively define success and failure, and/or the degree ofsuccess or failure of a product; (ii) identify and quantifyimprovement, lack of improvement, or degradation in ourproducts and processes; (iii) make meaningful and usefulmanagerial and technical decisions [6].A large body of literature on large scale information system

projects like ERP projects has been developed during the pastseveral decades [7] [8] [9]. Apart from the managerial factors,major reasons found in the literature for the failure of ERPprojects are the poor management of software processes. Thisrefers to increased project duration due to schedule slippage,mismatch between the company's requirements and the ERPsystem, less support provided by the ERP vendor duringmaintenance. To overcome this failure, we propose a SoftwareMetrics Plan (SMP) containing different software metrics tomanage software processes during ERP implementation.These software metrics will help the ERP project teammembers to keep track oftheir software processes during ERPimplementation.SMP can be extended further to include other aspects as

well. Different projects may require different levels of effort,resources and expertise. Two hypotheses were formulated andtested using the statistical techniques correlation and multiplecorrelations [14] [15] to validate SMP. The statistical analysison the data from an ERP project supports the two hypothesesof the study, leading to the conclusion that the softwaremetrics are momentous in ERP projects.

II. PROBLEM DEFINITIONThe total quality management (TQM) notion of prevention

rather than correction can be applied successfully in softwareengineering. As discussed earlier, ERP software processimprovement requires special consideration and the crucialfactors namely requirements stability, schedule slippage andtracking problems during maintenance are to be measuredusing software metrics in the proposed SMP. Project scheduleslippage and tracking problems during maintenance are not

Authorized licensed use limited to: CALIF STATE UNIV NORTHRIDGE. Downloaded on April 27, 2009 at 17:36 from IEEE Xplore. Restrictions apply.

uncommon. A key issue in Enterprise Resource Planning(ERP) implementation is how to find a match between theERP system and an organization's business processes byappropriately customizing both the system and theorganization [3]. This is badly affected due to the instability inthe requirements proposed by the customer and the poor

capability of the ERP vendor (ERP development team).The phase "Gap Analysis" in ERP implementation is the

step of negotiation between the company's requirements andthe functions an ERP package possesses [10]. Poorrequirements specification and its instability badly affect thegap analysis phase of an ERP project which in turn leads toschedule slippage and bubbles of problems during themaintenance phase of the software project. The failure ofmany software projects can be directly linked to the poor

requirements specification [11]. Even for those companies thathave successfully implemented large-scale informationsystems projects in the past, ERP implementation still presentsa challenge, because it is not simply a large-scale softwaredeployment exercise. Also as ERP implementation is oftenaccompanied by large-scale organizational changes, agilesoftware processes could not create much impact on the ERPprojects [16] [17]. Figure 1 shows the gap analysis phase inERP implementation. Our requirement is that the ERPpackage must exactly meet the company's requirements toensure successful implementation.

Company's ap Aalysis EPPackage

Requirement

Fig. 1. Gap Analysis Phase in ERP Implementation.

III. SOFTWARE METRICS PLAN (SMP)Software processes have an important influence on the

quality of the final software product, and for this reason

companies are becoming more and more concerned aboutsoftware process improvement. The successful management ofthe software process is necessary in order to satisfy the finalquality, cost and time of the marketing of the softwareproducts [12]. To improve the software processes [13], a greatvariety of initiatives have arisen like CMM, CMMI etc. Allthese initiatives focus on software processes for developing,implementing or improving a quality management system.ERP projects involve software processes that will exist even

after the implementation of the ERP system. In fact, the actualprocesses start only when the going alive is fixed duringimplementation.

Traditional software projects have requirements collectedfrom the customers for which the software is developed. Butin ERP projects, the software is made available to thecustomers readymade after fine tuning its functionality tomatch the exact requirements of the customers. The purpose ofSMP is to enhance the performance of ERP projects byimproving their software processes. The SMP will have a setof well-defined metrics that will deal with measuring therequirements stability, project schedule slippage and trackingthe problems arising during the maintenance phase. SMP can

also be further extended involving other aspects like effortdistribution, productivity etc if desired. SMP will contain thefollowing: (1) Source of data and how it will be captured (2)Periodicity of data capture (3) Formulae (4) Responsibility (5)Measurement process (6) its significance to the project.

TABLE I

Table I shows the various metrics in the SMP. In Table 1,

the actual no. of days is the difference between the date whena particular phase/project got completed and the date when a

particular phase/project started. Estimated no. of days is thedifference between the planned start date of a particularphase/project and the planned end date of a particularphase/project. Source of data for the SS metric is the projectschedule actually prepared. The schedule slippageresponsibility lies with the project manager. Analysis of SScould be done during every phase of the software project. SSis used to complete the project on time and to avoid scheduleslippage. Source of data for RSI metric is the initial set ofrequirements specified by the client together with the no. ofchanges made by him. The project manager is held responsiblefor higher value of RSI. This must be done during each phaseof the project. RSI is used to monitor the magnitude of changein requirements. It gives a picture of the clarity of therequirements in the customer's mind.

The ARP when compared with the CRP will indicate theamount of work still pending and will help him anticipate thework for the next month. CRP when compared with the ARPwill indicate the amount of work still pending and will helphim anticipate the amount of work which can be completedthe next month. AOP will give the time for which a problemof particular severity remains open and help in setting realisticschedule estimates. ACP will give the time taken to close a

problem of a particular severity and help in setting realisticschedule estimates. The responsibility for all these softwaremetrics lies with the project manager.

IV. METHODOLOGYAND DATA ANALYSIS

The goal of this study is to establish the significance ofsoftware metrics in ERP projects. We use correlation andmultiple correlations [14] [15] to test the two hypotheses

Name of the Metric Formula

Schedule Slippage f{(Actual No. ofDays -

Schdl SlppgeEstimated No. of days)(SS) Estimated No. ofDays} *100

per phaseNo. of requirement changed

Requirements Stability Index (added / deleted / modified)(RSI) Total No. of Initial

requirementsArrival Rate of Problems No. of problems reported by(ARP) the customers per monthClosure Rate of Problems No. of problems closed per(CRP) month

Age ofOpen Problems Sum oftime (days per month)(AOP) that problems have been open

No. of open problems permonth

Age ofClosed Problems Sum of time (days per month)(ACP) that problems have been

closed/ No. of closedproblems per month

Authorized licensed use limited to: CALIF STATE UNIV NORTHRIDGE. Downloaded on April 27, 2009 at 17:36 from IEEE Xplore. Restrictions apply.

formulated below to investigate the impact of one metric overanother. The relationship between two variables is such that achange in one variable result in a positive or negative changein the other and also greater change in one variable result in acorresponding greater change in the other. This is known ascorrelation. Positive or negative values of coefficient ofcorrelation 'R' between two variables indicate positive ornegative correlation. We have used Karl Pearson's Coefficientof correlation which is defined below. Note that R has no unitsand is a mere number. If there is some relationship betweentwo variables, their scatter diagram shall have pointsclustering near about some curve.

R= 1 /N ( ((xi -Cy)T*y)1 /N Zxi2-(T)2 11/N y2-(1

Equation (1) is used to compute the correlation coefficientR. Multiple correlations are used to find the degree of inter-relationship among three or more variables. The objective ofusing multiple correlations is to find how far the dependentvariable is influenced by the independent variables. We denotethe multiple correlations between x1, the dependent variableand x2, X3... Xn, independent variables, by R1..234 ...n. Themultiple correlation coefficient of x1 on x2 and x3 is the simplecorrelation coefficient between the observed value of x1 andits estimated value b12..3 x2 + b13. 2x3 denoted by E1. 23. Wedenote the multiple correlation of x1 on x2 by R1 (23). Equation(2) gives the value of R1 (23).

R123) = Co (Xl, E23) / (1 [var (x)] * V [var (l_2)] ) (2)

We set up the following two hypotheses to focus our studyand to facilitate statistical analysis:

Hypothesis 1: The metric RSI influences the metric scheduleslippage (SS)

Hypothesis 2: The metrics AOP, ACP determine maintenanceefforts (ME) (i.e) person-hours.

TABLE II

RSI 0.27 0.15 0.076 0.63 0.59 0.7SS 0.11 0.059 0.072 0.29 0.36 0.51

Table II provides the RSI and SS values computed based on

the data from an ERP project A. The coefficient of correlation'R' between RSI and SS is calculated as 0.9406. The value ofR, being greater than zero indicates a positive correlationbetween the two variables RSI and SS. Data obtained fromthe same project A were used to compute the simplecorrelations between the metrics ME (x1), AOP (x2) and AOC(X3) as r12 =0.863, r13 = 0.648 and r23 = 0.709 respectively.The multiple correlation R1 (23) computed using the equation(2) gives 0.864. Since R1 (23) iS very large, it follows that thevariables x2 and X3 have considerable influence on the variable

x1. In other words, regression equation of x1 on x2 and X3 willbe excellent. This data analysis gives a great deal of support toour two hypotheses formulated in this study. The clearestresult observed from the value of R and R1 (23) iS the strongrelationship between RSI and SS and the influence of AOCand ACP over ME. The most reasonable inference that can bedrawn from this study is the significance of software metricsin ERP projects as they act as a tool to effectively monitor thecrucial phase ofERP implementation (i.e.) gap analysis.

V. CONCLUSIONS AND FUTURE WORKThe conclusions in this paper are of two kinds: 1)

conclusions based on the statistical analysis and 2)conclusions on the usefulness of software metrics in ERPprojects. Statistical analysis shows that the software metricsproposed in this paper play a significant role for successfulERP implementation. There is sufficient evidence found fromthe literature review that the ERP projects are damagedheavily by poor software process management, especially thevarying requirements due to organizational changes andcustomization. These varying requirements lead to scheduleslippage.

Also we learn that large maintenance efforts are requiredfor ERP projects as the real software processes get life onlyafter ERP implementation. Regarding the usefulness of theproposed software metrics, we conclude that it is a pragmatic,useful method for improving the software processes in ERPprojects, because it enables the ERP team to resolve theimportant issues such as requirements instability, scheduleslippage and efforts required for maintenance. Furthermore,the SMP determines decisions for the ERP team and providesagainst the possibilities of various defects that might ariseduring or after the implementation. To summarize, softwaremetrics helps the ERP team to keep track of their project andpilot it in the right path as and when there is a deviation.A future research study could compare the performance of

various ERP projects using this SMP with those not using theSMP and the application of other statistical techniques likemultiple linear regressions, etc. If the software metrics arelinked with some software quality factors, then theperformance of the project can be further improved [18].Hence generating suitable metrics to consider quality aspectsof software processes will strengthen the developed softwareproduct. It is also proposed to develop a software tool toexecute SMP. In the future, we plan to generate a projectdatabase to make the results of SMP from each ERP projectpublicly available for the ERP team.

VI. REFERENCES[1] N. E. Fenton, "Software metrics: A rigorous approach", Chapman &

Hall, 1991.[2] Erick Stensurd and Ingunn Myrtveit, "Identifying high performance

ERP projects", IEEE Transactions on Software Engineering, Vol.29,No.5, May 2003

[3] Wenhong Luo and Diane M. Strong, "A framework for evaluating ERPimplementation choices", IEEE Transactions on EngineeringManagement, Vol.5 1, No.3, August 2004

[4] C. P. Holland and B. Light, "A critical success factors model for ERPimplementation", IEEE Software, pp. 30-36, May/June 1999

Authorized licensed use limited to: CALIF STATE UNIV NORTHRIDGE. Downloaded on April 27, 2009 at 17:36 from IEEE Xplore. Restrictions apply.

[5] D. Robey, J. W. Ross and M. C. Boudreau, "Learning to implemententerprise systems: An exploratory study of the dialectics of change",Journal ofManagement Information Systems, Vol. 19, No. 1, 2002.

[6] H. E. Dunsmore, "Software metrics: an overview of an evolvingmethodology", Information Processing & Management, Vol. 20, pp.183-192, 1984.

[7] H. C. Lucus, Implementation: The Key to Successful InformationSystems, New York: Columbia Univ. Press.

[8] R. P. Cerveny and G. L. Sanders, "Implementation and structuralvariables," Information Management, vol. 11, no. 4, pp. 191-198, 1987.

[9] V. Sambamurthy and L. J. Kirsch, "An integrative framework of theinformation systems development process," Decision Sci., vol. 31, no. 2,pp.391-411,2000.

[10] Alexis Leon, "Enterprise resource planning", New Delhi: Tata McGraw-Hill, 2000

[11] S. R. Chidamber and C. F. Kemerer, "A metrics suite for object-orienteddesign ", IEEE transactions on software engineering, Vol. 20, No. 6,June 1994

[12] B. A. Kitchenham, R. T. Huges and S. G. Linkman, "Modeling softwaremeasurement data", IEEE Transactions on Software Engineering, Vol.27, No. 9, pp. 788-804, September 2001

[13] S. Pressman, "Software engineering-A practioner's approach", India:Tata McGraw-Hill, 2001

[14] Sharma, Goyal, "Mathematical statistics", India: Krishna PrakashanMandir, 1994.

[15] D. M. Levine, M. Berenson, & D. Stephan, "Statistics for managers",New Jersey: Prentice Hall, 1999.

[16] S. W. Ambler, "Agile Modeling: Effective Practices for ExtremeProgramming and the Unified Process", Wiley, 2002.

[17] K. Beck, "Embracing changes with extreme programming", Computer,pp. 70-77, October 1999.

[18] Mohammad Aishayeb, Wei Li, "An empirical validation of object-oriented metrics in two different iterative processes", IEEE transactionson software engineering, Vol. 29, No. 11, November 2003.

VII. BIOGRAPHIES

S Parthasarathy, Lecturer in ComputerApplications, Department of Computer Scienceand Engineering, Thiagarajar College ofEngineering, Madurai, Tamil Nadu, India, is aB.Sc., M.C.A., M.Phil., P.G.D.B.A., P.G.D.P.M.professional with four years of teachingexperience. He is currently pursuing Ph.D. inComputer Science in the Anna University,Chennai, India. His current research interests areenterprise resource planning (ERP) and software

engineering. He has presented more than ten papers in national andinternational conferences in these areas.

N Anbazhagan, Senior Lecturer inMathematics, Thiagarajar College of Engineering,Madurai, Tamil Nadu, India, is an M.Sc.(Mathematics), M.Phil., P.G.D.C.A. professionalwith seven years of teaching and researchexperience. He was awarded the Ph.D. inMathematics in 2002. He received the Young

I / ~~Scientist Award from the DST, New Delhi, India,in 2004 and the Career Award for Young Teachers

from AICTE, New Delhi, India, in 2005. His current areas of research arestochastic inventory modeling and optimization techniques. He has publishedmore than twenty papers in national and international conferences andjournals in these areas.

Authorized licensed use limited to: CALIF STATE UNIV NORTHRIDGE. Downloaded on April 27, 2009 at 17:36 from IEEE Xplore. Restrictions apply.