When Do Refactoring Tools Fall Short

7/29/2019 When Do Refactoring Tools Fall Short

1/8

When Do Refactoring Tools Fall ShortMuhammad Taimur Khan and Javed Ferzund

AbstractRefactoring is the process of transforming code by preserving its behavior, to make it more maintainable and to

improve its design. A particular refactoring is identified by a specific name, some preconditions and a set of needed

transformations. Refactoring is a nontrivial and time consuming task because a lot of preconditions may need to be

evaluated and transformations may affect a number of code locations. So, efficient and reliable tool development for

refactoring is needed. A number of tools have been developed but analysis of these tools in terms of refactoring support andapplication to real projects is lacking. In this paper we present an empirical study to show the extent of refactoring provided

by the major refactoring tools available to-date. We evaluate the performance of famous refactoring tools by applying them

on large, real projects. It is found that most of the tools fall short in application of various refactoring types on large projects.

Index TermsEmpirical Studies, Refactoring Tools, Software Metrics, Software Refactoring.

1INTRODUCTION

OFTWARE maintenance is the most compromising phasein the software development life cycle. Most of thechanges requested during this phase cause conflicts

because new features or the rapid changes may not fit in theoriginal design of the system, which may result in duplicateand ambiguous code. Software refactoring provides thesolution to minimize the duplication and ambiguity in codeand promotes the efficient handling of maintenance phase.Refactoring is used to improve the design and quality all theway through the software development lifecycle. It increasesthe developer performance by allowing her to easily catchand fix the bugs because source code is more readable. Itsupports extensibility by transforming the code into arecognizable pattern.Refactoring process consists of three major phases: a) In theidentification phase, the source code to be re-factored is

identified. It is the duplicated or bad code, which needs to beremoved. The selected code is examined and analyzed toverify whether it is actually a bad/redundant code or not. b)In the proposal phase, selection of the refactoring candidateis made. Different candidates have different impact on thesource code. This phase gives a list of refactoring candidateswhich will be applied on the source code to enhance themaintainability. c) In the evaluation phase, appropriatemetrics are selected for evaluation of maintainability,reusability and understandability. These metrics arecompared before and after applying the refactoring [8].There are many different refactoring types having aparticular purpose. Some of these techniques minimize

redundancy of code, others improve level of abstraction.Some refactoring techniques can be applied to a group ofprogramming languages; others can be applied to a certaintype of language only. Commonly used refactoringtechniques include: renaming the name of a class or thevariable, accessing the field name with setter and gettermethods, moving the code to super class, moving the code toa subclass, moving code from one method to another method

for minimizing the complexity, and make generalization inthe code for more sharing.Refactoring may change the internal structure of the body of

the code but it does not alter the working of software. Everytime a refactoring is applied to a piece of code, it transformsthe code slightly reducing its redundancy or duplication. Asequence of refactoring is applied one by one in order toavoid any abrupt change in code. The changes made throughthe process of refactoring are so small that they are less likelyto go false, but affect the source code significantly in apositive way [17]. Refactoring applied throughout the lifecycle of software can improve the quality, maintainabilityextensibility and reliability of code.Some studies have shown that refactoring activities do notalways enhance the software quality but if applied in anineffective way can deteriorate software quality [14]

Refactoring quality can be evaluated by examining theexternal quality attributes. Refactoring is successful if thecorrectness, robustness, compatibility, efficiency, portabilityand functionality of software is maintained [13].Many developers hesitate to use refactoring, the reason beingmore manual amount of effort required for a minor changeand increased chances of introducing bugs in the re-factoredcode. This problem can be solved by automated refactoringtools. A number of refactoring tools exist and modern IDEsalso provide refactoring support such as IntelliJ IDEA [2],NetBeans [5] and Eclipse [1]. These IDEs provide speciamenu for refactoring in their menu bar. The developer canselect the refactoring technique from the menu bar and apply

refactoring on the selected code. Some refactoring tools likeRefactorIt [6] and JRefactory [3] provide refactoring supportfor the whole source file and automatically recommend therefactoring according to different refactoring methods. Thesetools and IDEs provide refactoring in an effective andcustomized way but the refactoring engine should be reliablebecause refactoring process may introduce new bugs and cancreate uncertainty in the system.In this paper we analyze the refactoring support andevaluate the refactoring performance of various refactoringtools. Our objective is to evaluate the applicability ofrefactoring tools and suggest some guidelines for the designof future refactoring tools. We conduct an empirical study on

S

M. T. Khan is with the Department of Computer Science, University ofSargodha, Sargodha, Pakistan.

J. Ferzund is with the Department of Computer Science, University ofSargodha, Sargodha, Pakistan.

JOURNAL OF COMPUTING, VOLUME 4, ISSUE 12, DECEMBER 2012, ISSN (Online) 2151-9617

https://sites.google.com/site/journalofcomputing

WWW.JOURNALOFCOMPUTING.ORG 5

2012 Journal of Computing Press, NY, USA, ISSN 2151-9617


2/8

the source code of two large projects, Eclipse and JEditor.Five refactoring tools are evaluated in this study includingRefactorit, JRefactory, Eclipse, IntelliJ IDEA, and NetBeans.Rest of the paper is organized as follows: in section 2, relatedwork is discussed. A case study is presented in section 3, andresults are discussed in section 4. Finally, the paper isconcluded in section 5.

2 RELATED WORK

A number of studies exist on refactoring techniques,refactoring issues and tools for refactoring. Mens andTourwe [13] in their survey on software refactoring focusedon software activity to determine the appropriate level ofabstraction in source code, design models or requirementdocument. The authors first identified the refactoring codeportion and then measured the degree to which methods andvariables of a class belonged together. During their behavioranalysis in refactoring, they suggested that applications maybe domain specific, real time software (execution time ofcertain operations, it should preserve all kind of temporalconstraints), embedded software (memory and power issuessolved by refactoring), safety critical software (safety that

needs to be preserved).Garrido and Johnson [8] discussed the difficulties in

refactoring C code with preprocessor directives. The authorsdefined preconditions and execution rules to maintaincorrectness of refactoring in the presence of macros andconditional directives. They proposed new refactorings formacro definitions and conditionals. Some enhancementswere suggested for program analysis and programrepresentation tools to correctly manipulate preprocessordirectives.

Henkel and Diwan [10] presented a lightweightapproach for evolving application programming interfaces(APIs), which do not depend on version control or

configuration management systems. Their approach wasbased on capturing the refactoring actions during APIevolution. The authors provided an implementation of theirapproach to capture and replay refactoring actions within anintegrated development environment. Savga andHeidenreich [15] discussed open issues of applyingrefactoring in the context of feature-oriented programming.The authors discussed impact of refactoring on the relationbetween the problem and solution spaces. They furthersuggested refactoring semantics to upgrade and test finalproducts of a product line.

Soares et al. [17] presented a tool to identifybehavioral changes in transformations. They tested their tool

on 9 transformations and were successful in detecting allbehavioral changes and not producing false alarms. Schferet al. [16] analyzed changes in concurrent behaviors causedby refactoring engines and developed techniques to makethem behavior preserving. The authors found that a broadrange of refactorings are not influenced by concurrency at all.Whereas other important refactorings could be madebehavior-preserving for correctly synchronized programs byusing their framework. The authors claimed that theirtechniques are easy to implement and require only minimalchanges to existing refactoring engines.

Jemerov [11] described the key architecturacomponents of IntelliJ IDEA involved in implementingrefactorings. The author also described some of the problemsfaced when implementing refactorings and possibledirections for future development. IntelliJ IDEA was the firstto implement Extract Method refactoring for Java and lateradded support for a wide array of refactorings. Mens et al[12] provided a detailed overview of existing research in thefield of software restructuring and refactoring, from a formaas well as a practical point of view. The authors proposed alist of open questions that indicated future researchdirections, and provide some partial answers to thesequestions as well.

Murphy-Hill et al. [14] addressed the effect ofmethod of data collection related to refactoring session onquality of results. The authors described 4 methods fordrawing conclusions about how programmers refactorcharacterized the assumptions made by each, and presenteda family of experiments to test those assumptions. Glynn andStrooper [9] discussed some issues of refactoring related tolevel of automation, stages of the refactoring process, subsetof the refactorings that could be applied and the complexityof refactorings. The authors presented a framework forevaluating software refactoring tool support based on theDESMET method.

Our work is similar to the previous work, as weevaluate the extent and type of refactoring provided byvarious tools. However, previous studies lack empiricastudies on the application of refactoring tools on largeprojects. We present a detailed evaluation of state of the artrefactoring tools by applying these tools on large projectsWe discuss the shortcomings of these tools and comparethese tools for various types of refactoring.

3 CASESTUDY

In order to analyze the types and extent of refactoringprovided by state of the art tools, we have designed anempirical study. For this study, five refactoring tools andsource code of two large projects is selected. In this sectionwe discuss the refactoring feature available in different toolsCharacteristics of source code and features of the projects arealso discussed.3.1 Refactoring Tools

Five tools are used in this study including RefactoritJRefactory, Eclipse, IntelliJ IDEA, and NetBeans. These toolsare selected based on their popularity and easy availability.IntelliJ IDEA:

It is the IDE (Integrated Development Environment) for java,which provides refactoring in its menu bar. The user selectsthe refactoring options from the menu bar and tool appliesthe refactoring on the selected code. So user manually appliesrefactoring on the source code. On the main refactoringmenu or on the context menu of the selection the desiredrefactoring is chosen or the equivalent keyboard shortcut ispressed if there exist any. In the dialog box that opens therefactoring options, there is an option to apply the changesinstantly, depending on the refactoring type. There is alsosupport for previewing the changes before actuallyperforming refactoring for certain refactoring.






3/8

IntelliJ IDEA provides the following common refactoringtypes: Change Class Signature, Change Method Signature ,Convert Anonymous to Inner, Convert to Instance Method,Copy / Clone, Encapsulate Fields, Extract Delegate, ExtractInclude File, Extract Interface, Extract Method, ExtractMethod Object, Extract Superclass, Generify Refactoring,Inline, Introduce Constant, Introduce Field, IntroduceParameter, Introduce Parameter Object, Introduce Property,Introduce Variable, Invert Boolean, Make Class Static, MakeMethod Static, Migrate, Move Refactorings ,Pull MembersUp, Push Members Down, Remove Middleman, RenameRefactorings, Replace Constructor with Builder, ReplaceConstructor with Factory Method, Replace Inheritance withDelegation, Replace Method Code Duplicates, Replace TempWith Query, Safe Delete, Type Migration, Use InterfaceWhere Possible and Wrap Return Value. This vast support ofrefactoring options effectively removes redundant data andthus improves efficiency.Eclipse Refactoring:Eclipse is also the open source IDE for java. Refactoringcommands are available from the context menu as well as inthe menu bar. When the refactoring operations are appliedone has the options to preview all of the changes that mightoccur when the refactoring operation would be performed.During this preview the tool provides information such as:

Possible potential problems List/record of changes

The list or record of changes includes the changes that willoccur after applying the refactoring. In case we do not usethe option of preview and perform the operations, the changewill be applied as a whole and the problem resulting fromthat would be shown. In such case a kind of problem mayoccur that will prohibit the execution of the refactoringprocess thus resulting in halting of the operation displayinglist of problems. So its a good approach to have a previewbefore applying a transformation.RefactorIT:It is a tool for Java developers. A developer can take sourcecode of any size and complexity and rework it into well-designed code by means of automated refactorings. It may beused as a stand-alone tool or installed as a plug-in. However,over the years RefactorIT lost market share because IDE'snow offer built-in refactoring tools (first Ecplise and later alsoNetBeans). Today, there are still quite a few developers usingit for their daily work. It is used for the audit and metrics ofthe code and it analyze the code and provides suggestion forthe improvement of code design.

JRefactory:

It is a tool that supports: move class between packages(repackage), rename class, add abstract parent class, addchild class, remove empty class, extract interface, push upfield, push down field, rename field, push up method, pushup abstract method, push down method, move methodextract method, and rename parameter.This tool comes as a command line option with GUI orwithout, and as a plugin for the JEditor (still underdevelopment), JBuilder, NetBeans, and Elixir IDEs. For

JBuilder and Elixir, you can switch from the UML diagram tothe source code.Net Beans IDE:An open source Integrated Development Environment forsoftware maintainers. It supports tools needed to createprofessional desktop, enterprise, web and mobile applicationwith Java Platform ,C/C ++, PHP, Java Script and Groovy.Refactoring Operations include: Rename, Introduce variableconstant, field, or method, change method parametersEncapsulate fields, Pull up, Push down, Move Class, CopyClass, Move Inner to Outer level, Convert Anonymous classto inner, Extract Interface, Extract super class, Use super typewhere possible, and safely delete.3.2 Projects UsedTwo large open source projects are selected for this study

JEditor and Eclipse. Source code of these projects is checkedout using Subversion. These projects are selected due to theirlarge size and long development history.

JEditor:JEditor 1.5 is a pure java editor (MDI). It provides someadvanced features: open, save, save as, compile and run javasource code. This version supports undo edits using stacks. Itexplains several uses of swings, threads, classes and stacksalso. JEditor is downloaded from (http://www.planetsource-code.com).Eclipse:

Source code of Eclipse is downloaded using Eclipse built inclient for the Concurrent Versioning System (CVS).

4 RESULTS

We used software audits and calculated software metrics forboth the projects. A software audit checks Java source codeagainst a set of coding standard rules. Three types of metricswere calculated: simple metrics, object oriented metrics andquality metrics. A short description along with thresholdvalues for the simple, object oriented and quality

TABLE 1

ADESCRIPTION OF SIMPLE METRICS

Simple Metrics Description Lower Limit Upper Limit

CLOC Comment Lines of Code It counts all lines that contain regular commentsand java doc comments

0 10

V(G) Cyclomatic Complexity Counts the number of code conditions giving anindication of how complex program is

1 10

DC Density of CommentsCLOC/LOC

This determines a density value for howcommented code is

.2 .4

EXEC Executable Statement This determines no of executable statements 0 20




http://www.jetbrains.com/idea/webhelp/convert-anonymous-to-inner.htmlhttp://www.jetbrains.com/idea/webhelp/convert-to-instance-method.htmlhttp://www.jetbrains.com/idea/webhelp/copy-clone-2.htmlhttp://www.jetbrains.com/idea/webhelp/encapsulate-fields.htmlhttp://www.jetbrains.com/idea/webhelp/extract-delegate.htmlhttp://www.jetbrains.com/idea/webhelp/extract-include-file.htmlhttp://www.jetbrains.com/idea/webhelp/extract-include-file.htmlhttp://www.jetbrains.com/idea/webhelp/extract-interface.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method-object.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method-object.htmlhttp://www.jetbrains.com/idea/webhelp/extract-superclass.htmlhttp://www.jetbrains.com/idea/webhelp/generify-refactoring.htmlhttp://www.jetbrains.com/idea/webhelp/inline.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-constant.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-field-2.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter-object.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-property.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-variable.htmlhttp://www.jetbrains.com/idea/webhelp/invert-boolean.htmlhttp://www.jetbrains.com/idea/webhelp/make-class-static.htmlhttp://www.jetbrains.com/idea/webhelp/make-method-static.htmlhttp://www.jetbrains.com/idea/webhelp/make-method-static.htmlhttp://www.jetbrains.com/idea/webhelp/migrate.htmlhttp://www.jetbrains.com/idea/webhelp/move-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/pull-members-up.htmlhttp://www.jetbrains.com/idea/webhelp/pull-members-up.htmlhttp://www.jetbrains.com/idea/webhelp/push-members-down.htmlhttp://www.jetbrains.com/idea/webhelp/remove-middleman.htmlhttp://www.jetbrains.com/idea/webhelp/rename-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/rename-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-builder.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-factory-method.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-factory-method.htmlhttp://www.jetbrains.com/idea/webhelp/replace-inheritance-with-delegation.htmlhttp://www.jetbrains.com/idea/webhelp/replace-inheritance-with-delegation.htmlhttp://www.jetbrains.com/idea/webhelp/replace-method-code-duplicates.htmlhttp://www.jetbrains.com/idea/webhelp/replace-temp-with-query.htmlhttp://www.jetbrains.com/idea/webhelp/replace-temp-with-query.htmlhttp://www.jetbrains.com/idea/webhelp/safe-delete-2.htmlhttp://www.jetbrains.com/idea/webhelp/type-migration.htmlhttp://www.jetbrains.com/idea/webhelp/use-interface-where-possible.htmlhttp://www.jetbrains.com/idea/webhelp/use-interface-where-possible.htmlhttp://www.jetbrains.com/idea/webhelp/wrap-return-value.htmlhttp://www.planet-source-code.com/http://www.planet-source-code.com/http://www.planet-source-code.com/http://www.planet-source-code.com/http://www.jetbrains.com/idea/webhelp/wrap-return-value.htmlhttp://www.jetbrains.com/idea/webhelp/use-interface-where-possible.htmlhttp://www.jetbrains.com/idea/webhelp/use-interface-where-possible.htmlhttp://www.jetbrains.com/idea/webhelp/type-migration.htmlhttp://www.jetbrains.com/idea/webhelp/safe-delete-2.htmlhttp://www.jetbrains.com/idea/webhelp/replace-temp-with-query.htmlhttp://www.jetbrains.com/idea/webhelp/replace-temp-with-query.htmlhttp://www.jetbrains.com/idea/webhelp/replace-method-code-duplicates.htmlhttp://www.jetbrains.com/idea/webhelp/replace-inheritance-with-delegation.htmlhttp://www.jetbrains.com/idea/webhelp/replace-inheritance-with-delegation.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-factory-method.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-factory-method.htmlhttp://www.jetbrains.com/idea/webhelp/replace-constructor-with-builder.htmlhttp://www.jetbrains.com/idea/webhelp/rename-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/rename-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/remove-middleman.htmlhttp://www.jetbrains.com/idea/webhelp/push-members-down.htmlhttp://www.jetbrains.com/idea/webhelp/pull-members-up.htmlhttp://www.jetbrains.com/idea/webhelp/pull-members-up.htmlhttp://www.jetbrains.com/idea/webhelp/move-refactorings.htmlhttp://www.jetbrains.com/idea/webhelp/migrate.htmlhttp://www.jetbrains.com/idea/webhelp/make-method-static.htmlhttp://www.jetbrains.com/idea/webhelp/make-method-static.htmlhttp://www.jetbrains.com/idea/webhelp/make-class-static.htmlhttp://www.jetbrains.com/idea/webhelp/invert-boolean.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-variable.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-property.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter-object.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-parameter.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-field-2.htmlhttp://www.jetbrains.com/idea/webhelp/introduce-constant.htmlhttp://www.jetbrains.com/idea/webhelp/inline.htmlhttp://www.jetbrains.com/idea/webhelp/generify-refactoring.htmlhttp://www.jetbrains.com/idea/webhelp/extract-superclass.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method-object.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method-object.htmlhttp://www.jetbrains.com/idea/webhelp/extract-method.htmlhttp://www.jetbrains.com/idea/webhelp/extract-interface.htmlhttp://www.jetbrains.com/idea/webhelp/extract-include-file.htmlhttp://www.jetbrains.com/idea/webhelp/extract-include-file.htmlhttp://www.jetbrains.com/idea/webhelp/extract-delegate.htmlhttp://www.jetbrains.com/idea/webhelp/encapsulate-fields.htmlhttp://www.jetbrains.com/idea/webhelp/copy-clone-2.htmlhttp://www.jetbrains.com/idea/webhelp/convert-to-instance-method.htmlhttp://www.jetbrains.com/idea/webhelp/convert-anonymous-to-inner.html


4/8

NCLOC Non Comments Line of Code This count all the lines that do not containcomments or blank lines

0 10

NOP Number of parameter This count number of parameters for a method orconstructor

0 4

LOC Total Lines of code The number of lines for a class including blank linesand comments.

5 1000

TABLE 2

ADESCRIPTION OF OBJECT ORIENTED METRICS

Object Oriented Metrics Description Lower Limit Upper Limit

A Abstractness This metric counts the ratio of abstract classes and interfaces for apackage.

0.0 0.5

Ca AfferentCoupling

Also known as Incoming Dependencies: This metric counts thenumber of classes from other packages that depend on classes inthe analyzed package.

0 500

DIT Depth in Tree This is the distance from the class to the root of the inheritancetree (0 for java.lang.Object).The preferred threshold values applyfor a class.

0 5

Ce Efferent Coupling Also known as Outgoing Dependencies:This metric is a measure for the number of types of the analyzedpackage which depend upon types from other packages.The preferred threshold values apply for a package.

0 20

I Instability (I = Ce/ (Ca + Ce))

Check to see how stable or instable your packages are designed.The preferred threshold values apply for a package.

0.7 1.0

NOTa Number ofAbstract Types

This metric counts the number of abstract classes and interfaces.The preferred threshold values apply for a package.

0 20

NOC Number ofChildren in Tree

This metric measures the number of direct subclasses of a class.The preferred threshold values apply for a class. 0 10

NOTc Number ofConcrete Types

This metric counts the number of concrete classes. The preferredthreshold values apply for a package. 0 80

NOTe Number ofExported Types

This metric count the number of classes and interfaces exportedoutside a package. The preferred threshold values apply for apackage.

3 50

NOF Number of Fields This metric counts the number of fields in a method (in local andanonymous classes).The preferred threshold values apply for amethod.

0 1

NOT Number of Types This metric count the number of classes and interfaces. Thepreferred threshold values apply for a package.

0 80

RFC Response forClass

This metric counts the number of distinct methods andconstructors invoked by a class. The preferred threshold valuesapply for a class.

0 50

WMC WeightedMethods perClass

This calculates the sum of cyclomatic complexity of methods for aclass. The preferred threshold values apply for a class. 1 50

NOA Number ofAttributes

This metric counts the number of fields in type (class orinterface).The preferred threshold values apply for a class.

0 5

metrics is presented in

Table 1, Table 2 and Table 3 respectively.The selected tools were unable to calculate the cyclomaticcomplexity and number of parameters metric for the Eclipseproject, as indicated in

Table 4. The only simple metrics calculated were related tolines of code.

The selected tools were able to calculate only the type relatedobject oriented metrics for the Eclipse project. They failed tocalculate the remaining object oriented metrics like WMCRFC, DIT and NOC etc. The values for the calculated objectoriented metrics for the eclipse project are presented in Table5.

TABLE 3

ADESCRIPTION OF QUALITY METRICS

Quality Metrics Description LowerLimit

Upper Limit

CYC Cyclic Dependencies This estimates how many cycles in which a package isinvolved by determining the number of times a packageis repeated in the dependency graph. The preferredthreshold values apply for a package.

0 1

DIP Dependency InversionPrinciple

This metric calculates the ratio of dependencies that haveabstract classes or interfaces as a target to the totalamount of dependencies for a class. The preferredthreshold values apply for a class.

0.3 1.0

DCYC Direct Cyclic Dependencies A direct cyclic dependency for a package counts everymutual dependency: The number of other packages, thepackage depends on and which in turn also depend onthe package. The preferred threshold values apply for apackage.

0 1






5/8

D Distance from the MainSequence

The perpendicular distance of a package from the mainsequence from the idealized line A + I = 1.

0.0 0.1

EP Encapsulation Principle This metric calculates the ratio of classes that are usedoutside of a package to the total amount of classes thatthe package contains.The preferred threshold valuesapply for a package.

0 0.6

LCOM Lack of Cohesion ofMethods

This is a measure for the cohesiveness of a class(calculated using the Henderson-Sellers method).The preferred threshold values apply for a class.

0 0.2

LSP Limited Size Principle The number of direct subpackages of a package.Thepreferred threshold values apply for a package.

0 10

MQ Modularization Quality The MQ of a Module Dependency Graph (MDG) that ispartitioned into k packages is the difference between theaverage inter- and intra-connectivity of the k packages.MQ is calculatable for the project node only, for thepackage nodes the intra-connectivity value is shown.

0 1000

NT Number of Tramps This metric counts the number of parameters in a class'methods, which are not used by its code.The preferredthreshold values apply for a method.

0 1

TABLE 4

SIMPLE METRICS FOR THE ECLIPSE PROJECT

SimpleMetrics

V(G) LOC NCLOC CLOC DC NOP EXEC

45290 29598 11899 0.263 2917

TABLE 5

OBJECT ORIENTED METRICS FOR THE ECLIPSE PROJECT

ObjectOrientedMetrics WM

C

RFC

DIT

NOC

Ce Ca I A NOT NOTa NOTc NOTe NOF NOA

376 32 344 274

Among the quality metrics, the selected tools could calculateonly the modularization quality for the Eclipse project, as

shown in Error! Not a valid bookmark self-reference.. Thetools failed to calculate the remaining quality metrics likedistance from the main sequence, cyclic dependencies, and

limited size principle etc.

For the JEditor project, similar is the case for simplemetrics. The selected refactoring tools were unable tocalculate the cyclomatic complexity and number ofparameters metrics, as indicated in

Table 7.

TABLE 6

QUALITY METRICS FOR THE ECLIPSE PROJECT

QualityMetrics

D cyc DCYC LSP DIP EP MQ NT LCOM

80.295

TABLE 7

SIMPLE METRICS FOR THE JEDITOR PROJECT

SimpleMetrics

V(G) LOC NCLOC CLOC DC NOP EXEC

948 771 167 0.176 300

TABLE 8

OBJECT ORIENTED METRICS FOR THE JEDITOR PROJECT

ObjectOrientedMetrics W

MC

RFC

DIT

NOC

Ce Ca I A NOT NOTa NOTc NOTe NOF NOA

0 9 1 0 10 0 10 1






6/8

TABLE 9

QUALITY METRICS FOR THE JEDITOR PROJECT

QualityMetrics

D cyc DCYC LSP DIP EP MQ NT LCOM

0 9 1.0 0.0 10

In case of object oriented metrics, more number of

metrics are calculated for the JEditor project as compared tothe Eclipse project. In addition to the type related metrics,afferent coupling, efferent coupling, instability andabstractness were also calculated by the selected tools, asshown inTable 8. It reveals that the refactoring tools fail to calculatethese metrics for the large projects.

Again for the JEditor project, more number of qualitymetrics was calculated as compared to the Eclipse project. Inaddition to the modularization quality, dependencyinversion principle, encapsulation principle, number oftramps and lack of cohesion of methods were also calculatedas shown in

Table9

. It also reveals that the refactoring tools fail tocalculate these metrics for the large projects, as size of theEclipse project is much larger than the JEditor project.

When the whole source code of projects is selected, the

behavior of different refactoring tools is different for thesame projects.Table 10 shows different refactoring candidates for theEclipse project. Second and third column of table indicatethat refactoring behavior of RefactorIt and JRefactory is notsimilar. Both tools have almost 45% refactoring candidates incommon but more than 50% refactoring candidates aredifferent.

Similarly, for the JEditor project 35% refactoringcandidates are common and more than 60% candidates aredifferent, as shown inTable 11. Less number of refactoring candidates is producedfor the JEditor project as compared to the Eclipse project

Further, for the Eclipse project differences among therefactoring tools are more apparent. It reveals that for largeprojects the refactoring tools fail to produce optimal output.

TABLE 10

REFACTORING BEHAVIOR FOR THE ECLIPSE PROJECT

Refactoring Behavior RefactorIT JRefactoryVariable Related characteristicsNot used member Found non-transient, non-static member. Please mark as transient or provide accessors Avoid variables with short names Avoid excessively long variable names like xmlCommentAttribute Unused assignment on variable

Null parameter(s) Possible unsafe assignment to a non-final static field in a constructor Private field could be made final. It is only initialized in the declaration or constructor. Variables that are final should be in all caps Variables that are not final should not contain underscores (except for underscores in standardprefix/suffix).

Initialization Related CharacteristicsStatements between declaration and first use early_declaration Avoid instantiating new objects inside loops Unused assignment on variable Private field could be made final. It is only initialized in the declaration or constructor The String literal 'target' appears 20 times in this file Avoid assigments in operands Avoid using 'while' statements without curly braces String "invalid XML file not imported: " was already used in this class, final string can be created. Switch statements should have a default label final field could be made static Avoid writing into array parameters such as 'result' If-else statementsThese nested if statements could be combined Deeply nested if..then statements are hard to read Avoid if (x != y) ..; else ..; If statement has no block Function Related CharacteristicsEmpty block or body Method body has less than 2 statements Avoid calls to overridable methods during construction Method level Cyclomatic Complexity Unused method parameter When doing a String.toLowerCase()/toUpperCase() call, use a Locale A method should have only one exit point, and that should be the last statement in the method All methods are static. Consider using Singleton instead. Alternatively, you could add a private






7/8

constructor or make the class abstract to silence this warningTwo strings compared with '==' operator -- expected str1.equals(str2) Method body has more than 20 statements Use block level rather than method level synchronization Class Related CharacteristicsA high number of public methods and attributes in an object can indicate the class may need to bebroken up for exhaustive testing may prove difficult Each class should declare at least one constructor Abstract classes should be named 'AbstractXXX' Avoid unnecessary constructors Avoid instantiation through private constructors from outside of the constructor's class Avoid concatenating non literals in a StringBuffer constructor or append Cyclomatic Complexity of Class Avoid instantiating new objects inside loops A value of 73 may denote a high amount of coupling within the class Avoid really long Classes Avoid unnecessary constructors - the compiler will generate these for you Avoid calling toString() on String objects; this is unnecessary Package Related CharacteristicsA high number of imports can indicate a high degree of coupling within an object Unnecessary import statements Suggestion Interface Avoid using implementation types like 'HashMap'; use the interface instead Avoid using implementation types like 'Vector'; use the interface instead

TABLE 11

REFACTORING BEHAVIOR FOR THE JEDITOR PROJECT

Refactoring Behavior RefactorIT JRefactoryVariable Related characteristics

Formed non transient, non static member (mark as transient or provide accesser) Variable with shorter name Avoid unused local variable Avoid method local variable with short name Avoid long variable name Variable that are not final should not contain under score Field name with method name Avoid concatenating with non literals Initialization Related Characteristics

Avoid instantially string object Null Assignment to object If-else with out curly brackets Avoid loop with curly brackets Function Related Characteristics

Method should have one exit point (It should be the last statement in methods. Method action performed has cyclic complexity of 15 Avoid to call overridable method during constructor Method is never overridden Class Related Characteristics

Class should have one constructor Cyclomatic complexity Jeditor Complexity 04 Height 15 Value of 38 may denoted high amount of coupling with in class Jeditor 1.0 appears 04 times.

Refactoring behavior in Refactor-It and JRefactory was notconsistent. All refactoring candidates were not applied bythese tools, only specific pattern was repeated during therefactoring process. Following output was obtained indifferent categories:

Variable Related characteristicsAvoid method local variable with short name

Initialization Related CharacteristicsNull assignment to object

Function Related CharacteristicsShould have only one exit point

Class Related CharacteristicsClass should have one constructor, Cyclomaticcomplexity

For the Net Beans IDE we found the following shortcomings:

We cannot add new behavior in refactoring menu. When we change class name from upper case to

lower case, Error is displayed first letter of classshould be capital.

When we apply the renaming in the source filerenaming is not applied on the comment code which






8/8

may bring ambiguity in the source code whichconsist of thousand lines of code.

Extract Method which is very common in therefactoring is not available in this version.

We apply the refactoring manually, code is selectedand refactoring is applied from the menu bar.

For the IntelliJ IDEA, following shortcomings were found:

New refactoring type was not supported in this toole.g. we cannot add new behaviors in these toolswhich can enhance the refactoring support.

When we apply the renaming the class namechanges from A to a, and no warning is displayedthat class name should be capital.

Renaming is not applied in the comment code whichincreases ambiguity.

Refactoring is applied on the selected code and wecannot apply all of the supported refactoring once ata time.

Extract methods during refactoring may bring morecomplexity which is difficult for understandingsource.

5 CONCLUSIONS

In this paper, we have presented an empirical study toanalyze the extent of refactoring provided by variousrefactoring tools. We evaluated the performance of famousrefactoring tools by applying them on large, real projects. Wetested five refactoring tools (IntelliJ IDEA, EclipseRefactoring, RefactorIt, JRefactory, Net Beans IDE) on twolarge projects (Eclipse, JEditor).It is found that most of thetools fall short in application of various refactoring types onlarge projects. Software metrics are an important measure toidentify the code to be refactored. The selected tools failed to

calculate most of the object oriented and quality metrics. Thenumber of failures was high for the larger project, in this caseEclipse.Refactoring behavior of different refactoring tools was notconsistent. All refactoring candidates could not be applied bythese tools, only a specific pattern was repeated during therefactoring process. New refactoring types were notsupported in these tools. Renaming could not be applied onthe commented code, which creates ambiguity. Manualselection of the refactoring candidates increases the timerequired for refactoring. It is more evident in the largeprojects.In future, we want to work on the design of refactoring tools.

REFERENCES

[1] Eclipse Refactoring, http://code.google.com/p/refactoring-plugin/

[2] IntelliJ IDEA, http://www.jetbrains.com/idea/[3] JRefactory, http://jrefactory.sourceforge.net/[4] JEditor, http://www.planet-source-code.com[5] Net Beans, http://netbeans.org/[6] RefactorIt, http://refactorit.sourceforge.net/[7] M. Fowler, K. Beck, J. Brant, W. F. Opdyke, and D. Roberts,

Refactoring: Improving the Design of Existing Code, Addison -Wesley, 1999.

[8] A. Garrido, and R. Johnson, Challenges of Refactoring CPrograms, Proceedings of the International Workshop onPrinciples of Software Evolution, IWPSE02, New York, USA, 2002.

[9] E. Glynn and P. Strooper, Evaluating software refactoring toosupport, Australian Software Engineering Conference, April 2006

[10]J. Henkel, and A. Diwan, CatchUp! Capturing and replayingrefactorings to support API evolution, International conference onSoftware Engineering, ICSE05, pp 274-283, 2005.

[11] D. Jemerov, Implementing Refactorings in IntelliJ IDEAProceedings of the 2nd Workshop on Refactoring Tools, WRT08New York, USA, 2008.

[12] T. Mens, S. Demeyer, B. Du Bois, H. Stenten, and P. Van Gorp, Refactoring: Current research and future trends, Electronic Notesin Theoretical Computer Science, Volume 82, Issue 3, Decembe2003, Pages 483499.

[13] T. Mens, and T. Tourwe, A Survey of Software Refactoring, IEEETransactions on Software Engineering, Vol 30, Issue 2, pp 126-1392004.

[14] E. Murphy-Hill, A. P. Black, D. Dig, and C. Parnin, GatheringRefactoring Data: a Comparison of Four Methods, Proceedings ofthe 2nd Workshop on Refactoring Tools, WRT08, New York, USA2008.

[15] I. savga, and F. Heidenreich, Refactoring in Feature-OrientedProgramming: open Issues, In Proceedings of the Workshop onModularization, Composition, and Generative Techniques forProduct Line Engineering (McGPLE 2008).

[16] M. Schfer, J. Dolby, M. Sridharan, E. Torlak, and F. Tip, Correcrefactoring of concurrent Java code, In Proceedings of ECOOP2010, pp 225249.

[17]G. Soares, D. Cavalcanti, R. Gheyi, T. Massoni, D. Serey, and MCornlio, Safe Refactor Tool for Checking Refactoring Safety, intools session at SBES 2009, pp 49-54.

Muhammad Taimur Khan received his MS Computer Science fromUniversity of Sargodha, Sargodha, Pakistan. He is working as a seniordeveloper at the University of Sargodha, Pakistan. His researchinterests include Software Refactoring, Software Testing and ReverseEngineering

Javed Ferzund received a Master degree in Computer Sciences in2003 and a Doctor of Informatics from Technical University of GrazAustria in 2009. His research interests include Software Maintenance &

Evolution, Reverse Engineering and Software Debugging. He iscurrently an assistant professor at the Department of Computer Scienceand Information Technology, University of Sargodha, Pakistan.




http://www.sciencedirect.com/science/journal/15710661/82/3http://www.sciencedirect.com/science/journal/15710661/82/3

Documents

When Do Refactoring Tools Fall Short