Feature Maps: A Comprehensible Software

Representation for Design Pattern DetectionHannes Thaller, Lukas Linsbauer, and Alexander Egye

24 December 2018

Правилов Михаил13.02.2019

1

Design Patterns (DP)● Are the generalization of the different adapted implementations

● May also circumvent deficiencies and inflexibilities in OO languages

● Examples:○ Builder

○ Decorator

○ Visitor

2

Pattern’s semantic● What the pattern does

● Why the pattern is needed

● Where it is useful

3

Design Pattern Detection (DPD)Used for:

● Preliminary analysis in maintenance and testing

scenarios

● Hinting at structures and dependencies and finding

performance-critical regions

4

Design Pattern Detection (DPD)Used for:

● Preliminary analysis in maintenance and testing

scenarios

● Hinting at structures and dependencies and finding

performance-critical regions

5

Problem in DPD● Patterns are only a guideline for implementing a specific

solution

● Each pattern can be implemented in various ways

6

Roles Mapping

7

Roles Mapping● Primary role

● Secondary role

● Pattern mapping:

● Equivalence class:

● Unique mapping

8

ML essential elements● Data

● Model

● Optimization procedure

● Evaluation

9

Data● Independent and identically distributed

● The observations are mutually independent

● Collected in the same fashion

● Preprocessing

10

Model● Convolutional Neural Networks (CNNs)

○ capabilities for computer vision problems

○ local correlations -> high-level features

○ reasonable amount of model parameters

● Random Forests

○ multiple randomly perturbed decision trees

○ smoother decision boundaries 11

Evaluation● Cross-Validation

● k-folds

● Nearly unbiased estimator

● Accuracy, Precision, Recall

● Matthews Correlation Coefficient

12

Design Pattern Detection Pipeline

13

Feature Extraction● Feature = Micro-Structure = Design Pattern (1-2 roles)

● Size prohibits variance in implementation

● Readable

● MS detectors are sub-graph filters by predicate

● Result: ASG’s sub-graphs annotated with MS roles

14

Candidate Sampling● Finds potential candidates

● Creates role mappings

● Huge search space

● Heuristic search:

a. sup → Component

b. sub → Composite

c. sib → Leaf 15

Feature Normalization: Approach

16

Feature Normalization: Issues

17

Design Pattern Inference● Learning models

● Input: feature maps

● Output: probability

18

Design Pattern Detection Study● Binary classification problem

● Multi-label classification problem

● Evaluated only the last two stages of the pipeline

19

Controlled Variables7 Experiment Parameters (ExP):

1. Patterns = {Singleton, Template Method, Composite,

Decorator}

2. Role Count = {1, 2, 3, 4}

3. Classification Model = {Random Forest, CNN}

20

Controlled Variables4. Negative-Positive Candidate Ratio = {1, 2, 4, 6, 8, 10}

5. Data Augmentation = {0, 1, 5, 10}

6. Optimization Budget = {200}

7. Instance Independence = {project-fold cross validation}

21

Response Variables● Accuracy

● Precision

● Recall

● F1 score

● MCC (primary)

22

Data Source

23

Procedures1. 67 different Micro-Structures extracted

2. Possible candidates sampled

3. Feature map for each candidate and pattern

4. Global unique role identifiers in [0; 161]

5. Controlled variables -> Response variables

24

Experiment Results

25

Experiment Results

26

Experiment Results, CNN● Average: Med = 0.646, IQR = [0.528; 0.772]

● Worst,Template Method: Med = 0.51, IQR = [0.43, 0.51]

● Best,Composite: Med = 0.79, IQR = [0.71, 0.85]

● Worst case variance = 0.16

● MCC variance on NPСR = 0.064

27

Experiment Results, Random Forest● Average: Med = 0.48, IQR = [0.29; 0.64]

● Worst,Decorator: Med = -0.35, IQR = [-0.38, -0.29]

● Best,Composite: Med = 0.79, IQR = [0.67, 0.83]

● Worst case variance = 0.14

● MCC variance on NCPR = 0.16

28

Experiment Results, Data Imbalance● Not applicable to Singleton

● CNN is more robust

29

Experiment Results, Independence● Tests for independence regarding MCC

● Different permutation counts:

○ CNN close to significant: p < 0.057

○ RF is significant: p < 3.42 * 10^(-16)

● Significant effect concerning NPCR:

○ CNN significant: p < 2.85 * 10^(-8)

○ RF significant: p < 0.002 30

Summary● Patterns with more roles are easier to detect

● Decline in performance with larger NPCR

● FMs fit well in the framework of CNNs

● Direct comparison with other results

31

Comparison

Zanoni et al., Accuracy Best Authors, Accuracy Average

Singleton (NPCR = 1.66) RF, 0.93 RF 0.73 (-20%), CNN 0.77 (-19%)

Template Method - ?

Composite (NPCR = 3) ν−SVCRBF 0.81 RF 0.90, CNN 0.93

Decorator (NPCR = 1.48) RF 0.82 CNN 0.79

32

Threats to Validity, Internal● P-MARt dataset:

○ Old projects

○ Misclassification

○ Overfitting:

■ RF: increasing tree number

■ CNN: Dropout, kernel and activation

regularization, and early stopping 33

Threats to Validity, External● Multiple patterns with different numbers of roles

● Different NPCRs

● Only one pattern for each number of roles

34

Conclusion● Feature Map - flexible and comprehensible source code

representation useful beyond DPD

● Robust performance for imbalanced datasets

● Compact fashion

● Future: more design patterns, bigger dataset, graph

representations natively algorithms35

Links● Article: https://arxiv.org/abs/1812.09873

● Dataset:

http://www.ptidej.net/tools/designpatterns/index_html#2

● Source code is not available

36