Embed Size (px)
Citation preview
136
CHAPTER 8
EVALUATION OF THE DEVELOPED SYSTEM
8.1 INTRODUCTION
This chapter describes the different techniques that were used to
evaluate the ontology and the proposed system.
8.2 EVALUATION OF SRMONTO
Ontology evaluation (Gangemi et al 2005) is the problem of
assessing a given ontology from the point of view of a particular criterion of
application. The content of ontologies should be evaluated before using it in
other ontologies or applications (Gómez-Pérez 2003). Assessing the quality of
an ontology is important for several reasons (Tartir et al 2005). This section
delineates the evaluation of the SRMONTO, an educational ontology
described in Chapters 3 and 4.
8.2.1 Quantitative Evaluation of Ontology
The Quantitative evaluation of the developed ontology,
SRMONTO, was performed based on the four major metrics, such as Class,
Property, Ratio and Axiom Metrics. Each metric has a set of parameters to
evaluate. The evaluation is based on the parameters of each metric against
each of the ontology separately.
137
8.2.1.1 Class Metrics
The following metrics are identified as class metrics. The statistics
of the Class Metrics of the SRMONTO is given in Table 8.1.
NoC - No of Classes
NoLC - Number of Leaf Classes
NoRC - Number of Root Classes
NoSpC - Number of Superclasses
NoSbC - Number of Subclasses
NoEqC - Number of Classes with Equivalent Class Expressions
NoHvR - Number of Classes with ‘HasValue’ restriction Axioms
AvDoI - Average Depth of Inheritance
MxDoI - Max Depth of Inheritance
Table 8.1 Statistics of the Class Metrics of the SRMONTO
METRICS SRIONTO SRAONTO SRPONTO SRTONTO SRCONTO
NoC 86 56 19 25 40
NoLC 17 34 14 18 16
NoRC 1 1 1 1 1
NoSpC 6 22 5 7 6
NoSbC 22 55 18 24 21
NoHvR 66 21 7 6 9
AvDoI 2.76 4.14 2.21 2.33 2.13
MxDoI 3 5 3 3 3
8.2.1.2 Property Metrics
The following metrics are identified as the property metrics. The
statistics of the Property Metrics of the SRMONTO is given in Table 8.2.
138
NoDtP - Number of Datatype Properties
NoOP - Number of Object Properties
NoAP - Number of Annotation Properties
Table 8.2 Statistics of the Property Metrics of the SRMONTO
Property
MetricsSRIONTO SRAONTO SRPONTO SRTONTO SRCONTO
NoOp 4 6 4 4 3
8.2.1.3 Ratio Metrics
The following metrics are identified as ratio metrics. This metric
depends on the class metrics, and is a type of indirect metric. The Statistics of
the Ratio Metrics is given in Table 8.3.
Specialization Ratio = NoSbC/NoSpC
Reuse Ratio = NoSbC/NoC
Table 8.3 Statistics of the Ratio Metrics
Ratio Metrics SRIONTO SRAONTO SRPONTO SRTONTO SRCONTO
Specialization
Ratio3.67 2.50 3.60 3.43 3.50
Reuse Ratio 0.96 0.98 0.95 0.96 0.95
8.2.1.4 Axiom Metrics
There are two metrics identified as axiom metrics. The statistics of
the Axiom Metrics of the SRMONTO is given in Table 8.4.
Equivalent Class
Has Value Restrictions
139
Table 8.4 Statistics of the Axiom Metrics of the SRMONTO
Axiom
MetricsSRIONTO SRAONTO SRPONTO SRTONTO SRCONTO
Has Value
Restrictions66 21 7 6 9
From the result of ratio metrics, it is inferred that the developed
ontology has a high reusability ratio, which provides a high degree of
reusability for the ontology in various applications. It is also noted that the
ontology has a high specialization ratio which establishes the fact that the
ontology is very informative.
The statistical analysis says that the SRIONTO has concepts twenty
times the number of properties. The SRAONTO has concepts eight times the
number of its properties. The SRPONTO has concepts four times the number
of its properties. The SRTONTO has concepts more than three times the
number of its properties. Similarly the SRCONTO has concepts more than
five times the number of its properties. Hence, in the resultant ontology, the
NoC will be higher than the NoP and is presumed to be a concept oriented
ontology. Figure 8.1 shows that the reusability ratio of the SRMONTO is
linear, because all the sub ontologies have the reuse ratio of more than 95%.
Figure 8.1 Qualitative Analysis of the SRMONTO using the Ratio Metrics
Ratio Metrics
0
0.5
1
1.5
2
2.5
3
3.5
4
SRI ONTO SRA ONTO SRP ONTO SRT ONTO SRC ONTO
Ontology
RatioSpecialization Ratio
Reuse Ratio
140
8.2.2 Comparison of the SRMONTO with other Educational
Ontologies
The constructed ontology i.e., the SRMONTO is compared with
three existing educational ontologies, such as the Cryptography Ontology
(Takahashi et al., 2005), Software Testing Ontology (Zhu and Huo, 2004) and
the ER Model Ontology (Boyce and Pahl, 2007).
8.2.2.1 SRAONTO Vs Cryptography Ontology
This section presents the comparative analysis of the SRAONTO
and the cryptography Ontology in Table 8.5 followed by its inferences.
Table 8.5 Comparison between the SRAONTO and the Cryptography
Ontology
METRIC SRAONTO Cryptography Ontology
NoC 56.00 21.00
NoSpC 22.00 9.00
NoSbC 55.00 20.00
AvDoI 4.14 2.91
MxDoI 5.00 4.00
Reuse Ratio 0.98 0.95
Specialization Ratio 2.50 2.22
The following inferences have been made from a comparative study
of the SRAONTO and the cryptography ontology.
The number of classes, subclasses and superclasses in the
SRAONTO is considerably higher than that of the
Cryptography onto.
The maximum and average depth of inheritance is higher in the
SRAONTO, which is an indication of the ontology being more
detailed and accurate to its domain.
141
The SRAONTO presents a higher level of reusability than
CryptographyOnto.
The SRAONTO makes better use of specialization of its classes,
indicated by its higher specialization ratio.
8.2.2.2 SRMONTO Vs Software Engineering Ontology (SEONTO)
This section presents the comparative analysis of the SRAONTO
and the SEONTO in Table 8.6 followed by its inferences.
Table 8.6 Comparison between the SRMONTO and the SEONTO
METRIC SRMONTO SEONTO
NoC 226.00 362.00
NoSpC 46.00 200.00
NoSbC 221.00 361.00
AvDoI 3.00 4.00
MxDoI 5.00 8.00
Reuse Ratio 0.98 0.99
Specialization Ratio 4.08 1.85
The following inference has been made from a comparative study
of the Software Risk Management ontology and Software Engineering
Ontology.
The SRMONTO has a high specialization ratio compared with
the software Engineering ontology; it proves to focus more
accurately on the chosen domain.
8.2.3 Qualitative Evaluation of the SRMONTO
The SRMONTO has also been evaluated by a semiotic set of
metrics based on the semiotics approach [Stamper et al., 2000] and earlier
work (Burton-Jones et al 2005). Since the SRMONTO is an educational
142
ontology, it is given to a set of students, who are the primary users of this
knowledge base, for evaluation. A set of 55 students from the II year CSE,
Jerusalem College of Engineering, Chennai-100, were given the evaluation
task. A one hour seminar was conducted for the students on “ Design and
Development of SRMONTO : An Educational Ontology representing
Software Risk Management Knowledge”. During the questionnaire session an
evaluation sheet was given to the students to be filled up. It is prepared based
on the Attributes of different metrics addressed in semiotic approach. The
sample evaluation sheet is given in Figure 8.2.
Figure 8.2 Sample Evaluation Sheet for the SRMONTO
143
The responses received from the students are shown in Table 8.7
and it is also shown using the bar chart in Figure 8.3.
Table 8.7 Evaluation based on the semiotic approach
Metrics Very high High Medium Low
Lawfulness 40 14 1 0
Richness 37 16 2 0
Interpretability 34 19 2 0
Consistency 37 16 2 0
Clarity 36 15 4 0
Comprehensiveness 39 13 3 0
Accuracy 38 15 2 0
Relevance 36 16 3 0
Figure 8.3 Qualitative Evaluation of the SRMONTO
Based on the above said evaluation, the qualitative analysis of the
SRMONTO is made and is presented in Table 8.8. Social Quality will be
calculated after deployment of the ontology.
144
Table 8.8 Result of Qualitative Analysis
Metrics Attributes / Criteria Degree
Syntactic
Quality
Lawfulness Very High
Richness Very High
Semantic
Quality
Interpretability Very High
Consistency Very High
Clarity Very High
Pragmatic
Quality
Comprehensiveness Very High
Accuracy Very High
Relevance Very High
8.3 EVALUATION OF THE ONTOLOGY BASED E-LEARNING
SYSTEM
The working of the system “AN INTELLIGENT ONTOLOGY-
BASED E-LEARNING TOOL FOR SOFTWARE RISK MANAGEMENT”
which has been described in Chapter 5 was demonstrated to a group of fifty
students. A feedback form consisting of a set of twenty questions was
distributed to the students and their assessment of the system from a learner’s
point of view was obtained. The set of questions is tabulated in Table 8.9. For
each question, the students can rate “A” for Excellent, “B” for Good, “C”
for Average and “D” for Poor. The results of the evaluation are depicted in
Figure 8.4.
145
Table 8.9 Feedback Questions
Question
No.Feedback Question
Q1 The e-Learning system provides content that exactly fits your
needs
Q2 The e-Learning system provides up-to-date content
Q3 The e-Learning system makes it easy for you to find the content
you need
Q4 The e-Learning system is user-friendly
Q5 The e-Learning system allows you to evaluate your learning
performance
Q6 The testing methods provided by the e-Learning system are fair
Q7 The e-Learning system enables you to choose what you want to
learn
Q8 The e-Learning system records your learning progress and
performance
Q9 The systems can upgrade your information literacy
Q10 You can use the learning management tools smoothly
Q11 You can study by yourself and take notes
Q12 You can choose various web resources to study effectively
Q13 The e-Learning system evaluates your subject knowledge
effectively
Q14 The e-Learning system provides secure testing environments
Q15 The e-Learning system provides sufficient and useful content
Q16 The e-Learning system can be made available for everyone
easily
Q17 The learning content has a proper syllabus
Q18 The e-Learning system has a very effective user interface
Q19 The e-Learning system allows you to apply the learnt
knowledge to solve problems
Q20 The course content meets your preferred learning style
146
Figure 8.4 Qualitative Evaluation of the Developed System by the
students
The system has also been demonstrated to a set of 25 software developers
who are working in the following companies Infonovum Technologies,
Chennai and Snowwood Infocom Technologies Pvt Ltd, Chennai. The results
of the evaluation are depicted in Figure 8.5.
Figure 8.5 Qualitative Evaluation of the Developed System by the
Software Experts
147
The feedback questions were categorized into five parameters and
Table 8.10 shows the categorization and overall grade of the system. The
consolidated results of the evaluation are shown in Figure 8.6.
Figure 8.6 Parameterized Evaluation of the Proposed System
Table 8.10 Overall Grade of the Proposed System
Sl.
No.Parameters Relevant Questions
Overall
Grade
1 Effectiveness of course material Q1,Q2,Q3,Q15,Q17
and Q20
Good
2 User-friendliness of the system Q4,Q16 and Q18 Excellent
3 Learner assessment Q5,Q6,Q13 and Q14 Excellent
4 E-learning system effectiveness Q7,Q8,Q9 and Q10 Excellent
5 Study process Q11, Q12 and Q19 Excellent
The proposed system has been evaluated qualitatively on the basis
of five parameters: Effectiveness of Course Material, User-friendliness of the
System, Learner Assessment, e-Learning System Effectiveness and Study
Process, as depicted in the Figure 8.4. Since the majority of the learners have
indicated that the effectiveness of the course material is “Good”, future
enhancements to the proposed system may revolve around increasing the
148
effectiveness of the course material. The course material may be rendered
more effective by facilitating dynamic, automated updation directly from the
ontology instead of periodic, manual updations.
The user-friendliness of the system has tied between “Excellent”
and “Good”, signifying that the user-interface of the system is indeed simple
and easy to use. The learner assessment performed by the system has received
an “Excellent” rating. The proposed system uses Bayesian networks to assess
the newly-registered learner. Also, the knowledge gained by the learner is
assessed at the end of the course through a computer-adaptive test. The rating
achieved for this parameter proves that the learners trust the fairness of the
system’s assessment procedures.
The overall effectiveness of the system, given by the “e-Learning
System Effectiveness” parameter has also received an “Excellent” rating. This
attests the fact that learners have indeed benefitted from using the e-learning
system that has enabled them to upgrade their information literacy. The
“Study Process” parameter has also been rated as “Excellent” by the learners.
The proposed system aids in the study process of the learner by providing
intelligent suggestions inferred from the Bayesian network model. The rating
obtained indicates that the learners have profited from the support for study
process provided by the system.
Finally the developed system is compared with MOT 2.0 [Ghali
and Cristea, 2009], iClass [Brady et.al., 2006], Interbook [Brusilovsky, 1999]
and AHA [Paul De Bra et.al, 2006]. Table 8.11 provides a summary of
various features exhibited in the above mentioned systems with the developed
system. The features are broken down as follows:
Knowledge Based – A system must be based on knowledge in order to
exhibit this feature. This can mean ontologies, simple concept maps,
taxonomies and so on.
149
Knowledge Authoring – A system must allow its authors to alter the
knowledge base in order to exhibit this feature.
Content Generation – A system must be capable of generating
content of some form in order to exhibit this feature. The content can
include tests, lessons, table of contents and so on.
Adaptive Delivery – A system must be capable of personalizing the
delivery of content based on information about the learner.
Overlay Model – A system must use an overlay model as its user
modeling technique in order to exhibit this feature. An overlay model
is a specific approach to user modeling.
Standards - A system must make use of standards in some of its functionality
in order to exhibit this feature. This can mean packaging standards, ontology
standards, user model standards and so on.
Table 8.11 Comparison of developed system with existing systems
Features MOT iClass Interbook AHA Proposed
System
Knowledge Based Yes Yes No Yes Yes
Knowledge Authoring Yes No No Yes Yes
Content Generation Yes Yes No Yes Yes
Adaptive Delivery Yes Yes Yes Yes Yes
Overlay Model Yes Yes Yes Yes Yes
Standards No Yes No No Yes
8.4 EVALUATION OF FAOMS
This Section describes the evaluation of the FAOMS, which was
explained in Chapter 7. Since the FAOMS involves four modules, each
module was unit tested with the help of the test applications generated, except
150
the Heuristic Function, which actually integrates all the three other modules;
hence the testing of this module alone is not necessary. Finally, an integration
testing of the FAOMS, was performed for five pairs of ontologies.
8.4.1 Lexical Matching
Different sets of terms were given to the Lexical Matching module.
The stemming of the suffix and the prefix and the corresponding comparison
produced results that were correct. Table 8.12 shows some of the test cases
used.
Table 8.12 Lexical Matching Test Cases
Inputs Stemmed value Value returned
Dependent
Depend
Depend
Depend
True
Dynamic
static
Dynam
Static
false
Compose
composition
Compos
composit
False
Reserve
reservation
Resev
Resev
True
Caress
Caring
Caress
Care
False
8.4.2 Semantic Matching
A Pair of terms was given to the Semantic Matching test
application, which retrieved the synsets (synonyms) from the WordNet. The
retrieved synsets were checked for any overlap, and produced outcomes were
precise. Table 8.13 shows some of the test cases used. As the retrieved synsets
are huge, only some of them are shown in the table.
151
Table 8.13 Semantic Matching Test Cases
Input Synset valueValue
returned
Dynamic
Active
{active, dynamic}: (used of verbs (e.g. `to run') and participial
adjectives (e.g. `running' in `running water')) expressing action
rather than a state of being
{dynamic}: of or relating to dynamics ………..
----------------------------------
{active, dynamic}: (used of verbs (e.g. `to run') and participial
adjectives (e.g. `running' in `running water')) expressing action
rather than a state of being
{active}: expressing that the subject of the sentence has the
semantic function of actor: "Hemingway favors active
constructions"
{active}: (of the sun) characterized by an increased occurrence of
sunspots and flares and radio emissions
true
Factor
High
{gene, cistron, factor}: (genetics) a segment of DNA that is
involved in producing a polypeptide chain; it can include regionspreceding and following the coding DNA as well as introns
between the exons; it is considered a unit of heredity; "genes were
formerly called factors"
………….
----------------------
{high, in_high_spirits}: happy and excited and energetic
{high, high-pitched}: used of sounds and voices; high in pitch or
frequency
{high}: (literal meaning) being at or having a relatively great or
specific elevation or upward extension (sometimes used in
combinations like `knee-high'); "a high mountain"; "highceilings"; "high buildings"; "a high forehead"; "a high incline"; "a
foot high"
{high}: far up toward the source; "he lives high up the river"
high"
false
8.4.3 Similarity Checking
For a pair of classes, properties are gathered along with their values
and the corresponding ‘p’ values are calculated. Table 8.14 shows a partial
output of the Similarity Checking test application.
152
Table 8.14 Similarity Checking Test Cases
Input OWL
file
Input Class Properties and
their values
Similar
properties
Max. No.
of Properties
No. of
similar
properties
P value
returned
Pizza
collaborative
Pizza
PolloAdAstra
CajunSpice
Topping
hasTopping
TomatoTopping
GarlicTopping
#CajunSpice
ToppingRedOnionTopping
SweetPepper
Topping
8 0 0
Pizza
collaborative
Pizza
CajunSpiceTopping
HotGreenPepper
Topping
hasSpiciness
Hot
hasSpiciness
Hot
has
Spiciness
1 1 1
8.4.4 Evaluation of Integrated System
The FAOMS is compared with the existing algorithms such as
Chimaera (McGuinness et al 2000), ONION (Mitra et al 2000) and PROMPT
(Noy et al 2000). Table 8.15 shows the comparison between FAOMS and
some existing ontology merging algorithm.
Table 8.15 Comparison between existing algorithms and the FAOMS
Sl.
No.Parameters CHIMAERA ONION PROMPT FAOMS
1 User Interaction More More More Nil
2 String Matching No No Yes Yes
3 Semantic Matching Yes Yes Yes Yes
4 Instance Matching No No No Yes
5 Structure Matching Yes No No Yes
6 Ontology Structure Ontologua IDL,
XML-
Based
FRD(s),
OWL
OWL
7 Matching Selection
is based on
--- Concept,
Properties
High
Value
No. Of
Properties
8 Normalization No No Yes
9 Automation Semi
Automatic
Semi
Automatic
Semi
Automatic
Fully
Automated
10 Additional
Resources
NA WordNet NA WordNet
153
Further a complete evaluation of the FAOMS was carried out. Two
different versions of each of the sub ontologies of SRMONTO were taken
into account for testing and evaluating the system. One set had the same
number of concepts and similar hierarchy to merge, two sets had the subset of
an other ontology taken into account, and two sets had subsets and a few other
different concepts. The measures used to evaluate the system are, Precision,
Recall and the F-measure.
8.4.4.1 Precision
The numbers of correct concepts in the final output are divided by
the number of concepts in the final output.
no.of correct concepts in outputprecision *100
no.of conceptsin output (8.1)
8.4.4.2 Recall
The numbers of correct concepts in the final output are divided by
the number of concepts in the final output that should have been correct.
no.of correct conceptsin outputrecall *100
no.of concepts that should becorrect in output (8.2)
8.4.4.3 F-measure
A measure that combines precision and recall is the harmonic
mean of precision and recall, the traditional F-measure.
precision * recallF 2* *100
precision recall (8.3)
154
In the following tests, the given input ontologies are in the OWL
format, and the concepts represent both the properties and classes.
Test 1: SRIONTO
One input ontology has 84 concepts, the other has 33 concepts, and
the corresponding merged ontology has 92 concepts. Based on the devised
measures, 90 concepts are correct, but 91 concepts should have been correct.
Test 2: SRAONTO
One input ontology has 61 concepts, the other has 59 concepts, and
the corresponding merged ontology has 62 concepts. Based on the devised
measures, the merged ontology was accurate.
Test 3: SRCONTO
One input ontology has 8 concepts, the other has 28 concepts, and
the corresponding merged ontology has 30 concepts. Based on the devised
measures, the merged ontology was accurate.
Test 4: SRPONTO
One input ontology has 16 concepts, the other has 19 concepts, and
the corresponding merged ontology has 20 concepts. Based on the devised
measures, the merged ontology was accurate.
Test 5:SRTONTO
One input ontology has 11concepts, the other has 25 concepts, and
the corresponding merged ontology has 26 concepts. Based on the devised
measures, the merged ontology was accurate.
155
The overall concepts that the system has tested are 230, the
concepts that are correct in the output are 228, the concepts that should have
been correct are 230, and the total concepts obtained as the output is 229.
Hence the precision is 228/229*100=99.56%, recall is 228/230=99.78% and
the f-measure is 99.66%.
The Table 8.16 shows the testing results of the FAOMS in a
summarized manner.
Table 8.16 Summarized Test Cases
Ontology No of conceptsNo of Concepts in
the Target Ontology
SRIONTOVer. I 84
92Ver. II 33
SRAONTOVer. I 59
62Ver. II 61
SRCONTOVer. I 08
30Ver. II 28
SRPONTOVer. I 16
20Ver. II 19
SRTONTOVer. I 11
26Ver. II 25
The values of each measure to evaluate the FOAMS are given in
Table 8.17.
Table 8.17 Values of Precision, Recall and the F-Measure
Sl. No. Test Precision Recall F-Measure
1 Test 1 97.82 98.90 98.35
2 Test 2 100.00 100.00 100.00
3 Test 3 100.00 100.00 100.00
4 Test 4 100.00 100.00 100.00
5 Test 5 100.00 100.00 100.00
156
Figures 8.7, 8.8 and 8.9 show the results of Precision, Recall and
the F-Measure respectively.
Figure 8.7 Test Results of the Measure “Precision”
Figure 8.8 Test Results of the Measure “Recall”
157
Figure 8.9 Test Results of the Measure “F-measure”
8.5 EVALUATION OF THE “ONTOLOGY BASED WEB
SERVICE FOR SRM”
This section describes the evaluation of the system “Ontology
based Web Service for Software Risk Management”, which is described in
Chapter 6. This system is evaluated by two different test cases.
8.5.1 Case Study – I
A company XYZ which comprises of a chain of super markets
decides to design a software for updating fluctuating prices of perishable
goods constantly. The characteristics of the software is set to be designed in
such a way that
It automatically updates the changes in all branches.
It connects with the pre existing server which controls the
details in all branches.
It concurrently changes the prices.
158
Notifies administrator when the prices fall below a certain
range.
Compatible with existing system.
Creates a log of all changes for every 24 hour period.
8.5.1.1 Non-Numeric factors
The results of the non-numeric factors for Case Study - I are
calculated from the values given by the user as the input to the system. The
instability of each factor is tabulated in Table 8.18.
Table 8.18 Instability of the Non Numeric Factors for Case Study I
S.No Factors Questions ValuesResult
(instabililty)
1 Environmental
adaptability
Q1
Q2
Q3
Q4
Q5
High
Very
High
high
high
74%
2 Legal adaptability Q1
Q2
Q3
Q4
Q5
high
medium
high
medium
high
62%
3 Political adaptability Q1
Q2
Q3
Q4
Q5
High
Very high
Medium
Medium
Low
58%
4 Socio cultural adaptability Q1
Q2
Q3
Q4
Q5
Very high
Medium
Medium
Medium
High
62%
159
Table 8.18 (Continued)
S.No Factors Questions ValuesResult
(instabililty)
5 Technology adaptability Q1
Q2
Q3
Q4
Q5
high
high
Medium
Very high
Medium
66%
6 Competitor adaptability Q1
Q2
Q3
Q4
Q5
Very
High
Very high
Very
High
Very
High
Very high
90%
7 Customer adaptability Q1
Q2
Q3
Q4
Q5
Low
very low
low
very low
low
22%
8 Public adaptability Q1
Q2
Q3
Q4
Q5
High
Medium
Medium
Medium
Medium
54%
9 Supplier adaptability Q1
Q2
Q3
Q4
Q5
Medium
High
Medium
High
Medium
58%
10 Machine adaptability Q1
Q2
Q3
Q4
Q5
High
Low
Very high
Medium
Medium
58%
160
Table 8.18 (Continued)
S.No Factors Questions ValuesResult
(instabililty)
11 Staffing adaptability Q1
Q2
Q3
Q4
Q5
Low
Low
Medium
Medium
Low
38%
12 Management adaptability Q1
Q2
Q3
Q4
Q5
Medium
Medium
Medium
Low
Low
42%
13 Material adaptability Q1
Q2
Q3
Q4
Q5
Very high
High
Medium
High
Medium
66%
14 Monetary adaptability Q1
Q2
Q3
Q4
Q5
Very high
Very high
Very high
Very high
Very high
90%
15 Ecological adaptability Q1
Q2
Q3
Q4
Q5
High
Medium
High
Medium
High
62%
8.5.1.2 Numeric Factors
The results obtained by the system before taking the final decision
are tabulated in Table 8.19.
161
Table 8.19 Results of the Numeric Factors – Case Study I
S.No Numeric factors Output Results possible
1 Net Present Value 225 Viable
2 Profit 10000 Viable
3 Cash flow 10500 Favourable
4 Absorption ratio 66% Not viable
5 Activity ratio 94.11% Viable
6 Calendar ratio 57.1% Viable
7 Capacity ratio 80% Viable
8 Current ratio 2.162 Viable
9 Depreciation tax shield 3200 Viable
10 Direct labour idle time variance 115.2 Not viable
11 Direct labour cost variance 232000 Unfavourable
12 Direct labour efficiency variance 18000 Favourable
13 Net cash flow 4000 Favourable
14 Net profit ratio 66% Favourable
15 Taxes 4000 Favourable
Figure 8.10 shows the diagrammatic representation of the
inferences made after considering the results of Non-Numeric and Numeric
Factors as mentioned in Tables 8.17 and 8.18 respectively. Finally the system
concludes that “The Project is 88.24 % Viable”.
Figure 8.10 Generated Report for Case Study I
162
8.5.2 Case Study – II
ABC Corporation, a government sector intends to implement a
software for analyzing candidate information of all applicants to any positions
in government sectors. The characteristics of the software is set to be
designed in such a way that
A central system for assimilating resources from various
centers.
Query based search for the user based on specific details.
Real time revision of data on new entries and updates of old
entries.
General forms for supporting all type of applicants.
Compatible with existing system.
Alerts for specific type of candidates.
8.5.2.1 Non-Numeric factors
The results of the non-numeric factors for Case Study - II are
calculated from the values given by the user as the input to the system. The
instability of each of the factors is tabulated in Table 8.20.
Table 8.20 Instability of the Non Numeric Factors for Case Study II
S.No Factors Questions ValuesResult
(instabililty)
1 Environmental adaptability Q1
Q2
Q3
Q4
Q5
High
Very High
high
high
high
58%
2 Legal adaptability Q1
Q2
Q3
Q4
Q5
high
medium
high
medium
high
30%
163
Table 8.20 (Continued)
S.No Factors Questions ValuesResult
(instabililty)
3 Political adaptability Q1
Q2
Q3
Q4
Q5
High
Very high
Medium
Medium
Low
50%
4 Socio cultural adaptability Q1
Q2
Q3
Q4
Q5
Very high
Medium
Medium
Medium
High
90%
5 Technology adaptability Q1
Q2
Q3
Q4
Q5
high
high
Medium
Very high
Medium
42%
6 Competitor adaptability Q1
Q2
Q3
Q4
Q5
Very High
Very high
Very High
Very High
Very high
74%
7 Customer adaptability Q1
Q2
Q3
Q4
Q5
Low
very low
low
very low
low
70%
8 Public adaptability Q1
Q2
Q3
Q4
Q5
High
Medium
Medium
Medium
Medium
74%
9 Supplier adaptability Q1
Q2
Q3
Q4
Q5
Medium
High
Medium
High
Medium
54%
164
Table 8.20 (Continued)
S.No Factors Questions ValuesResult
(instabililty)
10 Machine adaptability Q1
Q2
Q3
Q4
Q5
High
Low
Very high
Medium
Medium
62%
11 Staffing adaptability Q1
Q2
Q3
Q4
Q5
Low
Low
Medium
Medium
Low
82%
12 Management adaptability Q1
Q2
Q3
Q4
Q5
Medium
Medium
Medium
Low
Low
58%
13 Material adaptability Q1
Q2
Q3
Q4
Q5
Very high
High
Medium
High
Medium
56%
14 Monetary adaptability Q1
Q2
Q3
Q4
Q5
Very high
Very high
Very high
Very high
Very high
50%
15 Ecological adaptability Q1
Q2
Q3
Q4
Q5
High
Medium
High
Medium
High
62%
165
8.5.2.2 Numeric factors
The results obtained by the system before taking the final decision
are tabulated in 8.21.
Table 8.21 Results of the Numeric Factors – Case Study II
S.No Numeric factors Output Results possible
1 Net Present Value 225 Viable
2 Profit 3500 Not viable
3 Cash flow 3500 Favourable
4 Absorption ratio 80% Not viable
5 Activity ratio 102.5% Viable
6 Calendar ratio 79% Viable
7 Capacity ratio 104.16% Not Viable
8 Current ratio 1.6 Not viable
9 Depreciation tax shield Viable
10 Direct labour idle time variance 164 Not viable
11 Direct labour cost variance 1052170 Unfavourable
12 Direct labor efficiency variance Unfavourable
13 Net cash flow 1370 Favourable
14 Net profit ratio 92% Favourable
15 Taxes 2772 Favourable
Figure 8.11 shows the diagrammatic representation of the
inferences made after considering the results of the Non-Numeric and
Numeric Factors as mentioned in Tables 8.19 and 8.20 respectively. Finally
the system concludes that “The Project is 45.6 % Viable”.
166
Figure 8.11 Generated Report for Case Study II
8.6 SUMMARY
The SRMONTO was qualitatively and quantitatively evaluated and
it is compared with a few existing educational ontologies. The evaluation of
the SRMONTO has been done based on certain design criteria, the
quantitative analysis, and the qualitative analysis. Finally, the reuse ratio for
the ontology has been calculated from the identified metrics and it is found
that the constructed ontology is ready for usage. It is effectively used in the
ontology based e-learning system, and the system which has been developed
for this purpose qualitatively, assessed by a set of questions.
The evaluation of the FAOMS says that it is fully automatic; that is,
it is a non-human intervention system. The user just has to give the file path
that contains the ontologies to be merged. It uses lexical, semantic and
structural matching with the help of a loosely defined heuristic strategy, that
functions even if one matching method is unsuccessful. It has high precision,
recall and f-measure values.
167
The Ontology based Web Service for Software Risk Management is
evaluated by two different case studies. It learns new information as it
encounters new scenarios and these scenarios make the system act genuinely
intelligent. As the system goes through new scenarios and new instances it
develops its knowledge base, and hence, becomes Artificially Intelligent, with
the scope of producing results based on scenarios encountered in the past.
