Security Metrics for Java
Bytecode Programs
Sponsored By
*Queensland University of Technology, Australia^Aljouf University, Saudi Arabia
Dr. Bandar Alshammari ^
Prof. Colin Fidge*
Dr. Diane Corney*
Overview
We have developed a set of object-oriented security metrics.
They are capable of assessing the security
of a compiled program from the point of view
of potential information flow.
In this talk we describe:
Our previous work on design security metrics
Our current work on Java bytecode security metrics.
Our work on developing a tool that is capable of capturing these two
types of metrics from existing programs.
2 of 31
Introduction
No system is completely secure but there are still ways of reducing
security risks and vulnerabilities.
Many software security studies focus on coding vulnerabilities, such
as buffer or integer overflows or issues associated with the run-time
environment, such as component isolation.
Instead we consider software security from the perspective of
potential information flow through a program‟s object-oriented
module structure using a set of security metrics.
Security metrics which quantify the security level of a given
program could offer a „pushbutton‟ solution.
3 of 31
Overview of Security Metrics
4 of 31
Most existing security metrics are based on high-level code which does not
always give reliable results.
Low-level metrics, e.g., those derived from Java bytecode instructions could
provide better information about the executable program.
Instead, our security metrics assess the potential flow of „classified‟
information by measuring the accessibility of selected data items based on
the security design principles of “reducing the size of the attack surface”
and “granting least privilege”.
They allow software developers to easily assess the impact of code
modifications on overall program security and to compare the relative
security of different versions of the same program.
OBJECT-ORIENTED DESIGN-LEVEL
SECURITY METRICS
Part I
(Previous Work)
5 of 31
Relevant Security Design Principles
Assign the least privilege
to do the job
Minimise the size of the
attack surface
6 of 31
Our security metrics are concerned with information flow.
They provide a simple approach for identifying where security
vulnerabilities might occur in a given design\program from the
perspective of information flow of confidential data.
Overview of Security Metrics
Code-Level Metrics
Multi-Class Design Metrics
Individual-Class Design Metrics
7 of 31
Our class design metrics are concerned with information flow
through individual object-oriented classes.
They provide a simple approach for identifying where security
vulnerabilities might occur in a single class from the perspective of
information flow of confidential data.
Two properties are covered: the accessibility of classified
attributes (Data Encapsulation), and interactions between
classified and non-classified attributes (Cohesion).
Overview of Class Design Metrics
8 of 31
We have interpreted „security‟ to mean control over data
confidentiality, i.e., the ability to read classified values, and data
integrity, i.e., the ability to update classified variables.
Designer must explicitly highlight „classified‟ data fields by as per
UMLsec.
Methods are considered „classified‟ if they read or write classified data.
Designer must explicitly highlight how classified methods read (access)
or write (mutate) classified data using SPARK-like annotations.
Classes are considered „critical‟ if they contain classified data.
A notion of Security for Object-Oriented Classes
9 of 31
Security Metrics for Individual Classes
CIDA Proportion of classified attributes which are not private instance attributes
CCDA Proportion of classified attributes which are not private class attributes
COA Proportion of classified methods which are not private
CMAIProportion of mutator methods which may mutate classified attributes to
all mutators in the scope of classified attributes
CAAIProportion of accessor methods which may access classified attributes to
all accessors in the scope of classified attributes
CAIWProportion of methods which may interact with classified attributes to all
methods in the scope of classified attributes
CMW Proportion of classified methods to all methods in the class
10 of 31
Security-Related Annotations
Classified fields and
classes explicitly
annotated, as per UMLSec
Relationships between attributes, parameters and return values explicitly specified, as per SPARK
11 of 31
Security of Alternative Class Designs
0
0.2
0.4
0.6
0.8
1CIDA
CCDA
COA
CMAICAAI
CAIW
CMW
ContactNos 4 ContactNos 5
12 of 31
Overview of Multi-Class Design Metrics
Our multi-class design metrics are concerned with information flow
through multiple object-oriented classes.
They measure the compositional properties of security-annotated
object-oriented designs.
They are concerned with five properties which are related to the
overall design of an object-oriented program:
composition, coupling, extensibility, inheritance and design size.
13 of 31
Security Metrics for Multi-Class Designs
Part II: Design > Multi-Class Design Metrics
CCCProportion of actual cross-class links to classified attributes to all possible links
in the scope
CPCCInverse of the proportion of critical component classes to all critical classes
(rewards inner critical classes)
CCE Proportion of extensible critical classes to all critical classes
CMEProportion of extensible classified methods to all classified methods (in all
classes)
CSP Proportion of critical superclasses to all critical classes
CSIProportion of classes which (may) inherit from critical classes to all possible
inheritances (penalises placing critical classes higher in the hierarchy)
CMI Proportion of inheritable classified methods to all classified methods
CAI Proportion of inheritable classified attributes to all classified attributes
CDP Proportion of critical classes to all classes
14 of 31
OBJECT-ORIENTED CODE-LEVEL
SECURITY METRICS
Part II
(Current Work)
15 of 31
Overview of Code-Level Metrics
Most existing security code metrics focus on high level source code
but this approach fails to consider compiler-specific code generation.
Program code contains more detail than designs.
Local variables, control flow and defined but unused
classified fields.
Relationships between attributes, parameters and return values can
be calculated statically.
SPARK-like „derives v from w‟ annotations are not needed.
Programmer merely needs to annotate those fields believed
to be „classified‟—others can be calculated.
16 of 31
Security Metrics for Java Program Code
CIDA, …Most metrics as before, but with finer, language-specific
distinctions between access modifiers
RPBA boolean value representing whether the Java program imports
the Java reflection package (1) or not (0)
CWMPProportion of methods which write classified attributes to all
classified methods
UACAProportion of classified fields that are „defined‟ (assigned to) but
not subsequently used to all classified fields
UACM Proportion of uncalled classified methods to all classified methods
UCAC Proportion of never-used critical classes to all critical classes
CSCP Proportion of critical serialised classes to all critical classes
Program
statistics
Absolute measures are also produced (number of critical classes,
number of classified attributes, etc)
17 of 31
Case Study
18 of 31
We conducted an experiment with several large-scale open source
Java programs to show the validity of our security metrics.
We used a tool that we developed to assess the relative security of
different versions of the same program.
Our hypothesis was that a program‟s level of security should, on
average, improve over time, as bugs are fixed and the program code
is improved, although the addition of new security-critical
functionality may cause a worsening of overall security levels.
Case Study – Approach and Annotations
19 of 31
We began with five existing open source Java security projects.
For each project, we chose a specific version which was modified in
a number of subsequent updates, to fix bugs found in the previous
releases.
We manually annotated at the Java source code level a number of
attributes in each project to be „classified‟, choosing attributes whose
names and associated code comments indicated that they are likely
to store confidential data.
We annotated the same attributes for all the different releases of the
same program in order to make our comparisons fair.
Case Study
20 of 31
Case Study
21 of 31
Case Study
22 of 31
Case Study
23 of 31
AUTOMATED TOOL FOR SECURITY
EVALUATION
Part III
24 of 31
26
Annotated Java
source code
compiled using
external Java
compiler
26 of 31
Overview of Java Bytecode Security Analyser
Annotated Java
source code Added
Bytecode
analysed
automatically
27 of 31
28 of 31
CONCLUSION
Part IV
29 of 31
Conclusion
In this work we have described a number of security metrics for object-
oriented programs which are measurable at the level of bytecode
instructions.
Using this approach we capture the exact behavior of a Java program
in the Java Virtual Machine, which gives accurate results.
They provide developers with a simple way of identifying and fixing
security vulnerabilities which might occur from the perspective of
information flow of confidential data.
The security metrics were captures using a tool which analyses Java
bytecode, applied to actual large-scale Java projects.
30 of 31
Thank You
31 of 31