View
142
Download
2
Category
Tags:
Preview:
DESCRIPTION
The goal of this research is to bridge the fields of Augmented Reality and Network Management (including Security), and demonstrate the benefits of using an Augmented Reality interface to improve the coupling of logical data to physical Network Access Devices. The initial problem this research attempts to address is the distancing of users from the physical network infrastructure by traditional Network Management and security systems. This distancing leads to users tending more to the Network Management and security systems instead of the physical hardware.A framework was developed to interconnect with existing management systems and perform data interchange in order to create virtual incarnations, which are then overlaid as three-dimensional representations on to a realtime video stream. Creating an Augmented Reality interface with which the user can view network management and security data, whilst in the presence of the network hardware. Design choices for this framework were partly driven by subjects’ responses to a preliminary survey.In order to evaluate the effect of the framework, an experimental prototype was designed and developed. The prototype implemented a subset of the framework functionality, and was also developed to detect and highlight one style of network-based attack. This prototype was then used by 10 subjects to evaluate the effect of the framework on the defined problem.All subjects were not able to detect and diagnose an attack simulation using traditional Network Management software, but all detected and correctly identified at least 1 attack simulation when using the experimental prototype! 87% of all attack simulations presented with the experimental prototype were identified correctly, with 7 subjects correctly diagnosing all 3 attack simulations. The evaluation provided insight into the effect of the framework, and avenues for future development and research.
Citation preview
AUGMENTED REALITY FOR NETWORK MANAGEMENT AND SECURITY
By
Nathan L. Reynolds
A DISSERTATION
Submitted to
The University of Liverpool
in partial fulfilment of the requirements for the degree of
MASTER OF SCIENCE
06/09/2010
ABSTRACT
AUGMENTED REALITY FOR NETWORK MANAGEMENT AND SECURITY
By
Nathan L. Reynolds
The goal of this research is to bridge the fields of Augmented Reality and Network Manage-
ment (including Security), and demonstrate the benefits of using an Augmented Reality inter-
face to improve the coupling of logical data to physical Network Access Devices. The initial
problem this research attempts to address is the distancing of users from the physical network
infrastructure by traditional Network Management and security systems. This distancing leads
to users tending more to the Network Management and security systems instead of the physi-
cal hardware.
A framework was developed to inter-connect with existing management systems and perform
data interchange in order to create virtual incarnations, which are then overlaid as three-
dimensional representations on to a real-time video stream. Creating an Augmented Reality
interface with which the user can view network management and security data, whilst in the
presence of the network hardware. Design choices for this framework were partly driven by
subjects’ responses to a preliminary survey.
In order to evaluate the effect of the framework, an experimental prototype was designed and
developed. The prototype implemented a subset of the framework functionality, and was also
developed to detect and highlight one style of network-based attack. This prototype was then
used by 10 subjects to evaluate the effect of the framework on the defined problem.
All subjects were not able to detect and diagnose an attack simulation using traditional Network
Management software, but all detected and correctly identified at least 1 attack simulation
when using the experimental prototype! 87% of all attack simulations presented with the ex-
perimental prototype were identified correctly, with 7 subjects correctly diagnosing all 3 attack
simulations. The evaluation provided insight into the effect of the framework, and avenues for
future development and research.
I hereby certify that this dissertation constitutes my own product, that where the language of
others is set forth, quotation marks so indicate, and that appropriate credit is given where I
have used the language, ideas, expressions, or writings of another.
I declare that the dissertation describes original work that has not previously been presented
for the award of any other degree of any institution.
Student, Supervisors and Classes:
Student name: Nathan L. Reynolds
Student ID number: 1033161
GDI name: Yongge Wang
RMT (GDI) class ID: ComputingReserachMethodsTraining.
DA name: Taly Sharon
DST (DA) class ID: ComputingAdvisorClass.
DECLARATION
that this dissertation constitutes my own product, that where the language of
others is set forth, quotation marks so indicate, and that appropriate credit is given where I
have used the language, ideas, expressions, or writings of another.
the dissertation describes original work that has not previously been presented
for the award of any other degree of any institution.
Signed,
Nathan L. Reynolds
Supervisors and Classes:
Nathan L. Reynolds
1033161
Yongge Wang
ComputingReserachMethodsTraining.2010.01.28.202
Taly Sharon
ComputingAdvisorClass.2008.11.27.214
that this dissertation constitutes my own product, that where the language of
others is set forth, quotation marks so indicate, and that appropriate credit is given where I
the dissertation describes original work that has not previously been presented
Nathan L. Reynolds
ACKNOWLEDGEMENTS
First, I would like to acknowledge the support, encouragement and understanding that
my wife and son, Alison and Austin, have shown for the past three years. Without their selfless
attitudes I would not have been able to complete this undertaking. Austin, you’ve only known
life with a dad hard at study. I’m looking forward to our new found time together.
I would also like to thank my dad, Tony, whose encouragement throughout my child-
hood and hacking of my code helped me find the passion for Information and Computer Secu-
rity. Thanks also to my mum, Gerry, for reminding me as a child, that there is a world beyond
computing. I would like to thank my mother-in-law, Sheila, who offered support and congratula-
tions whenever I received grades.
I would also like to acknowledge the support of all the Laureate Online Education staff
that have assisted me, guided me, and encouraged me throughout my study. With special
thanks to Taly Sharon for her advice, encouragement and patience throughout the dissertation
process. I’d like to thank the professional course facilitators, whose style and example consis-
tently provoked the best quality work possible from me, especially Yongge Wang and Lelia
Lividas. Also, there were many classmates I encountered throughout the programme who in-
spired me, and through their responses, encouraged well researched and provoking debate,
thank you. Thank you also to the student support and enrolment teams, who’ve handled every
one of my queries expertly.
Thank you to my employer, Rockwell Automation, for the support and the opportunity
given, as well as to all my colleagues and professional contacts and all those who set aside
time to participate in the preliminary survey and the framework evaluation. Your participation is
much appreciated.
v
TABLE OF CONTENTS
Page
LIST OF TABLES viii
LIST OF FIGURES ix
Chapter 1. Introduction 1
1.1 Scope 1
1.2 Problem Statement 1
1.3 Approach 2
1.4 Outcome 3
1.5 Document Structure 3
1.6 Chapter Summary 4
Chapter 2. Background and review of literature 5
2.1 Virtual Reality 5
2.2 Augmented Reality 5 2.2.1 Mobility ............................................................................................................... 7 2.2.2 Data Representation ............................................................................................. 8 2.2.3 Tracking ............................................................................................................... 9 2.2.4 Collaboration ..................................................................................................... 10
2.3 Network Management and Security 10
2.4 Current State 12
2.5 Research Field Inter-Relationships 15
2.6 Related Work 17
2.7 Chapter Summary 18
Chapter 3. Framework analysis and design 19
3.1 Preliminary Survey 19 3.1.1 Operational Commitment ................................................................................... 19 3.1.2 Management Systems ........................................................................................ 21 3.1.3 Summary and Conclusions ................................................................................ 22
3.2 Proposed Solution 23
3.3 Design Methodology 23
3.4 User Interface 24 3.4.1 Primitives ........................................................................................................... 26
3.5 Component Design 28 3.5.1 Fiducial Marker .................................................................................................. 28 3.5.2 AR Viewer ......................................................................................................... 29 3.5.3 AR Middleware .................................................................................................. 30 3.5.4 Data Flow and Inter-Component Transport ....................................................... 31
vi
3.5.5 AR Viewer Identification and Authentication ................................................... 32
3.6 Chapter Summary 32
Chapter 4. Prototype Design and Implementation 33
4.1 Methodology 33
4.2 Scope 33
4.3 Design 33 4.3.1 Environment....................................................................................................... 34 4.3.2 eXtensible Markup Language ............................................................................ 35
4.4 Implementation 35 4.4.1 ActionScript3 ..................................................................................................... 36 4.4.2 PHP: Hypertext Preprocessor............................................................................. 37 4.4.3 Hardware ............................................................................................................ 37
4.5 Data 37 4.5.1 Algorithms ......................................................................................................... 38
4.6 User Interface 39
4.7 Development Network 41
4.8 Chapter Summary 42
Chapter 5. Evaluation and Results 43
5.1 Testing and Evaluation Network 43 5.1.1 Additional Software ........................................................................................... 44 5.2.1 Normal State ...................................................................................................... 44 5.2.2 Client Attacks .................................................................................................... 45 5.3.1 Consent and Initial Survey ................................................................................. 49 5.3.2 Attack Simulations ............................................................................................. 51 5.3.3 Post Simulation Survey ...................................................................................... 54
5.4 Evaluation Conclusion 61
Chapter 6. Conclusions 63
6.1 Conclusions 63
6.2 Lessons Learned 64
6.3 Prospects for Further Work 65
6.4 Summary 66
REFRENCES CITED 68
Appendix A. Preliminary Survey 71
A.1 Briefing 71
A.2 Questions 72
A.3 De-Briefing 74
A.4 Results 74
vii
Appendix B. Set-up of the Evaluation Environment 75
B.1 Installation of the AR Middleware 75
B.2 Installation of CactiEZ 77
B.3 Client Configuration 78
B.4 Preparing the Environment 82
Appendix C. Framework Evaluation Survey 83
C.1 Briefing 83
C.2 Questions 83
C.3 Functional Testing 85
C.4 Response to the Presented Framework 86
C.5 Improvement Feedback 88
C.6 De-Briefing 88
C.7 Results 88
viii
LIST OF TABLES
Page
Table 1: Primitive shapes and their associated meaning within the framework ......... 26
Table 2: Primitive shapes and their associated meaning within the framework ......... 27
Table 3: Primitive colours and their associated meaning within the framework ........ 27
Table 4: List of software used in the developing the prototype ................................. 36
Table 5: List of hardware to be used in the prototype ............................................... 37
Table 6: Relevant Object Identifiers (OIDs) as data sources .................................... 38
Table 7: Additional software unrelated to direct development .................................. 41
Table 8: Additional software required for testing ..................................................... 44
ix
LIST OF FIGURES
Page
Figure 1: Approach taken for execution of the project ............................................... 2
Figure 2: Simplified representation of an RV Continuum (Milgram et al. 1994, p.
283) .................................................................................................................... 5
Figure 3: A comparison of Human-Computer Interface (HCI) styles (Rekimoto &
Nagao 1995, p. 30) .............................................................................................. 7
Figure 4: Example of Fiducial markers from Wagner (2007, p. 45) .......................... 10
Figure 5: Logical depiction of 4D Architecture (Yan et al. 2007, p. 2) ..................... 12
Figure 6: Mac Track plug-in view in Cacti Network Management System ............... 13
Figure 7: Host status detail in Nagios XI Network Management System .................. 14
Figure 8: Physical incarnation of ar_switch host ...................................................... 15
Figure 9: Distribution of operational hours .............................................................. 20
Figure 10: Operational tasks requiring physical access to devices ............................ 21
Figure 11: Network Management System installation count .................................... 22
Figure 12: Security Information and Event Management system installation count .. 22
Figure 13: Mock-up UI of the Framework's Augmented Reality Interface ............... 24
Figure 14: Mock-up user interface, with port information callout activated. ............. 25
Figure 15: Use-Case for ambient interface. .............................................................. 26
Figure 16: Component inter-connections ................................................................. 28
Figure 17: UML Statechart for AR Viewer .............................................................. 29
Figure 18: UML Statechart for AR Middleware ....................................................... 31
Figure 19: Distributed component data flow, including a third-party Network
Management System ......................................................................................... 31
Figure 20: Use-Case for experimental prototype. ..................................................... 34
Figure 21: Prototype component inter-connections. ................................................. 34
Figure 22: Prototype UI displaying two virtual incarnations .................................... 40
Figure 23: Prototype UI displaying a warning status ................................................ 40
Figure 24: Prototype UI displaying an error status ................................................... 41
Figure 25: Network diagram of prototype development network.............................. 42
Figure 26: Network diagram of prototype evaluation simulation .............................. 43
Figure 27: Augmented reality interface depicting normal network state. .................. 45
Figure 28: Attack from a single source to multiple targets. ...................................... 46
Figure 29: Attack from a single source to a single target. ......................................... 47
Figure 30: Attack from multiple sources against all other clients. ............................ 48
Figure 31: Untargetted attack from single source against all other hosts. .................. 49
Figure 32: Network Management Systems Intallation Count ................................... 50
Figure 33: Security Information and Event Management Installation Count............. 50
Figure 34: High Level Categorization of Operational Tasks ..................................... 51
Figure 35: Correct identifications using the AR prototype ....................................... 53
Figure 36: Detection times using the AR prototype .................................................. 53
Figure 37: Would the framework improve your network management environment?57
Figure 38: Would the framework improve your Security Information and Event
Management environment? ............................................................................... 58
Figure 39: Was the prototype easy to use? ............................................................... 59
Figure 40: Question 1 .............................................................................................. 72
Figure 41: Question 2 .............................................................................................. 72
Figure 42: Question 3 .............................................................................................. 73
x
Figure 43: Question 4 .............................................................................................. 73
Figure 44: Question 5 .............................................................................................. 73
Figure 45: Question 6 .............................................................................................. 74
Figure 46: Turnkey Linux Configuration Console ................................................... 75
Figure 47: BackTrack 4’s Start NETWORK option ................................................. 79
Figure 48: BackTrack 4’s Setup SSHD option ......................................................... 80
Figure 49: Resetting root’s password using passwd ................................................. 81
Figure 50: Fiducial marker for Cisco Ethernet switch .............................................. 82
Figure 51: Question 1 .............................................................................................. 83
Figure 52: Question 2 .............................................................................................. 84
Figure 53: Question 3 .............................................................................................. 84
Figure 54: Question 4 .............................................................................................. 85
Figure 55: Questions 5 thru 8 .................................................................................. 86
Figure 56: Question 9 .............................................................................................. 87
Figure 57: Question 11 ............................................................................................ 87
Figure 58: Question 13 ............................................................................................ 87
Figure 59: Question 15 ............................................................................................ 87
Figure 60: Question 17 ............................................................................................ 87
Figure 61: Question 19 ............................................................................................ 88
1
Chapter 1. INTRODUCTION
The project reviews the development of Mixed Realities (MR), and the potential benefits
to operational effectiveness of Network Management and security personnel. This project
also presents a framework and experimental prototype for an Augmented Reality (AR)
interface to network management and security data, and an evaluation of the framework.
This chapter presents the project scope, the problem statement, and the proposed ap-
proach and outcome.
1.1 Scope
This work attempts to bridge the fields of Augmented Reality (AR) and Network Manage-
ment (including Security). Demonstrating potential benefits in providing situational data
relating to physical Network Access Devices (NADs) for hands-on network management
and security incident response. This will be achieved through a framework for an AR in-
terface for coupling Network Management and Security data with physical NADs within
data centres or process networks. The presented framework will be implemented as an
experimental prototype in an isolated experimental network. The prototype capability will
focus upon coupling of network management and security data to physical assets, and
will not be a full implementation of the framework. Evaluation of the framework will include
user testing of the prototype within an experimental network using a number of scenarios,
and through user observations.
1.2 Problem Statement
Network Management Systems (NMSs) and Security Information and Event Management
(SIEM) systems are primarily presented in windowed Graphic User Interfaces (GUIs) or
Virtual Reality (VR) simulations. NMSs are typically focused upon controlling, monitoring,
alerting and reporting upon the link and flow state of NADs (Haggerty & Seetharaman
1998, p. 73 – 74). Whilst SIEM systems are typically used to manage and correlate data
2
(Kent 2006, p. 3-2) from sources not associated with traditional network management.
SIEMs focus upon event data from perimeter enforcement points (E.G. Firewalls), hosts,
Intrusion Detection / Prevent Systems.
When NADs are local to operational personnel, it is common that data required to support
a physical interaction (such as physical disconnection of a host) can be time consuming
to interpret. Such information is traditionally presented via UI types that inherently dis-
tance operatives from the physical network infrastructure. Users tend more to the man-
agement systems, instead of to the physical infrastructure. In order to provide more value
for NMS and SIEM data as a tool, a different approach to presenting the data is required.
1.3 Approach
This project reviews the fields of MR and network management research, and aims to
present a measurably effective framework and prototype to demonstrate potential benefits
of AR interfaces for viewing network management and security data in relation to physical
network infrastructure. This framework consists of the design of an AR solution which can
retrieve data from multiple sources via standard protocols, and create virtual incarnations
of data overlaid on to a live video stream to augment the NADs. Figure 1 depicts the ap-
proach taken for the execution of the project, including the design, development, and
evaluation phases.
Figure 1: Approach taken for execution of the project
The methodology used reflected the project’s focus upon Human-Computer-Reality inter-
action, and consists of a modular design which supports distributed components, multiple
data sources and managed delivery. Design choices in the framework were driven by the
results and analysis of a preliminary survey which was used to collect data regarding op-
erational support of NMS and SIEM systems.
3
Experimental prototyping is used to implement a proof of concept installation of the
framework, in order to conduct an evaluation of the framework. The prototype was tested
through a process of functional testing using various states of network management data.
An evaluation of the framework was conducted through an on-line survey which consisted
of interactive attack simulation scenarios using a freely available NMS, and the prototype
AR Network Management and Security software. Data from this evaluation was used to
measure the effectiveness of the proposed framework and identify areas of improvement.
1.4 Outcome
The scholarly contributions of this project consist of the presented framework for an AR
for network management and security. The framework is demonstrated through the
evaluation of an experimental prototype. The prototype has been used to confirm that
physical NADs can be augmented with network management and security data to assist
in the diagnosis of, and the remediation of faults. Illustrating the benefits of reducing com-
plexity and the divide between logical information and physical presence in data-centres
and process areas. This paradigm change in network management and security inter-
faces can assist first responders in identifying and handling incidents, whilst maintaining
mobility, increase the ability to collaborate and reduce training requirements.
1.5 Document Structure
This dissertation document is organised as follows. The next chapter, Chapter 2 – Back-
ground and review of literature presents a review of network management and security
and of existing MR paradigms focusing on AR. Also, related academic work, and the cur-
rent state of commercial management products. Following, Chapter 3 – Framework
analysis and design provides an analysis of the preliminary survey, and details of the pro-
posed framework. Followed by Chapter 4 – Prototype design and implementation details
the design of the experimental prototype, and the implementation of the prototype in an
4
isolated development network. Chapter 5 – Results and evaluation presents an analysis
of the framework evaluation survey results and feedback from users. Finally, Chapter 6 –
Conclusions details the summary of the project, including identified gaps in the framework
and potential opportunities for future improvements and research.
The appendices are organised as follows. Appendix A – Preliminary Survey details the
questions posed in the preliminary survey, and the results of the survey. Followed by Ap-
pendix B – Set-up of the Evaluation Environment details the configuration of the isolated
evaluation network. Finally, Appendix C – Framework Evaluation Survey details the ques-
tions posed in the evaluation survey, and the results of the evaluation.
1.6 Chapter Summary
The next chapter presents a literature review of existing research in the fields of MR and
network management, a summary of related work, and a review of the current state of
network management GUIs.
5
Chapter 2. BACKGROUND AND REVIEW OF LITERATURE
This chapter presents a review of the development of the AR interface paradigm, inherent
benefits for the realm of network management and security, and a discussion on network
management protocols, and architectures.
2.1 Virtual Reality
The concept of MR is depicted as a Reality-Virtuality (RV) continuum in Milgram et al.
(1994, p. 283). Figure 2 illustrates an RV continuum, in which one end of the continuum
represents environments which incorporate predominantly real-world, or primarily physical
incarnations. Whilst the opposing end of the continuum represents environments which
are predominantly Virtual Reality (VR), or consisting of primarily virtual incarnations.
Figure 2: Simplified representation of an RV Continuum (Milgram et al. 1994, p. 283)
VR can be used to represent both data and virtual incarnations of physical objects (Conn
et al. 1989, p. 7 – 8). However because VR consists primarily of media presentation and
little media input from reality, it fails to address the relationship between physical incarna-
tions, and virtual incarnations of associated data.
2.2 Augmented Reality
Coupling of data to physical incarnations can be achieved by augmenting reality. AR is a
term which was coined Tom Caudell and David Mizell from Boeing (Höllerer & Feiner
2004, p. 3) and the term describes the use of media-based representation of data. Typi-
cally, AR is used to describe graphical representations overlaid on to a still or moving im-
age, thereby augmenting the image or video stream with data which is not immediately
physically apparent, yet can be contextually relevant. Because AR primarily takes input
6
from the real world, and then applies media, it is a concept which is unique in that the
process of coupling data to a physical incarnation is an inherent trait (Mackay 1998, p. 13)
Figure 3 presents four separate interface paradigms. Figure 3a represents a typical win-
dowed GUI, in which the user interacts entirely through manipulating two-dimensional
virtual incarnations. This level of interface does not take inputs from physical incarnations,
and therefore does not perform coupling of incarnations, or present situation-based con-
text. Rekimoto & Nagao (1995, p. 29.) states “GUIs cannot deal with real world contexts,
GUIs assume an environment composed of desktop computers and users at a desk,
where the real world situation is less important.” This statement has lost value in the in-
tervening time, as mobility has become ubiquitous; however the gap is still relevant, as
even standard mobile computing draws user attention to the computer, and away from the
real world situation.
Figure 3b demonstrates a VR interface which encapsulates the user’s interaction with the
real world (Rekimoto & Nagao 1995, p. 30.) This isolation of the user’s senses from the
real world can be addressed by implementing an Augmented Virtuality (AV) (Milgram &
Kishino 1994, p. 4).
Figure 3c illustrates ubiquitous computing, in which computers are prevalent in the real
world (Weiser 1993), and interaction with the real world drives interaction with the inte-
grated computers.
Finally, Figure 3d presents an AR in which the real world is viewed through the computer,
which utilizes the real world as an input, which can then be augmented with data to create
an output to be consumed by the user. The computer has become something in which the
real world is sampled through.
7
Figure 3: A comparison of Human-Computer Interface (HCI) styles (Rekimoto & Nagao
1995, p. 30)
The AR interface paradigm is not solely a human-computer interface paradigm, but
through using the real world as an input, changes the user experience of the real world.
The interface paradigm therefore becomes a Human-Computer-Reality Interface para-
digm. Brooks (1996, p. 64) described the inherent effect of an AR system as ‘Intelligence
Amplification (IA)’. As AR can be used to complement reality with data that is typically
hidden, or un-correlated, this data can then be presented in a situational context.
AR research primarily focuses upon applying visual elements in which to represent data
or transform the perception of the physical incarnation. However, multiple senses can be
utilized individually, or in a complementary fashion (Azuma 1997, p. 9 – 10). Audio and
tactile stimulus can be used along-side visual stimulus; however additional points of hu-
man interaction will require specialised equipment (Azuma et al. 2001), further pushing an
implementation beyond the capabilities of consumer hardware.
2.2.1 Mobility
Ubiquitous mobile computing is pre-requisite for the adoption level of AR as a viable al-
ternate interface (Wagner 2007.) In recent years, there has been convergence in the Per-
8
sonal Display Assistant (PDA) and Mobile Telephony fields which has yielded powerful
mobile computing platforms which are also highly-connected using a multitude of wireless
protocols for connectivity.
Many of these consumer level devices – such as Apple’s iPhone and Google’s Android-
based devices – now have AR capable applications available (Srinivasan et al. 2009).
Although tracking within these applications is commonly driven by Global Positioning Sys-
tem (GPS) and digital compass data. The common form-factor is similar to that presented
by Wagner (2007, p. 4) as a ‘Handheld AR’, a flat-screen device with an integrated cam-
era. Such devices are used to create a form of ‘see-through interface’ (Bier et al. 1993), in
which the user uses the device to sample both the AR interface, and reality (Wagner
2007.) This sampling of virtual incarnations, and physical incarnations simultaneously
gives the user a natural flexibility in choosing when and when not to view data. Using the
device as a ‘cursor’ (Wagner 2007) to highlight a physical incarnation and then discarding
the AR to access the physical incarnation.
Development of handheld AR for common mobile devices has also become less incum-
bent, as frameworks and toolkits for both low and high level programming languages are
more widely available (Wang et al. n.d.).
2.2.2 Data Representation
By its definition, an AR interface must combine the virtual with reality (Azuma 1997, p. 2).
To effectively combine and maintain value of both ‘feeds’, an AR implementation should
not detract from the immersion of reality, but when displaying data, it must render it in a
fashion which is noticeable and intuitive. Wagner (2007, p. 171) states “While it is tantaliz-
ing to create unique user interfaces that are optimal for the specific applications, it is more
important to stick to the user interface conventions of the target device.” Whilst this
statement referred to Operating System (OS) and User Interface (UI) widget variance be-
tween platforms, it also reflects that an interface for reality should be augmented in a con-
sistent fashion.
9
Data in an AR can be expressed in a multitude of languages, at different levels of abstrac-
tion. The Extensible 3D (X3D) standard (web|3D n.d.) is an eXtensible Mark-up Language
(XML) based standard draft, which supersedes the Virtual Reality Modelling Language
(VRML). X3D can be used to describe virtual incarnations, by defining scene information
such as placement of virtual objects, texture, colour, size, ETC. Before rendering of virtual
objects, network management and security data sources can be presented as a series of
sensors and actuators (EEML.org 2008) through the use of the Extended Environments
Markup Language (EEML). EEML is a schema which is used to describe sensor data
format from physical or virtual incarnations (EEML.org 2008.) EEML could be used as a
data abstraction layer to provide vendor agnostic representation of data, prior to conver-
sion to virtual incarnations in an AR environment.
2.2.3 Tracking
In order for the AR application to determine where to overlay data in an image, the appli-
cation must be able to determine the orientation, and positioning of the viewing device in
relation to the target object. This location data must be sampled at an appropriate gradi-
ent (e.g., site, suite or network equipment rack) using a suited model (Mantoro & Johnson
2003, p. 47 – 53) in order to best determine the position of the user in relation to their sur-
roundings and target objects (Fay 2004, p. 57).
Fiducial marker tracking is commonly used in handheld AR, when operating in a prepared
environment, primarily because of the reduced CPU utilisation over other tracking tech-
niques. Figure 4 shows three types of Fiducial markers (from left to right); the template
marker, an ID marker which is used to represent 12 bits and The DataMatrix ISO stan-
dard marker, which can represent dense patterns of data (Wagner 2007, p. 45).
Figure 4: Example of
ISO DataMatrix markers as Fiducial
curity could be deployed as a dual
set-tagging solutions for asset inventory.
potential to distribute augmentation
mobile device and a server.
2.2.4 Collaboration
Due to inherent nature of handheld AR, direct human
gral to a solution, whilst simultaneous assisted interaction can also take place. Fuhrmann
et al. (1998) identifies the potential for increased collaboration through augmented real
ties; however the research focuses towards the
Brown et al. (2003) presents an event
asynchronous communication, in order to better handle sporadic network connectivity.
Wagner (2007) presents similar framework called ‘Muddleware’, which uses an XML
based communication component between clients and server, which can be used to cr
ate multi-user AR applications.
2.3 Network Management and Security
The Simple Network Management Protocol
for querying, and setting data and counters stored in network aware devices.
has been integral to many types of devices since the e
Example of Fiducial markers from Wagner (2007, p. 45)
Fiducial markers in an AR for network management and s
e deployed as a dual-use label, as they are also used in some physical a
tagging solutions for asset inventory. Wagner (2007, p. 16 – 18) demonstrate
potential to distribute augmentation – in particular tracking calculations – between a cl
Due to inherent nature of handheld AR, direct human-to-human collaboration can be int
gral to a solution, whilst simultaneous assisted interaction can also take place. Fuhrmann
potential for increased collaboration through augmented real
ties; however the research focuses towards the VR end of the mixed reality spectrum.
presents an event-driven multi-user AR application using a form of
ion, in order to better handle sporadic network connectivity.
Wagner (2007) presents similar framework called ‘Muddleware’, which uses an XML
based communication component between clients and server, which can be used to cr
Network Management and Security
Simple Network Management Protocol (SNMP) is a common standardised protocol
for querying, and setting data and counters stored in network aware devices. The protocol
has been integral to many types of devices since the early beginnings of the Internet, and
markers in an AR for network management and se-
they are also used in some physical as-
demonstrates the
between a client
human collaboration can be inte-
gral to a solution, whilst simultaneous assisted interaction can also take place. Fuhrmann
potential for increased collaboration through augmented reali-
end of the mixed reality spectrum.
user AR application using a form of
ion, in order to better handle sporadic network connectivity.
Wagner (2007) presents similar framework called ‘Muddleware’, which uses an XML
based communication component between clients and server, which can be used to cre-
(SNMP) is a common standardised protocol
The protocol
arly beginnings of the Internet, and
11
now exists in three versions (Frye et al. 2003). Individual counters are accessed using an
Abstract Syntax Notation One (ASN.1) namespace addressable Object identifier (OID).
Support is also provided for addressing OIDs in a more human readable format by using
standard, vendor and device specific Management information base files (MIBs).
The International Organisation for Standardization’s (ISO) Common Management Infor-
mation Protocol (CMIP) was a competing standard, and can also be utilised over TCP/IP
(Warrier et al. 1990). However, CMIP did not gain the equivalent saturation as SNMP.
Network Management has a multitude of abstracted layers including physical devices, to
network topology, to application data and ad hoc peer-to-peer overlays (Pras et al. 2007,
p. 105), and it is clear that one Human Interface paradigm will not be adequate for man-
agement of geographically distributed components and metadata relevant to Business
Support Systems (BSS), to minutiae of individual NADs and data (and metadata) for Op-
erations Support Systems (OSS).
These layers of abstraction in terms of control and management can be controlled by a
centralized decision process as described in Greenberg et al. (2005) as the ‘4D Architec-
ture’. The 4D Architecture defines four sub-planes of the control plane of the network. The
sub-planes (as shown in Figure 5) are decision, dissemination, discovery, and data
(Greenberg et al. 2005, p. 47). The Data sub-plane is the state capabilities of the network
infrastructure. The Discovery sub-plane represents the ability of network infrastructure to
discover logical connectivity (Yan et al. 2007, p. 2). This includes neighbour discovery
protocols, and route discovery protocols. The Dissemination sub-plane is used for control
and management data, which can originate from the decision sub-plane. Finally, the De-
cision sub-plane draws information from the discovery, and vicariously the data sub-
planes, and makes decisions based upon the input.
12
Decision
Disemmination
Discovery
Data
Intelligence
Figure 5: Logical depiction of 4D Architecture (Yan et al. 2007, p. 2)
With common practice of decentralised management of NADs (Al-Shaer et al. 2009, p.
37) a significant change in visualization of network management data could be instrumen-
tal to changing this limited form of management, but the effective visualization would re-
quire context and data from OSSs and BSSs. In order to reduce perceived complexity of
network management data visualization (Maltz n.d.), layers of abstraction within the inter-
face are required (Al-Shaer et al. 2009, p. 38).
2.4 Current State
The current state of commercial NMSs and SIEM systems has moved from single access
models, to rich applications, and now as Rich Internet Applications (RIAs). There are mul-
tiple drivers which have led to this state of affairs, most of which are commercially ori-
ented. The requirement for remote support and mobile network engineers has contributed
to zero-footprint tools in which no installation is required for the client. This in culmination
with the rise (or return) to Software as a Service (SaaS) or Utility Computing has given
web-based interfaces (including RIAs) an edge in reduced cost of ownership, and the
ability to outsource infrastructure and operational support.
Management) as a Service provided via Managed Security Solution
also driven by customers (Nicolett & Kavanagh 2009
Both groups of tools available to the commercial market are presented using windowed
GUIs. For example, Figure 6 presents the web
tual incarnation of a Cisco Ethernet switch as it is being monitored for unusual M
cess Layer (Layer 2) traffic.
Figure 6: Mac Track plug
Figure 7 demonstrates the similar
Host Status Detail in Nagios XI for a Cisco Ethernet switch.
ity to outsource infrastructure and operational support. SIM (Security Information
Management) as a Service provided via Managed Security Solution Providers (MSSPs)
Nicolett & Kavanagh 2009, p. 6) as well as providers.
ols available to the commercial market are presented using windowed
presents the web-based GUI of Cacti, demonstrating a vi
tual incarnation of a Cisco Ethernet switch as it is being monitored for unusual Media A
Mac Track plug-in view in Cacti Network Management System
similar web-based GUI of Nagios XI, this image illustrates the
Detail in Nagios XI for a Cisco Ethernet switch.
SIM (Security Information
Providers (MSSPs) is
ols available to the commercial market are presented using windowed
based GUI of Cacti, demonstrating a vir-
edia Ac-
Nagios XI, this image illustrates the
Figure 7: Host status detail in Nagios XI Network Management System
Figure 8 shows an image of the physical incarnation of the
the previous examples, demonstrating that the data displayed from the traditional Network
Management GUIs is difficult to draw contextual meaning when viewing the physical
NAD.
Host status detail in Nagios XI Network Management System
shows an image of the physical incarnation of the ar_switch host showing in
revious examples, demonstrating that the data displayed from the traditional Network
Management GUIs is difficult to draw contextual meaning when viewing the physical
host showing in
revious examples, demonstrating that the data displayed from the traditional Network
Management GUIs is difficult to draw contextual meaning when viewing the physical
15
Figure 8: Physical incarnation of ar_switch host
AR has now received mainstream exposure, with AR applications available on platforms
including: mobile telephones, games consoles, general use computers, and via applets
delivered via the World-Wide Web (WWW). However, few of the popularised applications
perform any useful commercial function such as data representation, and are primarily
entertainment driven.
2.5 Research Field Inter-Relationships
Network management via a VR interface is suited to situations when logistical distances
of physical components inherently prevent the ability to couple data and presence
(Crutcher et al. 1993, p. 13). However, larger cross-continental networks do not solely
exist at such an abstract layer. They are comprised of physical connections, and equip-
ment hosted in data-centre environments, which require hands-on tasks to be performed
on them.
By comparison of a VR network management interface to a traditional 2D windowed GUI,
Crutcher et al. (1993, p. 5 – 7) concludes that a VR interface will provide more ‘direct con-
16
trol and observation’. However, both interfaces are capable of providing an equivalent
level of direct control and observation, as they both do not effectively couple data to
physical incarnations, and only couple data to virtual incarnations (Mackay 1998, p. 13 –
14). This approach assumes that the human operative performs the coupling through a
cognitive process. In order to execute this, the operative must be familiar with the network
topology, and the physical devices. Within the use for network management and security,
users are already isolated from the physical incarnation of their networks by traditional
GUIs, and VR would continue to isolate users.
AR couples both physical incarnations with virtual incarnations, and so would not isolate
users from physical reality, but instead uses reality as a source of data. The use of aug-
mented realities implies mobile technology; unlike typical NMSs. Freeing the users from
interacting with a standard network management workstation (Fay 2004, p. 56) will enable
more agile hands-on network management activities.
Data representation is particularly imperative when handling information representing a
physical device which is considered to be data dense. As an example, whilst Harrop’s &
Armitage’s (2006) representation of network and security events as in-game avatars func-
tions well in a VR interface, it is unlikely to convert well to an AR platform for coupling in-
carnations. Similarly, not all network management and security data will suit to coupling
with a physical device, particularly more abstract meta-data of irrelevant components, or
sum of components. For example, Maltz’s (n.d.) summarization metrics of complexity and
reachability may be applicable to the network as a whole, but may not prove useful when
coupled with individual NADs. The decision on applicability in the case of AR can be ad-
dressed through understanding of workflow, requirement definition and interface design
(Mackay 1998, p. 14.)
Consistent three-dimensional (3D) representation of network management or security
event data is a field which has not received significant study. With no agreed correct ap-
proach, and only examples of VR implementations (which seek to re-define reality as op-
posed to enhance reality), no formal foundation is set for effective ways of communicating
urgency, importance, or anomalous data in AR.
17
Identification and close to real-time visualization of unusual traffic is an emerging area in
Network Management, whereas historically visualization of traffic has been concerned
with traffic trending and resource utilization, primarily for tactical and strategic planning
(Pras et al. 2007, p. 106). There is also potential to improve upon collaboration through
automated context aware data representation, in which data from corresponding
neighbours or end-points could be used in an in-direct augmentation of a physical NAD.
2.6 Related Work
Use of MRs for network management data has been the subject of a mixture of academic
research, yet none specifically address the potential benefits of coupling virtual incarna-
tions with physical incarnations (Jacquet, Bourda & Bellik 2007, p. 164) in order to facili-
tate fault detection, or attack detection in networked environment.
VR GUIs for network management have been explored earlier. Crutcher et al. (1993) pre-
sents a VR interface for the management of geographical distributed broadband connec-
tions, which utilizes context in order to adjust visualization. Harrop & Armitage (2006)
solve the requirements for specialist navigation capabilities by reducing complexity of the
simulation using a 3D cross-platform computer game engine, whilst simultaneously add-
ing implicit collaboration capabilities. Sterritt (2002) details the benefit of a human in cor-
relating network events, including benefits of presence and ability to recognise patterns
and relationships requiring a human-centred network management interface.
Crutcher et al. (1993, p. 16) concluded with the observation “progress is rapid, and we
believe that, by the end of this decade, for many applications, 3D graphics environments
will supersede the 2D systems that are now in common use.” The replacement of one
form of isolating interface with another is a prediction which did not come to fruition.
Fay (2004) reviewed the conceptual benefits of a mobile Network Operations Centers
utilizing AR network management and collaboration tools aboard U.S. naval ships. How-
ever the assessment was purely conceptual and no functional prototype evaluated. This
18
research was also conducted during a period where hardware required for implementing
an AR was still considered specialist hardware and therefore not widely available.
Jacquet, Bourda & Bellik (2007) provide a generic framework for addressing attributes
from multiple aggregates of sensors in a generic manner in a networked ubiquitous envi-
ronment, and include SNMP driven attributes, however, the research did not seek to draw
data from existing NMSs, nor demonstrate the coupling of network management data with
a physical incarnation.
Whilst there are application frameworks for producing AR implementations, no implemen-
tations are available which present network management data in an AR.
2.7 Chapter Summary
This chapter presented the results of the literature search and review, including details of
the current state of commercial UI interfaces for NMS and SIEM systems, and research
into MR interfaces, and the inherent benefits associated with AR interfaces.
The next chapter presents the analysis of the preliminary survey, the design methodology
used and the proposed framework.
19
Chapter 3. FRAMEWORK ANALYSIS AND DESIGN
This chapter presents the proposed distributed framework for an AR capable network and
security management system interface, and middleware component. It includes a prelimi-
nary survey analysis carried out to drive certain framework design.
3.1 Preliminary Survey
A sample of 33 subjects completed the preliminary survey which was directed towards
Information and Communication Professionals (see Appendix A). The primary function of
the preliminary survey was to gain better understanding in the installation base of NMSs
and SIEM systems. Understanding of common levels of operational commitment to net-
work management and security tasks was also sought.
3.1.1 Operational Commitment
Subjects were surveyed in two areas of operational commitment; how many operational
hours were spent in their organization were spent using NMS and SIEM systems and per-
forming hands-on physical work with NADs. Secondly, subjects were asked to categorize
the hands-on physical work performed with NADs. These topics were selected to gain
understanding of the average regular time commitment to both management systems,
and to also understand tasks performed outside of those systems. The task categories
selected were tasks that could be assisted by network management and security data,
and would therefore likely benefit from coupling of assets and data.
Figure 9 illustrates the distribution of hours (per month) between two operational catego-
ries; using NMS and SIEM systems, and performing physical tasks with NADs. From this
information it is possible to note a potential gap in available tools to assist operational
‘hands-on’ tasks. As notably more time is spent performing physical tasks, which the win-
dowed GUIs of existing NMS and SIEM systems would inherently introduce separation
between data and physical incarnations. This separation does not support that physical
access, but instead may introduce errors and inconsistencies.
Figure 9
The categories of physical tasks ar
curity tasks:
Network Management Categories:
• Commissioning Network Access Devices
• De-commissioning Network Access Devices
• Adding network connectivity (‘patching’)
• Removing network connectivity (‘disconnecti
Security Categories:
• Responding to unusual bandwidth utilization
• Responding to unusual usage (not bandwidth related)
• Responding to suspected malicious activity
Figure 10 details the categorization given to the reported operational tasks that required
physical access to devices. With
agement tasks consisted of 64%
Hands-
Hours
57%
Distribution of Operational Hours
9: Distribution of operational hours
physical tasks are divided up into network management tasks, and s
Network Management Categories:
Commissioning Network Access Devices
commissioning Network Access Devices
Adding network connectivity (‘patching’)
Removing network connectivity (‘disconnection’)
Responding to unusual bandwidth utilization
Responding to unusual usage (not bandwidth related)
Responding to suspected malicious activity
details the categorization given to the reported operational tasks that required
With the additional high-level categorization network ma
64% of operational tasks requiring physical access, whilst
NMS & SIEM
Hours
43%-On
Hours
57%
Distribution of Operational Hours
divided up into network management tasks, and se-
details the categorization given to the reported operational tasks that required
categorization network man-
of operational tasks requiring physical access, whilst
security tasks consisted of 31%
that security personnel also require physical access to
dent response.
Figure 10: Operational tasks requiring physical access to devices
3.1.2 Management Systems
Subjects were also surveyed upon which NMS and SIEM systems had been adopted at
their place of work. The results assisted in identifying commonality for integration capabil
ties between the more common products.
base of the selected NMS and SIEM systems amongst the subjects. HP Network Ma
agement Center and Cisco MARS were the two leadin
supports an XML capable Application Programming Interface (API) for integration with
third-party systems (Cisco Syste
HP Network Automation (NA), which provides an API that supports Simple Object Access
Protocol (SOAP) service calls (Hewlett
Removing network
connectivity ('disconnection')
Responding to unusual
bandwidth utilization
Responding to unusual usage
(not bandwidth related)
Responding to
suspected malicious
Operational Tasks Requiring Physical
Access to Devices
31%, and other and unknown tasks at 5%. This demonstrates
that security personnel also require physical access to NADs during investigation or inc
Operational tasks requiring physical access to devices
Management Systems
were also surveyed upon which NMS and SIEM systems had been adopted at
. The results assisted in identifying commonality for integration capabil
ties between the more common products. Figure 11 and Figure 12 show the installation
base of the selected NMS and SIEM systems amongst the subjects. HP Network Ma
and Cisco MARS were the two leading available products. Cisco MARS
supports an XML capable Application Programming Interface (API) for integration with
Cisco Systems n.d.). Whilst HP Network Management Center utilizes
HP Network Automation (NA), which provides an API that supports Simple Object Access
Hewlett-Packard Development Company 2009.)
Commissioning Network
Access Devices
De-commissioning Network
Access Devices
Adding network connectivity
('patching')
Removing network
connectivity ('disconnection')
Responding to
suspected malicious
activity
Other
Unknown
Operational Tasks Requiring Physical
Access to Devices
This demonstrates
s during investigation or inci-
were also surveyed upon which NMS and SIEM systems had been adopted at
. The results assisted in identifying commonality for integration capabili-
show the installation
base of the selected NMS and SIEM systems amongst the subjects. HP Network Man-
g available products. Cisco MARS
supports an XML capable Application Programming Interface (API) for integration with
). Whilst HP Network Management Center utilizes
HP Network Automation (NA), which provides an API that supports Simple Object Access
Figure 11: Network Management System installation count
Figure 12: Security Information and Event Management system installation count
3.1.3 Summary and Conclusions
The results from the preliminary survey illustrate details on the division of op
commitment and tasks to maintaining a network infrastructure. Amongst the subject
more time is being committed to hands
time using NMS and SIEM tools
to fully compliment the working practices of
With 31% of activities requiring hands
is evident that security data is an important source of information to the subjects.
tant information was also gathered regarding the products which were being used by the
2
NMS Installation Count
2 4
SIEM Installation Count
Network Management System installation count
Security Information and Event Management system installation count
and Conclusions
The results from the preliminary survey illustrate details on the division of operational
commitment and tasks to maintaining a network infrastructure. Amongst the subject
more time is being committed to hands-on management and manipulation of NADs, than
time using NMS and SIEM tools. This suggests that the current toolset may not be suited
to fully compliment the working practices of a majority of the subjects.
With 31% of activities requiring hands-on access to NADs being security related tasks, it
is evident that security data is an important source of information to the subjects.
tant information was also gathered regarding the products which were being used by the
8
3 7
1
1210
NMS Installation Count
1 32
1311
SIEM Installation Count
Security Information and Event Management system installation count
erational
commitment and tasks to maintaining a network infrastructure. Amongst the subjects,
s, than
e suited
s being security related tasks, it
is evident that security data is an important source of information to the subjects. Impor-
tant information was also gathered regarding the products which were being used by the
23
subjects. The more popular commercial products identified have capable Application Pro-
gramming Interfaces, which can be used to extract data for use in a third-party systems,
such as an AR interface.
3.2 Proposed Solution
In order to better equip network and security personnel in performing physical ‘hands-on’
tasks with NADs and reduce the gap between virtual and physical incarnations, the pro-
posed solution is to provide existing network and security data in a mobile and contextual
form using AR.
This framework converts data from existing management systems into virtual incarna-
tions, and then overlays three-dimensional representations of those virtual incarnations
on to a video stream of reality, in a prepared environment. Thereby creating an AR inter-
face with which the user can view network management and security data, whilst in the
physical presence of the associated NAD.
As the framework is designed to interface with existing network management and security
systems its function is comparable to those systems. Network management data such as
physical port state and device state which are available in traditional NMSs, will be avail-
able under this framework. Similarly, security data available from traditional SIEM sys-
tems will also be available using this framework. However, this framework will present the
data in a manner which couples the data to the physical incarnation.
3.3 Design Methodology
The design methods used in the framework design reflect the project’s focus upon Hu-
man-Computer-Reality interaction, and the mechanisms for data-interchange and ab-
straction between the multiple components. The design methodology used includes Uni-
fied Modelling Language (UML) Statecharts, a component inter-connection diagram, and
a data-flow diagram describing data
processes. The 4D Architecture definition is also reflected in the design
3.4 User Interface
Figure 13 illustrates both the function of the framework, and the framework’s AR interface
with additional callouts to highlight each element. The environment depicted has been
prepared using Fiducial ISO DataMatrix markers, and coloured cuboids are used to re
resent the state of Ethernet ports by augmenting the
tors. The red cuboid acts as a virtual incarnation, representing an improper port state,
whilst the green cuboids represent proper port state.
Figure 13: Mock-up UI of the Framework's Augmented Reality Interface
As illustrated in Figure 14, the user interface primarily consists of the display of the au
mented video output. Each of the three
object. The user can ‘tap’ one of the virtual incarnations to display more detailed inform
tion. The Port Summary callout will be used sparingly in order to only view information
ISO Data-Matrix markers
Correctly functioning port
flow diagram describing data-interchange between components and sub
The 4D Architecture definition is also reflected in the design methodology
illustrates both the function of the framework, and the framework’s AR interface
ith additional callouts to highlight each element. The environment depicted has been
ISO DataMatrix markers, and coloured cuboids are used to re
state of Ethernet ports by augmenting the Registered jack 45 (RJ45) conne
rs. The red cuboid acts as a virtual incarnation, representing an improper port state,
whilst the green cuboids represent proper port state.
up UI of the Framework's Augmented Reality Interface
, the user interface primarily consists of the display of the au
mented video output. Each of the three-dimensional virtual incarnations is an interactive
object. The user can ‘tap’ one of the virtual incarnations to display more detailed inform
ummary callout will be used sparingly in order to only view information
Correctly functioning
Port not functioning correctly
No data available for port
interchange between components and sub-
methodology.
illustrates both the function of the framework, and the framework’s AR interface
ith additional callouts to highlight each element. The environment depicted has been
ISO DataMatrix markers, and coloured cuboids are used to rep-
connec-
rs. The red cuboid acts as a virtual incarnation, representing an improper port state,
, the user interface primarily consists of the display of the aug-
dimensional virtual incarnations is an interactive
object. The user can ‘tap’ one of the virtual incarnations to display more detailed informa-
ummary callout will be used sparingly in order to only view information
which either cannot be displayed as a 3D primitive, or would prove counter
display as a 3D primitive.
Figure 14: Mock-up user interface, with port information callout activated.
Figure 15 details the primary use
viewing of reality through the AR interface, and the detection of
Fiducial marker is detected, the connected management systems are queried for logical
information, which is then tran
overlay to the video stream.
splayed as a 3D primitive, or would prove counter-intuitive
up user interface, with port information callout activated.
details the primary use-case for the framework. Ambient usage represents the
viewing of reality through the AR interface, and the detection of Fiducial markers. Once a
marker is detected, the connected management systems are queried for logical
information, which is then transformed into virtual incarnations and rendered as
intuitive to
Ambient usage represents the
Once a
marker is detected, the connected management systems are queried for logical
sformed into virtual incarnations and rendered as a 3D
26
User
Ambient UsageDisplay AR
representation of data
«extends»
No fiducial marker
detected
Fiducial marker
detected
«uses» «uses»
Prepare AR
representation of data
«uses»Query management
data
«uses»
«uses»
Activate virtual
incarnation
Identify physical
incarnation
«extends»
Display AR
representation of incarnation's
data
«extends»
«uses»
Figure 15: Use-Case for ambient interface.
3.4.1 Primitives
The primitives for the 3D objects used within the AR UI represent the various port connec-
tors available to NADs. For example, cuboids are used to represent the RJ45 connector.
This enables an alpha blended 3D object to be overlaid on the video stream, whilst mini-
mising the occlusion of physical data in the video stream. Table 1 details 3D primitives
used to represent connections via common Ethernet connectors.
Primitive Shape Intended Meaning
Cuboid Representing connectors that are cuboid in shape, such
as the RJ45.
Dual Cuboid Represents fibre connectors (such as SC Duplex Type)
that have dual connectors. Each cuboid representing a
separate fibre cable.
Dual Cylinder Represents fibre connectors (such as ST Duplex Type)
that have dual connectors. Each cylinder representing a
separate fibre cable.
Table 1: Primitive shapes and their associated meaning within the framework
Information from counters and sources which have more variance to data than a series of
states will be represented using graphs coupled with the corresponding connector. For
example, when viewing bytes received and bytes transmitted for an Ethernet switch port
27
the connector for that port will be augmented with a histogram primitive. The histogram
will consist of two bars, each representing bytes received and bytes transmitted. Table 2
details the graph primitives for the framework.
Primitive Shape Intended Meaning
Histogram A histogram will be used to represent states which have
dual data. For example, bytes in and bytes out will be rep-
resented by a two-bar histogram.
Pie-chart A pie-chart will be used to represent states that have a
least three counters. For example, representation of traffic
proportion based upon Internet Control Message Protocol
(ICMP), User Datagram Protocol (UDP) and Transmission
Control Protocol (TCP) traffic will be represented as a pie-
chart.
Table 2: Primitive shapes and their associated meaning within the framework
As shown in Table 3, a ‘traffic light’ system of colouring has been adopted to represent
the state of virtual incarnations in an easily identifiable manner.
Primitive Colour Intended Meaning
Red Device is experiencing an incident. This incarnation is in a
failed state, or is associated with the source of an incident
Amber / Yellow Device is experiencing an incident. This incarnation is in a
failing state, or is the target of an incident.
Green Device is experiencing an incident, but this incarnation is
in a functioning state.
Table 3: Primitive colours and their associated meaning within the framework
28
3.5 Component Design
In order to interact with existing Network Management and Security Information Event
Management systems, the framework proposed will consist of number of interfaces. Fig-
ure 16 illustrates at a high level the distributed component interconnects in the framework.
The Fiducial Marker is captured by Visual Input, which is interpreted by the AR Viewer.
The AR Viewer then creates a Simple Object Access Protocol (SOAP) request for X3D
data from the AR Middleware. Upon receipt of the request, the AR Middleware then re-
quests XML data via a SOAP request to each of the connected management systems.
The response is then converted to virtual incarnations described as X3D content which is
then rendered by the AR Viewer to Visual Output.
Figure 16: Component inter-connections
3.5.1 Fiducial Marker
In order to reduce processing overhead on a mobile platform, the framework utilizes a
fiducially prepared environment for device identification and tracking. It is a common prac-
tise in many organizations to already prepare environments for the purpose of financial
asset tracking, and so adding Fiducial markers can be integrated into an existing asset
management process workflow, or as a replacement for asset tags.
ISO DataMatrix markers offer enough variance to provide a unique identifier, which will be
used to associate the marker with the virtual incarnations of the device it represents. This
unique identifier should not be a new system, but an existing identifier such as, Partially
or Fully Qualified Domain Name (PQDN or FQDN), IPv4 or IPv6 address.
29
3.5.2 AR Viewer
The distributed nature of the framework will be beneficial to mobility, as data processing
logic will be implemented on a server platform, freeing up the mobile platform processing
resources.
The AR Viewer is a client component which takes video input, detects the presence and
orientation (in relation to the video camera angle) of Fiducial markers, then instigates a
Web Oriented Architecture (WOA) service call to request information regarding detected
markers. The response from the service call is then interpreted into virtual incarnations,
which are overlaid on to the video stream to produce the final visual output.
The initiating service call is a SOAP call to a web service (AR Middleware) which is pre-
sented using the Web Service Definition Language (WSDL). The response to this service
call will be formatted as X3D. The response solely includes 3D geometry data for the AR
Viewer to render. The AR Viewer is therefore reduced to a small amount of data process-
ing and logic, dependent upon responses from the AR Middleware component for instruc-
tions on the manner of rendering a 3D overlay to the video output. Figure 17 depicts a
Unified Modelling Language (UML) Statechart of the AR Viewer component.
No marker detected
Marker detected Initiate AR Middleware call
Receive responseInterpret X3D data
Receive video input
Render visual output
No response
Figure 17: UML Statechart for AR Viewer
As the AR Viewer component is coupled to the AR Middleware component using stan-
dardised WOA service calls, it will be possible to deliver the AR Viewer using either a na-
30
tive code package or a mobile code package, permitting additional choice regarding the
client platform.
3.5.2.1 AR Viewer Hardware Platform
In order to support a Human-Computer-Reality Interface, hardware selection of the client
must take into account factors dependent upon the deployment environment. These fac-
tors will include form-factor, mobility and connectivity. It is anticipated that a beneficial
platform is the next generation of tablet computing devices equipped with a video camera.
3.5.3 AR Middleware
The AR Middleware is a server component which receives requests for information from
the AR Viewer. These requests are simple in format and describe the unique identifier of
the asset, retrieved from the Fiducial marker as it came into view of the video camera.
The AR Middleware then initiates data requests to the associated third-party NMS and
SIEM systems using the most appropriate connectivity mechanism and Application Pro-
gramming Interface (API), as API specifications will be different between vendors. Based
upon the response from the management systems, the AR Middleware then selects a
configuration template, which matches the device model and hardware configuration. The
configuration template is used to map physical ports of the base NAD, and installed
hardware modules to virtual 3D locations offset from the Fiducial marker.
The configuration template is then populated with virtual incarnations of the retrieved
data, and communicated back to the AR Viewer as a response to the initial request. The
AR Viewer then renders the virtual incarnations. In 4D Architecture terms, the AR Mid-
dleware is the Intelligence, which retrieves data via the Discovery plane, and then proc-
esses data to present in the Decision plane (Yan et al. 2007, p. 2).
Figure 18 depicts a Unified Modelling Language (UML) Statechart of the AR Middleware
component.
31
Figure 18: UML Statechart for AR Middleware
3.5.4 Data Flow and Inter-Component Transport
Figure 19 illustrates the data flow of the framework’s distributed components and the
processes which also handle data within each component.
Figure 19: Distributed component data flow, including a third-party Network Management
System
Transport for WOAs is provided using standard WWW protocols, such as the Hypertext
Transfer Protocol (HTTP) and Hypertext Transfer Protocol over Secure Socket Layer or
Transport Layer Security protocols (HTTPS). HTTPS will be used in the framework for
communication between the AR Viewer and AR Middleware, and – where available – be-
tween the AR Middleware and connected NMS and SIEM systems, as the protocol pro-
vides end-to-end session encryption, authenticity and integrity.
32
3.5.5 AR Viewer Identification and Authentication
Due to the coupling of the AR Viewer and AR Middleware component, it is imperative for
uninterrupted operation that AR Viewer requests are made and serviced in a timely man-
ner. This limitation also applies to the initial requests from connect NMS and SIEM sys-
tems, but not subsequent requests for the same devices, as responses can be cached by
the AR Middleware.
Identification and authentication of the AR Viewer to the AR Middleware component is
important to the security of the system as NMS and SIEM data is sensitive and should
remain confidential. However, in order to maintain responsiveness and reduce process
latency, which can be introduced through the cryptographic steps required for strong iden-
tification and authentication, the framework will use a form of token-based identification
and authentication.
The AR Viewer will connect to the AR Middleware, verify the X.509 certificate which is
presented via HTTPS, and then proceed to supply the AR Middleware with a salted hash
of a pre-shared key over the encrypted communications channel. Upon successful au-
thentication, the AR Middleware will return a reusable token, which the AR Middleware
can use for all following requests for the session.
3.6 Chapter Summary
This chapter presented the results of the preliminary survey and the proposed distributed
framework. The next chapter presents the design and implementation of the framework
prototype.
33
Chapter 4. PROTOTYPE DESIGN AND IMPLEMENTATION
This chapter presents the design and implementation of the experimental prototype of the
proposed framework.
4.1 Methodology
The prototype was developed using an exploratory and experimental prototyping ap-
proach, which supported the exploratory nature of the research, whilst allowing for further
evolution in the future.
4.2 Scope
The prototype is designed to demonstrate the benefits of the framework as a graphical
AR interface to network management and security data. Therefore the prototype’s scope
does not cover the full framework, but is limited to a single use scenario and without inte-
gration with third-party NMSs and SIEM products.
In order to demonstrate the potential for the framework to contribute towards network
management and security, the prototype will implement an algorithm to detect ARP cache
poisoning attacks, and adjust the virtual incarnations to highlight the source of the attack.
Manwani (2003, p. 7) states that an ARP cache poisoning attack is “the act of introducing
a specious IP-to-Ethernet address mapping in another host’s ARP cache.” This practise
can be used to create man-in-the-middle attacks.
4.3 Design
The prototype implements a focused use-case scenario, which is shown in Figure 20.
This use-case represents the ambient usage of the AR which includes: detection of Fidu-
cial markers, retrieving network management data, and displaying relevant virtual incar-
nations.
34
Figure 20: Use-Case for experimental prototype.
Figure 21 details the component inter-connections for the prototype. This is similar to the
framework’s inter-connections, except for two primary alterations: Instead of Simple Ob-
ject Access Protocol (SOAP) calls over HTTPS from the AR Viewer to the AR Middle-
ware, the prototype implements HTTP GET requests in order to query the AR Middle-
ware. There is also no integration with third-party NMS and SIEM systems; instead the
AR Middleware component performs SNMP get requests against the evaluation Ethernet
switch to retrieve network management data.
Figure 21: Prototype component inter-connections.
4.3.1 Environment
The development environment required a Fiducial marker. For the prototype ARToolkit
style ID markers were selected. This reasoning for this selection was that the tracking
framework (FLARToolkit) natively supports ID markers and so their use would improve
the rendering frame rate over alternatives. Also, marker variance was not required as the
35
implementation would be limited in scale. The presence of and ID marker enables the
FLARToolkit framework to track position and orientation information of the marker, and
the associated NAD in relation to the position and orientation of the camera. By attaching
these ID markers in specific positions, the 3D locations of the physical ports are assumed
to lie at specific offsets in 3D space from the ID marker.
4.3.2 eXtensible Markup Language
When servicing the connection from the AR Viewer, the AR Middleware component
responds using arbitrary XML, which is then interpreted by the AR Viewer. The XML
response consists of parent elements for each physical port, and child elements which
are used to signal the AR Viewer to how they are to be rendered. The following XML
describes the state of a single port (port number 1) with instructions to render the virtual
incarnation in the colour red:
<physicalports>
<physicalport>
<portnumber>1</portnumber>
<red>1</red>
<yellow>0</yellow>
<green>0</green>
</physicalport>
</physicalports>
4.4 Implementation
In order to support the experimental prototyping process, it was necessary to identify
technologies with existing frameworks and features to support 3D graphics rendering,
video capture and output, Fiducial marker tracking, and Representational State Transfer
(REST). Table 4 details the software packages and frameworks which were used in pro-
ducing the prototype.
36
Software Title Description Web Site
SnmpB SNMP MiB browser and associated
Cisco MiBs
http://sourceforge.net/projects/snmpb/
http://www.cisco.com/public/sw-
center/netmgmt/cmtk/mibs.shtml
Eclipse Galileo Primary cross-platform Integrated
Development Environment (IDE)
http://eclipse.org/
AXDT ActionScript3 capable plug-in for
Eclipse Galileo
http://axdt.org/
FLARToolkit ActionScript3 port of ARToolkit http://www.libspark.org/wiki/saqoosha
/FLARToolKit/en
FLARManager Development framework for FLAR-
Toolkit
http://words.transmote.com/wp/flarma
nager/
Papervision3D Three-dimensional (3D) graphics
library for ActionScript3
http://www.papervision3d.org/
Apache HTTP
Server
Free/Libre Open Source Web server http://httpd.apache.org/
PHP: Hypertext
Preprocessor
Interpreted language for web devel-
opment
http://php.net/
Mozilla Firefox Free/Libre Open Source Web
browser
http://www.mozilla.com/firefox
Firebug /
Flashbug
Web development and debugging
tools for Mozilla Firefox.
http://getfirebug.com/
http://blog.coursevector.com/flashbug
Adobe Flash
Debug Player
Adobe Flash Player for executing
Adobe Flex application
http://www.adobe.com/support/flashpl
ayer/downloads.html
Table 4: List of software used in the developing the prototype
4.4.1 ActionScript3
ActionScript3 language was selected for the development of the AR Viewer component
based upon multiple factors. It has received wide community support for use in web-
based ARs, which is demonstrable by FLARToolkit and FLARManager. The existence of
AR frameworks for ActionScript3 also made the language suitable for rapid prototyping.
The Adobe Flex Software Development Kit (SDK) has also received support as a mobile
platform runtime for Rich Internet Applications (RIAs), and may be a deciding component
in the predicted upcoming tablet computing resurgence.
37
4.4.2 PHP: Hypertext Preprocessor
PHP language was selected for the development of the AR Middleware component. PHP
is well suited to handling of HTTP requests that have been handed off by the web server
and formatting suitable response headers and content. PHP translates scripts which are
requested of a web server. Because of this a network aware server process did not need
to be developed. This made PHP suitable for the AR Middleware component.
4.4.3 Hardware
In order to augment a physical incarnation such as a NAD, additional equipment was re-
quired. The list of sourced equipment is shown in Table 5.
Hardware Description
Cisco 2900XL Series
Switch
SNMP capable switching network access device
Microsoft LifeCam Cinema
HD
Generic web-cam, required to obtain video input
Table 5: List of hardware to be used in the prototype
The Cisco 2900XL Ethernet switch represents the physical incarnations which were aug-
mented in the prototype. The device is also a source of network management data, ac-
cessible via SNMP.
4.5 Data
Due to the ‘real-time’ nature of the interface and the system, the network management
data used was state related, and so therefore was live data from the development envi-
ronment. For the single use nature of the prototype, two tables of information were used
in detecting ARP poisoning attacks. This data is related to the mapping of Internet Proto-
col (IP) addresses to Media Access Control (MAC) address, and determining which inter-
face the MAC addresses were discovered on. Table 6 details the relevant SNMP Object
Identifiers (OIDs) from which this data is retrieved.
38
MiB (OID) Description
iso.org.dod.internet.mgmt.mib-
2.ip.ipNetToMediaTable.* (1.3.6.1.2.1.4.22)
Mapping table for Internet Protocol (IP) address
and associated MAC addresses
iso.org.dod.internet.mgmt.mib-
2.dot1dBridge.dot1dTp.dot1dTpFdbTable.*
(1.3.6.1.2.1.17.4.3)
Bridging table, can be used to determine which
physical interfaces MAC addresses were
learned on
Table 6: Relevant Object Identifiers (OIDs) as data sources
4.5.1 Algorithms
SNMP data from the ipNetToMediaTable and dot1dTpFdbTable are compared in order to
determine the total occurrences of each MAC address from the ipNetToMediaTable to
each MAC address report by the dot1dTpFdbTable. The AR Middleware is responsible for
processing network management data into an XML-based response for the AR Viewer.
The AR Middleware for the prototype identifies ARP Poisoning attacks using this algo-
rithm (in PHP):
// Set a counter for the first loop.
$i = 0;
// Step through each entry of the ipNetToMediaTable.
while( isset( $class_ipNetToMediaTable->ipNetToMediaTable[$i] ) ) {
// Set a counter for the nested loop.
$n = 0;
// Step through each entry of the dot1dTpFdbTable table.
while( isset( $class_dot1dTpFdbTable->dot1dTpFdbTable[$n] ) ) {
// Check MAC addresses from both tables for a match.
if( $class_ipNetToMediaTable-
>ipNetToMediaTable[$i]["ipNetToMediaPhysAddress"] == $class_dot1dTpFdbTable-
>dot1dTpFdbTable[$n]["dot1dTpFdbAddress"] ) {
// Increment the matchCount for the appropriate port.
$class_networkAccessDevice->incrementMatchCount(
$class_dot1dTpFdbTable->dot1dTpFdbTable[$n]["dot1dTpFdbPort"] - 13 );
// Break from this iteration of the loop.
break;
}
$n ++;
}
$i ++;
}
When the matchCount is equal to 1, one match has been found between the ipNetToMe-
diaTable and dot1dTpFdbTable table. This is considered normal, and the output XML will
result in a green virtual incarnation. If matchCount equals 2, this is considered abnormal
and will result in an amber virtual incarnation. Any value greater than 2 will result in a red
virtual incarnation. If additional hubs or switches were connected, their upstream ports
39
would also be represented as red virtual incarnations. However, in a single switch envi-
ronment, the algorithm will not exhibit that behaviour.
The AR Middleware XML response is interpreted by the AR Viewer component, once the
response has been loaded into a data structure. The AR viewers steps through each
physical port element within the XML and alters the virtual incarnation of the correspond-
ing port in order to change its material and visibility. In this manner, virtual incarnations
are re-used and only materials are modified. This bolsters frame rate and responsiveness
as destroying and re-creating cuboids would be a more intensive process. This algorithm
(in ActionScript3) performs this task:
// Step through each physicalport entry.
for each ( var portElement:XML in portList ) {
// If the physicalport entry is for the same port
// of the parent loop...
if ( portElement.portnumber.text() == i ) {
// Determine the appropriate material for the
// Virtual Incarnation.
if (portElement.red.text() > 0 ) {
this.viArray[i].replaceMaterialByName( viRed50Material,
'all' );
} else if( portElement.yellow.text() > 0 ) {
this.viArray[i].replaceMaterialByName( viYellow50Material,
'all' );
} else if( portElement.green.text() > 0 ) {
this.viArray[i].replaceMaterialByName( viGreen50Material, 'all' );
}
// Trace the data for debugging purposes.
trace ("Port " + portElement.portnumber.text() + " R: " +
portElement.red.text() + " Y: " + portElement.yellow.text() +
" G: " + portElement.green.text() );
// Signal the Virtual Incarnation to be
// visible when the scene is rendered.
this.viArray[ i ].visible = true;
}
}
4.6 User Interface
Figure 22 illustrates the functioning interface of the AR Viewer. Two ‘green’ virtual incar-
nations are showing, representing each of the two connections which are also shown in
the video stream output of the UI. Clearly visible and affixed on the left of the Ethernet
switch is the ID marker for tracking purposes.
40
Figure 22: Prototype UI displaying two virtual incarnations
Figure 23 demonstrates the state of the virtual incarnations in the event on an ARP cache
poisoning attack from one host, against the other. As the host connect to Ethernet port 4
is now registering two MAC addresses, the virtual incarnation has changed colour to indi-
cate unusual behaviour. Ethernet port 9’s virtual incarnation is no longer displayed, as the
state data relating to the IP address to MAC address relationship is no longer associated
with the port.
Figure 23: Prototype UI displaying a warning status
41
Figure 24 depicts the same host attacking two other hosts on the switch. As Ethernet port
4 now has three MAC addresses associated with it, the virtual incarnation has become
red, indicating an attack state.
Figure 24: Prototype UI displaying an error status
4.7 Development Network
In order to develop the experimental prototype, it was necessary to execute both the AR
Middleware and AR Viewer on separate devices. By connecting both devices to the
Ethernet switch state data was also generated to facilitate testing. Table 7 lists the addi-
tional platform components which were required to host the AR Middleware component.
Because the AR Viewer is delivered as a compiled Adobe Flash file, any Flash capable
web-browser can be used as a client.
Software Title Description Web Site
VMWare Work-
station
Hardware virtualisation platform http://www.vmware.com/
TurnKey Linux
(LAMP)
GNU/Linux distribution for appliance
based installations
http://www.turnkeylinux.org/
Table 7: Additional software unrelated to direct development
42
The TurnKey Linux LAMP distribution is available as a VMWare image, and is prepared
with LAMP applications Linux, Apache, MySQL and PHP. Therefore, requiring very little
configuration (see Appendix B). Figure 25 illustrates the topology of the development
network.
Figure 25: Network diagram of prototype development network
4.8 Chapter Summary
This chapter presented implementation of the proposed framework in limited scope
through the design and development of the experimental prototype.
The next chapter presents the implementation of the experimental prototype within an
evaluation network environment and the testing performed.
43
Chapter 5. EVALUATION AND RESULTS
This chapter presents the method of functional testing by using attack simulations, and
evaluation of the experimental prototype and framework.
5.1 Testing and Evaluation Network
The experimental prototype was tested in an isolated network environment. Figure 26
illustrates the topology of the network, which is an extension of the existing development
network. The development network was extended by the introduction of 4 additional client
devices. Server Computer, Client Computer and Cisco Ethernet LAN Switch. These de-
vices held the same base set-up and configuration state previously used in the develop-
ment environment. State data from these additional devices led to the creation and dis-
play of more virtual incarnations, and also allowed for flexibility in launching simulated
attacks from alternate devices.
Figure 26: Network diagram of prototype evaluation simulation
44
5.1.1 Additional Software
In order to utilize the additional client computers and to launch attack simulations in which
to test the experimental prototype, additional software was required. Table 8 details the
software required. BackTrack 4 is a GNU/Linux distribution which is pre-loaded with many
dual-use cracking and auditing tools. The clients were booted from a ‘live’ Digital Versatile
Disc (DVD) of BackTrack 4. CactiEZ is a freely available traditional NMS that is also dis-
tributed as a ‘live’ DVD. Finally, Ettercap is a man-in-the-middle attack tool capable of
performing ARP cache poisoning attacks. Ettercap was used to simulate attacks
Software Title Description Web Site
BackTrack 4 GNU/Linux distribution for
penetration testing
http://www.backtrack-linux.org/
CactiEZ Traditional Network Manage-
ment System
http://cactiez.cactiusers.org/
Ettercap Utility for implementing man-
in-the-middle attacks, including
ARP cache poisoning
http://ettercap.sourceforge.net/
Table 8: Additional software required for testing
5.2 Functional Testing
In order to test the experimental prototype, different states were introduced into the
evaluation environment. These states consisted of normal running state, and simulated
ARP cache poisoning attacks.
5.2.1 Normal State
All the devices connected to the evaluation network were booted, and executing their
normal components. Figure 27 depicts the normal idle state as observed by viewing the
Ethernet switch through the AR Viewer application. It should be noted that Port 2x’s vir-
tual incarnation is depicted in an attack state because the server connected to this port
was hosting two virtual machines, therefore accounting for three MAC addresses.
45
Figure 27: Augmented reality interface depicting normal network state.
5.2.2 Client Attacks
In order to test the prototype’s functional capacity to detect and highlight ARP cache poi-
soning attacks. ARP cache poisoning attacks were conducted to generate appropriate
state data. These attack simulations comprised of targeted attacks, in that they attacked
selected hosts, and untargeted attacks, all hosts were attacked simultaneously.
Clients used Ettercap to simulate attacks. The command line switches used were ‘–T’, ‘–
q’ and ‘–M arp:remote’. ‘–T’ instructs Ettercap to only present a text interface, whilst ‘–q’
suppresses packet dump output to the console. Finally, ‘–M arp:remote’ specifies that the
attack mode is ARP cache poisoning.
Figure 28 illustrates the state of the AR interface during an attack simulation. Client 4 is
performing an attack against clients 1 thru 3. Note that client 4’s virtual incarnation has
become red, whilst client 1 thru 3’s Ethernet ports’ virtual incarnations have disappeared
due to non-existent state data for the relevant Ethernet ports.
In order to conduct this simulation, Ettercap was evoked using the following command on
client 4: ettercap –T –q –M arp:remote –i eth1 /192.168.1.68-70/ //
Figure 28: Attack from a single source to multiple targets.
Figure 29 details the interface during another attack simulation. Client 4 is performing an
attack against client 1. Note that as client 4 is only attacking one other host, only two
MAC addresses will have been discovered on the corresponding port, and so the virtual
incarnation has become yellow, instead of red.
In order to conduct this simulation, Ettercap was called using the following command on
client 4: ettercap –T –q –M arp:remote –i eth0 /192.168.1.68/ //
Client 1 Client 4 Client 2 Client 3
Figure 29: Attack from a single source to a single target.
Figure 30 shows the interface during a simulated attack scenario. Client 4 is performing
an attack against clients 1 thru 3, whilst client 3 is performing an attack against clients 1,
2, and 4. Client 4’s attack simulation was executed using the command:
ettercap –T –q –M arp:remote –i eth0 /192.168.1.68-70/ //
Client 3’s attack simulation was executed with the command:
ettercap –T –q –M arp:remote –i eth0 /192.168.1.68-69,71/ //
As two attacks were occurring simultaneously, the Ethernet switch only retained state
data for the last attack to execute. Only Client 3’s attack data is shown.
Client 1 Client 4
Figure 30: Attack from multiple sources against all other clients.
Finally, an untargeted attack was executed from Client 2, using the command:
ettercap –T –q –M arp:remote –i eth0 // //
This attack attempted to target all discovered network hosts, including the AR Middleware
and AR Viewer hosts. Figure 31 shows that as Client 2 was poisoning entries for all hosts;
no state data was available for any port other than Client 2’s port.
Client 1 Client 2 Client 3 Client 4
49
Figure 31: Untargetted attack from single source against all other hosts.
5.3 Framework Evaluation
The evaluation of the framework was conducted via monitoring usage of the experimental
prototype under controlled conditions. This evaluation consisted of three sections and
responses were recorded using an on-line survey which subjects completed (see Appen-
dix D.) Time measurements were also taken as part of a set of interactive scenarios. All
subjects were familiar with Ethernet switches as they were all Information and Communi-
cation Technology or Control System professionals with network infrastructure experience
in either generic network infrastructure or industrial process networks.
5.3.1 Consent and Initial Survey
10 subjects participated in the framework evaluation. They were presented with an elec-
tronic form of consent to inform the subject of evaluation monitoring, and capture the sub-
ject’s permission to monitor the evaluation. The form captured basic subject information
and network management and security systems used, as well as the type of operational
tasks which required physical hands-on access to NADs, which they performed.
Subjects were questioned on their use of
33 illustrate the installation base
results show a high percentage of subjects didn’t know which NMS and SIEM systems
were in place. Whilst initially this suggest
be introduced by the subjects which were more familiar with Control Systems, other than
generic Information Technology systems. With this understanding the results
sent a common semantic separation between the two fields of expertise.
Figure 32: Network Management Systems Intallation Count
Figure 33: Security Information and Event Management Installation Count
NMS Installation Count
SIEM Installation Base
Subjects were questioned on their use of NMSs and SIEM systems. Figure 32 and
installation base for each commercially available suite selected.
results show a high percentage of subjects didn’t know which NMS and SIEM systems
initially this suggests unfamiliarity with the systems this was likely to
be introduced by the subjects which were more familiar with Control Systems, other than
generic Information Technology systems. With this understanding the results may repr
separation between the two fields of expertise.
: Network Management Systems Intallation Count
: Security Information and Event Management Installation Count
1 2
7
NMS Installation Count
1
1
8
SIEM Installation Base
and Figure
These
results show a high percentage of subjects didn’t know which NMS and SIEM systems
with the systems this was likely to
be introduced by the subjects which were more familiar with Control Systems, other than
may repre-
Subjects were also queried on the operational tasks performed which require physical
‘hands-on’ access to NADs. Using the high level categorization which
analysis of the preliminary survey results
tasks were divided up into network management tasks, and security tasks
shows that 81% of the tasks selected were network management related, whilst only 16%
were security related.
Figure 34: High Level Categorization of Operational Tasks
This demonstrates that a majority of the subjects were primarily experienced in diagnosis
of network connectivity related faults
Boolean states (connected / disconnected). Whereas security related tasks were repr
sented as areas of uncertain states outside of normal operational baselines.
This suggests that subjects may respond well to the simple dat
framework, which consists of defined states.
5.3.2 Attack Simulations
Subjects were provided with access to
interface using CactiEZ, and the
High Level Categorization of
Operational Tasks
e also queried on the operational tasks performed which require physical
Using the high level categorization which was defined in the
analysis of the preliminary survey results (see Chapter 3.) The categories of physical
divided up into network management tasks, and security tasks. Figure
shows that 81% of the tasks selected were network management related, whilst only 16%
: High Level Categorization of Operational Tasks
This demonstrates that a majority of the subjects were primarily experienced in diagnosis
of network connectivity related faults, which – for the purpose of this question – consist of
Boolean states (connected / disconnected). Whereas security related tasks were repr
sented as areas of uncertain states outside of normal operational baselines.
This suggests that subjects may respond well to the simple data representation in the
framework, which consists of defined states.
provided with access to a freely available traditional network management
the AR prototype interface, in order to monitor the Cisco
Network
Management
81%
Security
16%
Other
3%
High Level Categorization of
Operational Tasks
e also queried on the operational tasks performed which require physical
was defined in the
of physical
Figure 34
shows that 81% of the tasks selected were network management related, whilst only 16%
This demonstrates that a majority of the subjects were primarily experienced in diagnosis
consist of
Boolean states (connected / disconnected). Whereas security related tasks were repre-
a representation in the
anagement
, in order to monitor the Cisco
52
Ethernet LAN switch in the evaluation network. At an adjusted random time and without
notification, the Evaluation Administrator introduced ARP cache poisoning attacks origi-
nating from different selected clients. Subjects were timed between introduction of the
attack, and their acknowledgement of the attack, with a limit placed at three minutes after
introduction of the attack. The subject’s identification of the source of the attack was also
recorded. Each group of simulations consisted of one simulation using the traditional
Network Management interface, and three simulations using the AR prototype interface.
5.3.2.1 Traditional Network Management Interface
Out of all the simulations executed using the traditional Network Management interface,
none of the subjects were able to identify that an attack was initiated, nor identify a sus-
pected Ethernet port which the attack was originating from. Most subjects continued to
investigate for the full 3 minutes; one gave up, and expressed frustration before continu-
ing with the evaluation. This simulation led some subjects, at the end of the evaluation, to
express disbelief that it was possible to complete the simulation with the traditional Net-
work Management interface. The solution to the simulation was then presented for these
subjects.
In summary, 0% of the attacks were discovered or diagnosed, and therefore response
times were unable to be recorded.
5.3.2.2 Augmented Reality Prototype Interface
Each subject participated in three attack simulations using the AR prototype interface in
order to monitor the Cisco Ethernet LAN switch to detect the attack and identify the sus-
pected Ethernet port which the attack was originating from. All subjects attained at least
one correct identification of the origin of the attack, and all simulations were responded to
within the allotted 3 minutes each. 70% of the subjects achieved a correct identification of
all 3 simulations. Figure 35 details the frequency of correct identifications per subject.
Figure 35: Correct identifications using the AR prototype
87% of all simulations conducted with the AR prototype interface were successfully dia
nosed with the correct originating Ethernet port.
detecting that the attack had been introduced
onds, and a high of 82 seconds.
corded in each simulation.
Figure 36: Detection times using the AR prototype
0
1
2
3
4
5
6
7
0
Su
bje
cts
Correct Identifications per Subject
Correct Identifications Using AR
0
2
4
6
8
10
12
Sim
ula
tio
ns
Response Time (in seconds)
Detection Times Using AR Prototype
: Correct identifications using the AR prototype
% of all simulations conducted with the AR prototype interface were successfully dia
the correct originating Ethernet port. The average response time recorded for
detecting that the attack had been introduced was 21.08 seconds, with a low of
, and a high of 82 seconds. Figure 36 illustrates the frequency of detection times r
: Detection times using the AR prototype
1 2 3
Correct Identifications per Subject
Correct Identifications Using AR
Prototype
Response Time (in seconds)
Detection Times Using AR Prototype
Correct Identification
Incorrect Identification
% of all simulations conducted with the AR prototype interface were successfully diag-
recorded for
seconds, with a low of 3 sec-
detection times re-
Correct Identification
Incorrect Identification
54
The average response time for solely correct answers was 23.5 seconds. 1 of the 30
simulations conducted was unable to be completed due to an irreproducible fault experi-
enced in the prototype.
5.3.3 Post Simulation Survey
Subjects were also surveyed in order to gather qualitative information from their percep-
tions and experience of the evaluation process. The survey included areas of framework
improvement and additional functionality that the subjects felt would be useful.
5.3.3.1 Does the framework presented improve trouble-shooting
times?
70% of the subjects recorded “significant improvement” in trouble-shooting times when
using the AR framework, and the remaining 30% noted “some improvement”.
The additional comments garnered in response to this question provided insight in to the
subjects’ experience with the AR. Comments included:
• “Very easy to identify the originating port of the attack. Easy to see when the at-
tack starts and stops”
• “Much the simpler than wading through complex switch interface sogtware [sic]”
One subjected highlighted an issue with the prototype implementation and its sensitivity to
lighting conditions; “Lighting dependent and no legend for red/green identification”. This
was an intermittent issue with the AR toolkits, which was aggravated by changing ambient
light conditions. This resulted in symptoms such as slow Fiducial marker detection, virtual
incarnations not correctly aligned with the Fiducial marker, and dark areas of the image
being misidentified as the Fiducial marker, which resulted in virtual incarnations appearing
throughout the image.
The comment also suggested a legend for further explanation of the primitives used. The
request for further information via a point of reference was a common trend throughout all
of the feedback. One subject noted, “Graphical representation needs some key for inter-
55
pretation; given that I would expect the framework to improve troubleshooting.” Thereby
highlighting that improving trouble-shooting quality is a worthwhile objective in addition to
reducing response times.
One in-depth comment was, “Simple up/down or red/green indications lead engineers
quickly to the cause of an incident, but there are many tools which give a graphical repre-
sentation of a device with similar outputs that reduce the requirement for additional hard-
ware. The comparison during the evaluation of a full management platform to a [sic]
up/down indication is not as fair comparison. In our organisation we use use [sic] an ex-
tensive tool (Spectrum) that can again provide up/down indication via a simple interface
that is comparable to this tool. Although having a real time interface that can be used to
guide on-site engineers to physical connection from a central management group is a
positive point, as a support group we do hold some pictures of equipment but this quickly
go out of date and are not reliable.” This comment embodies a discussion point raised
during the Literature Search and Review that AR could assist in Network Management
when physical incarnations are within view of the user, yet the Human-Computer-Reality
interface paradigm of AR does not suit the use of network management of geographically
dispersed networks from a central point. The benefits of AR to on-site collaboration and
remote direction also appear important.
One subject noted that the framework did not require experience in order to facilitate de-
tection and diagnosis of the attack, unlike the traditional NMS, “With almost no experience
with the standard tool I was unable to identify and attack. The framework improved this
considerably”. One subject stated that the effect was, “Obvious and immediate.”
5.3.3.2 Does the framework effectively couple logical data with physi-
cal presence?
All subjects observed a positive effect in using virtual incarnations coupled with physical
incarnations. 50% stated there was “very effective coupling”, and 50% stated there was
“effective coupling.” Comments included:
56
• “The graphics over lay on the physical switch makes is very easy to relate with
logical data from the network”.
• “Prtolem [sic] port easy to identifyb [sic]”
• “Yes, quite clearly, without explanation.”
These comments highlight that the coupling was effective, as it was easy to identify the
simulated attack Ethernet ports. These comments also suggest that the primitives used
were easy to understand and infer state information from.
One subject stated, “Yes, real time and up to date.” demonstrating that effective coupling
is not merely a matter of over-laying logical data on to a physical incarnation, but timeli-
ness of the ephemeral state data is also important to the subject to effectively couple.
An issue with the prototype was raised with the comment “Small issue regarding the
counting of port number because of perspective view and no reference (grey blocks)
when fault occurred.” This issue was likely to be induced by the form-factor the prototype
was presented in and the limited visual definition of the prototype. Had the prototype been
presented in a handheld format this subject may have felt it easier to switch between the
AR and reality more quickly, using data from the AR and reality in the simulations.
5.3.3.3 Would the framework improve your network management en-
vironment?
Figure 37 details that 60% of subjects thought that their NMS environment would show
“significant improvement” using the framework, 30% thought “some improvement”, whilst
10% anticipated “no improvement”. This shows that overall; subjects thought the frame-
work would be a beneficial tool to deploy for additional network management functionality.
Figure 37: Would the framework improve your network management environment?
Accuracy of detection and resulting actions, as well as collabo
comments:
• “It [the framework] would help reduce the risk of people making physical errors
like patching.”
• “Yes, for guiding onsite staff to precise devices and connections.”
• “Simplified attack detection”
• “As I have very little experience with SNMP tools the graphical alert would
prove detection of an issue”
This demonstrates a crossover area between
ability of a service resides in interests of effective network management, and is also a key
tenet of security. The framework
sion Detection/Prevention Systems which may be uncoupled from an SIEM suite.
Some improvement
30%
Would the framework improve your
network management environment?
: Would the framework improve your network management environment?
Accuracy of detection and resulting actions, as well as collaboration were common in
would help reduce the risk of people making physical errors
“Yes, for guiding onsite staff to precise devices and connections.”
Simplified attack detection”
“As I have very little experience with SNMP tools the graphical alert would
prove detection of an issue”.
crossover area between NMS functionality, and SIEM. Where avai
ability of a service resides in interests of effective network management, and is also a key
tenet of security. The framework could also be considered a valuable interface for Intr
sion Detection/Prevention Systems which may be uncoupled from an SIEM suite.
Significant
improvement
60%
No improvement
10%
Would the framework improve your
network management environment?
: Would the framework improve your network management environment?
ration were common in
would help reduce the risk of people making physical errors
“As I have very little experience with SNMP tools the graphical alert would im-
Where avail-
ability of a service resides in interests of effective network management, and is also a key
considered a valuable interface for Intru-
One subject drew a direct comparison between traditional network management tools and
the framework stating, “I can see the dramatic impro
agement tools.”
5.3.3.4 Would the framework improve your Security Information and
Event Management environment?
Subjects responded to this question in an unsure manner which is reflected in conditional
statements, uncertainty and assumption
shows that 30% of subjects noted that the framework would show
ment” and 50% noted “some improvement”,
Figure 38: Would the framework improve your Security Information and Event
Comments with conditional statements included:
• “Would need to understan
tions require less skilled staff to monitor items and make escalations based on
simple status.”
• “Would need to see how this data can be linked and correlated with other data
sources..”
No improvement
Would the framework improve your
Security Information and Event
Management environment?
One subject drew a direct comparison between traditional network management tools and
I can see the dramatic improvement upon simply looking at ma
Would the framework improve your Security Information and
Event Management environment?
Subjects responded to this question in an unsure manner which is reflected in conditional
ssumption captured in the comments provided. Figure
shows that 30% of subjects noted that the framework would show “significant improv
improvement”, and finally 20% registered “no improvement.”
: Would the framework improve your Security Information and Event
Management environment?
onditional statements included:
Would need to understand the capabilities of the tool. Simple up/down indic
tions require less skilled staff to monitor items and make escalations based on
“Would need to see how this data can be linked and correlated with other data
Significant
improvement
30%
Some
improvement
50%
No improvement
20%
Would the framework improve your
Security Information and Event
Management environment?
One subject drew a direct comparison between traditional network management tools and
vement upon simply looking at man-
Subjects responded to this question in an unsure manner which is reflected in conditional
Figure 38
improve-
“no improvement.”
: Would the framework improve your Security Information and Event
d the capabilities of the tool. Simple up/down indica-
tions require less skilled staff to monitor items and make escalations based on
“Would need to see how this data can be linked and correlated with other data
These comments denote some uncertainty with the
tential for data interchange between different existing sources of data
also highlight additional areas in which the framework could contain explicit definition. For
example, correlation of attack data is normally one role of an SIEM system, and so ther
fore would not be a function performed by the framework.
data in association with physical incarnations and collaboration with other framework u
ers could be an area of potential extension to the framework.
There were also distinctly positive comments
• “The port status overlay makes it very easy to spot attacking ports or suspected
ports.”
• “Simplified information and management environment”
5.3.3.5 Was the prototype easy to use?
Figure 39 details that 50% of subjects thought the prototype was
thought it was “somewhat easy”
that registered “somewhat difficult”
tacks and identifying sources during the attack simulations.
Figure
Somewhat easy
40%
Somewhat difficult
Was the prototype easy to use?
some uncertainty with the extent of the framework and the p
tential for data interchange between different existing sources of data. These comments
also highlight additional areas in which the framework could contain explicit definition. For
correlation of attack data is normally one role of an SIEM system, and so ther
fore would not be a function performed by the framework. However, correlation of attack
data in association with physical incarnations and collaboration with other framework u
rs could be an area of potential extension to the framework.
There were also distinctly positive comments:
The port status overlay makes it very easy to spot attacking ports or suspected
Simplified information and management environment”.
prototype easy to use?
details that 50% of subjects thought the prototype was “very easy” to use, 40%
“somewhat easy”, and 10% thought it was “somewhat difficult”. The subject
“somewhat difficult” also received a 100% success rate in detecting a
tacks and identifying sources during the attack simulations.
Figure 39: Was the prototype easy to use?
Very easy
50%
Somewhat difficult
10%
Was the prototype easy to use?
framework and the po-
. These comments
also highlight additional areas in which the framework could contain explicit definition. For
correlation of attack data is normally one role of an SIEM system, and so there-
However, correlation of attack
data in association with physical incarnations and collaboration with other framework us-
The port status overlay makes it very easy to spot attacking ports or suspected
to use, 40%
. The subject
also received a 100% success rate in detecting at-
60
Primarily subjects commented that the prototype was easy to work with, with clear data
representation:
• “It was easy to understand and work with the ar.”
• “Clear to interpret results”
• “Visually intuitive.”
Two subjects also noted that additional information regarding the primitives used for data
representation would have been of further benefit. These subjects stated:
• “No explanation of the on screen indicators was given, no ‘click here’ to see a de-
scription of the fault.”
• “Would have liked more explanation of the graphical representation.”
Whilst the framework did support context sensitive callout menus in the event that a user
interacts with a virtual incarnation, this function was not implemented in the prototype.
Finally, one subject commented, “Image jumped around”. This was an intermittent occur-
rence introduced during the simulations and was caused by multiple factors. The thresh-
olds set regarding Fiducial marker tracking and changing ambient light levels were the
primary cause. Additionally, the author noted a small amount of ‘drift’ in the positioning of
the virtual incarnations which were furthest from the Fiducial marker, as errors in tracking
were amplified in virtual incarnations further from the origin.
5.3.3.6 Please detail any additional functionality, or improvements
upon existing functionality that you would add to the frame-
work
This question was posed to provide the subjects with an opportunity to record any further
observations and potential improvements, which could not be categorized through previ-
ous questions. Two subjects again highlighted the benefits of providing additional infor-
mation in regards to identifying the virtual incarnations, and explanation as to their state
changes.
61
• “Once alerted, guidance/identification information for cause or error.”
• “Text labels to the graphic blocks”.
One subjected suggested a potential additional algorithm to assist in remediating another
form of attack, “I think It [sic] would be beneficial to be able to over lay virus worm attacks
in the ar.” This form of usage would be possible if drawing information from an SIEM sys-
tem, which in turn was receiving input from a managed Anti-Virus solution.
5.4 Evaluation Conclusion
The attack simulations conducted with the AR prototype showed an average detection
time of 21.08 seconds and 87% of all attack simulations using the framework resulted in
correct identification of the source of the attack. None of the attack simulations conducted
on traditional Network Management software resulted in a successful outcome. This im-
provement through using the framework was also perceived by the subjects, with all 10
subjects recording improvement in trouble-shooting times and coupling of logical data to
physical incarnations. 90% of subjects also recorded that the prototype was easy to use.
In regards to existing systems, 90% of subjects noted that the framework would improve a
Network Management environment, and 80% thought the framework would also improve
a SIEM environment.
The framework evaluation results were primarily positive for the framework, demonstrat-
ing that detection and accuracy of diagnosis of the ARP cache poisoning attack was sig-
nificantly improved when compared to the traditional NMS which was also tested. Feed-
back suggested positive experiences for most subjects, as well as perceived potential for
adoption and growth of the framework. One common theme prevailed throughout the
comments captured in multiple questions was the requirement for additional information,
both ambient information, and virtual incarnation specific information. Additionally, it was
noted that the AR framework not only reduced time to detect, but also yielded a high level
of accuracy in identification of the source of the attack.
62
5.5 Chapter Summary
This chapter presented the configuration of the evaluation network and the process and
results from functional testing of the experimental prototype. This chapter also presented
the framework evaluation process, from initial questioning to attack simulations, and fi-
nally recording feedback from the subjects. Analysis of the evaluation was also given, and
demonstrated that the framework had a positive effect in aiding detection and identifica-
tion during attack scenarios.
The next chapter presents the project conclusions, including lessons learned, future activ-
ity and avenues for future academic research.
63
Chapter 6. CONCLUSIONS
This chapter presents the summary of the conclusions, and lessons learned. Suggestions
for further research are also given in this chapter.
6.1 Conclusions
The primary conclusion is that the use of an AR interface for viewing network manage-
ment and security data, and coupling the data with physical components has demon-
strated benefits over two-dimensional windowed network management and security GUIs.
These benefits are:
• Improvement in identifying state changes within physical network infrastructure.
The functionality of the framework assists in communicating state changes effi-
ciently to the user.
• Considerable high levels of accuracy in identifying a physical incarnation through
the corresponding virtual incarnation.
• Added value to extending existing NMSs and SIEM systems to include an AR in-
terface.
• Demonstrated the effect of Intelligence Amplification (IA) (Brooks 1996, p. 64)
providing users, with little experience or training, a tool which enables them to still
detect and identify network state changes.
Improvements in detection and identification are shown through the framework evaluation
by the measurement of subjects’ detection response times, and the accuracy of their di-
agnosis. This, along with evaluators’ comments, demonstrated that an AR interface for
network management and security could be an additional complementary tool, which is
beneficial to staff who access NADs physically, as opposed to via isolated (from reality)
traditional interfaces. Such a tool could be useful to staff in data centres and process net-
work installations.
64
Evaluators that specialised in industrial networking solutions, and had little generic Infor-
mation and Communication Technology networking experience noted the ease of use
even with lack of experience. The AR framework provided simple state information repre-
sented in a recognisable fashion, and coupled with the physical device to infer logical
state against a physical presence. This trait circumvented the requirement to understand
the scenario in order to diagnose the attack, and instead evaluators were provided em-
phasized information using graphical representation that effectively portrayed urgency
and negative state.
Subjects of the preliminary survey (see Chapter 3) spent 57% of network management
and security operational time performing hands-on tasks with NADs. The efficiency im-
provements provided by the AR framework could provide considerable benefits to this
operational commitment, including the potential for the framework to reduce the time
commitment, and reduce the level of required training and experience.
6.2 Lessons Learned
The prototype suffered from two key issues which became prevalent during evaluation.
The toolkits used were sensitive to changing ambient light conditions. In order to combat
this, ambient light levels were altered prior to each subject’s participation in the evalua-
tion. Such sensitivity would not be suitable in a production scenario, but may also have
been avoidable in the experimental prototype through the selection of an alternate AR
toolkit. This issue may have also been avoided or minimised through the adjustment of
the thresholds set in the configuration of the selected AR toolkit after installation of the
evaluation network.
The second key issue was reduced accuracy in tracking the orientation of the Fiducial
marker in relation to the camera. This was impacted by many factors, including ambient
lighting, and resolution of processed images. The symptom witnessed by the tracking is-
sue was the observation of ‘drift’ in tracker orientation, in which virtual incarnations would
appear noticeably detached from their physical incarnations. This affect became more
65
pronounced for the virtual incarnations of higher numbered Ethernet ports as they were a
further distance from the 3D scene’s origin.
Finally, the form factor proposed for presentation of the framework was that of a handheld
device such as a tablet computer. However, the prototype was presented by using a web-
cam attached to a laptop. This choice was made partly due to lack of device availability of
a commercial handheld or tablet device which supported full Adobe Flash 10 applications.
Such devices are only now (at time of writing) becoming available with the release of the
Android 2.2 OS. The use of the laptop and the webcam to work around this issue intro-
duced a limitation to the interactive element during the evaluation of the framework. Sub-
jects did not move the webcam, and so the viewing angle of the Cisco Ethernet switch
remained static. This perhaps removed subjects from a level of interaction with the physi-
cal incarnation, by making the transition between an augmented and non-augmented re-
ality unintuitive. This may have also been compounded by subjects responding to the
evaluation survey from the same laptop in which the AR prototype was also executing.
6.3 Prospects for Further Work
The experimental prototype of the framework presented in this dissertation project utilized
simplified primitives for data representation. There is potential for further work to be con-
ducted on the effect of complex data representation, such as histograms and pie-charts,
as virtual incarnations. Such primitives could be used to represent bandwidth utilization
and traffic analysis, thereby providing additional NMS functionality to the AR interface.
Also, complex data representation could be utilised to represent additional security data,
such as Intrusion Detection System alerts or enterprise Anti-Virus console activity. Addi-
tionally there is potential for an investigation of the possible benefits of utilizing animations
– or tweening – to enhance the communication of state data through virtual incarnations.
For example, Ethernet ports that are associated with the source and destination of a TCP
stream could be coupled to each other through animated representation of the traffic flow.
66
In order to resolve the ‘drift’ issue observed with virtual incarnations furthest from the ori-
gin of the 3D scene, it is the author’s opinion that multiple tracking and identification tech-
niques may be combined in order to complement each other. For example, the ISO
DataMatrix Fiducial marker could be utilised for asset identification and placement of ori-
gin, whilst Natural Feature Tracking (NFT) (Nuemann & You 1999, p. 53 – 54) could si-
multaneously be applied to determine location of Ethernet ports and other physical incar-
nations on the identified device. The data utilized for the NFT could be dynamically as-
signed from the device template, specified by data encoded in the Fiducial marker.
Collaboration featured as a prominent subject in the results from the framework evalua-
tion survey. A number of subjects commented on the potential to direct on-site resources.
By ‘tagging’ ports from a centralized GUI, on-site resources could use the AR interface to
physically identify and work with the tagged port. This form of collaboration could be be-
tween a traditional windowed GUI for the centralized Operations Centre, and the AR inter-
faces at remote data centres. Collaboration could be bi-directional, and could also include
AR interface users collaborating together to resolve issues highlighted by an Operations
Centre. It is the author’s opinion that there is potential for further research in the field of
collaborative network management and security tools implemented as an AR.
Finally, as handheld devices which support Adobe Flash 10 are now available, and with
Adobe Flash 10 capable tablet devices coming soon. It is the author’s opinion that there is
potential for additional research to assess the benefits of the form factor upon the frame-
work. Additionally, there is potential for research in the effect that the handheld and tablet
form factors may have upon collaboration in data centres and process networks.
6.4 Summary
The framework presented and evaluated has shown to potentially have considerable
benefits in providing data relating to physical NADs for hands-on network management
and security incident response, through the coupling of logical data – represented as vir-
tual incarnations – with physical incarnations. This effect was noted by the evaluation
67
subjects, who all successfully detected and identified the source of at least one attack
simulation when using the prototype. This was also demonstrated by the comments gar-
nered from the evaluation subjects, which were mostly positive, and highlighted the po-
tential of the framework to perhaps provide additional benefits when coupled with existing
network management and security systems.
68
REFRENCES CITED Al-Shaer, E., Greenberg, A., Kalmanek, C., Maltz, D.A., Ng, T.S.E. & Xie, G.G. (2009)
'New frontiers in internet network management', ACM SIGCOMM Computer Commu-nication Review, vol. 39, no. 5, pp. 37-39. D.O.I.: 10.1145/1629607.1629615
Azuma, R.T. (1997) 'A Survey of Augmented Reality', Presence: Teleoperators and Vir-
tual Environments, vol. 6, no. 4, pp. 355-385. D.O.I.: 10.1.1.35.5387 Azuma, R.T., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B. (2001) ‘Recent
advances in augmented reality’, Computer Graphics and Applications, IEEE, vol.21, no.6, pp.34-47, Nov/Dec 200. D.O.I.: 10.1109/38.963459
Bier, E.A., Stone, M.C., Pier, K., Buxton, W. & DeRose, T.D. (1993) 'Toolglass and magic
lenses: the see-through interface', in Proceedings of the 20th annual conference on Computer graphics and interactive techniques, ACM New York, NY, USA, Anaheim, CA, pp. 73-80.
Brooks, F.P. (1996) 'The computer scientist as toolsmith II', Communications of the ACM,
vol. 39, no. 3, pp. 61-68. D.O.I.: 10.1145/227234.227243 Brown, D., Julier, S., Baillot, Y. & Livingston, M.A. (2003) 'An Event-Based Data Distribu-
tion Mechanism for Collaborative Mobile Augmented Reality and Virtual Environ-ments', in Proceedings of the IEEE Virtual Reality 2003, IEEE Computer Society Washington, DC, USA.
Cisco Systems, I. (n.d.) 'User Guide for Cisco Security MARS Local Controller, Release
4.2.x - Cisco Security MARS XML API Reference', Cisco Systems, Inc. [Online]. Avail-able from: http://www.cisco.com/en/US/docs/security/security_management/cs-mars/4.2/user/guide/local_controller/appxml.html (Accessed 21st May 2010).
Conn, C., Lanier, J., Minsky, M., Fisher, S. & Druin, A. (1989) 'Virtual environments and
interactivity: windows to the future', ACM SIGGRAPH Computer Graphics, vol. 23, no. 5, pp. 7-18. D.O.I.: 10.1145/77277.77278
Crutcher, L.A., Lazar, A.A., Feiner, S.K. & Zhou, M. (1993) 'Management of Broadband
Networks Using a 3D Virtual World', IEEE Parallel and Distributed Technology, pp. 1-25. D.O.I.: 10.1.1.44.9612
EEML.org (2008) 'Extended Environments Markup Language: EEML', Hague Design +
Research Ltd. [Online]. Available from: http://www.eeml.org/#specification (Accessed 19th March 2010).
Fay, J.J. (2004) 'Transforming Fleet Network Operations With Collaborative Decision
Support And Augmented Reality Technologies', Postgraduate, Naval Postgraduate School, United States of America.
Frye, R., Levi, D., Routhier, S. & Wijnen, B. (2003) 'Coexistence between Version 1, Ver-
sion 2, and Version 3 of the Internet-standard Network Management Framework', Internet Engineering Task Force [Online]. Available from: http://datatracker.ietf.org/doc/rfc3584/ (Accessed 26th March 2010).
Fuhrmann, A., Löffelmann, H., Schmalstieg, D. & Gervautz, M. (1998) 'Collaborative
Visualization in Augmented Reality', IEEE Comput Graph Appl, vol. 18, no. 4, pp. 54-59. D.O.I.: 10.1109/38.689665
Greenberg, A., Hjalmtysson, G., Maltz, D.A., Myers, A., Rexford, J., Xie, G., Yan, H.,
Zhan, J. & Zhang, H. (2005) 'A clean slate 4D approach to network control and man-agement', ACM SIGCOMM Computer Communication Review, vol. 35, no. 5, pp. 41-54. D.O.I.: 10.1145/1096536.1096541
69
Haggerty, P. & Seetharaman, K. (1998) 'The benefits of CORBA-based network man-agement', Communications of the ACM, vol. 41, no. 10, pp. 73-79. D.O.I.: 10.1145/286238.286250
Harrop, W. & Armitage, G. (2006) 'Real-time collaborative network monitoring and control
using 3D game engines for representation and interaction', in Proceedings of the 3rd international workshop on Visualization for computer security, ACM New York, NY, USA, Alexandria, Virginia, USA, pp. 31-40.
Höllerer, T.H. & Feiner, S.K. (2004) 'Mobile Augmented Reality' in Telegeoinformatics:
Location-Based Computing and Services, ed H Karimi & A Hammad, Taylor &Francis Books Ltd.
Jacquet, C., Bourda, Y. & Bellik, Y. (2007) 'A Component-Based Platform for Accessing
Context in Ubiquitous Computing Applications', Journal of Ubiquitous Computing and Intelligence, vol. 1, no. 2, pp. 163-173. D.O.I.: 10.1166/juci.2007.205
Kent, K. & Souppaya, M. (2006) 'Guide to Computer Security Log Management: Recom-
mendations of the National Institute of Standards and Technology', National Institute of Standards and Technology [Online]. Available from: http://cs-www.ncsl.nist.gov/publications/nistpubs/800-92/SP800-92.pdf (Accessed 12th July 2010).
Mackay, W.E. (1998) 'Augmented reality: linking real and virtual worlds: a new paradigm
for interacting with computers', in Proceedings of the working conference on Advanced visual interfaces, ACM New York, NY, USA, L'Aquila, Italy, pp. 13-21.
Maltz, D. (n.d.) 'Unraveling the Complexity of Network Management', USENIX [Online].
Available from: https://www.usenix.org/events/nsdi09/tech/full_papers/benson/benson_html/ (Ac-cessed 17th March 2010).
Mantoro, T. & Johnson, C. (2003) 'User Mobility Model in an Active Office' in Lecture
Notes in Computer Science, Springer Berlin, Heidelberg, pp. 42-55. Manwani, S. (2003) 'ARP Cache Poisoning Detection and Prevention', Master of Com-
puter Science, San Jose State University, United States of America. Milgram, P., Takemura, H., Utsumi, A. & Kishino, F. (1994) 'Augmented Reality: A Class
of Displays on the Reality-Virtuality Continuum', SPIE, vol. 2351, pp. 282-292. D.O.I.: 10.1.1.83.6861
Milgram, P. & Kishino, F. (1994) ‘A Taxonomy of Mixed Reality Visual Displays’, IEICE
Transactions on Information Systems, vol. E77-D, no. 12, pp. 1 – 15. D.O.I.: 10.1.1.102.4646
Neumann, U. & You, S. (1999) ‘Natural feature tracking for augmented reality’, IEEE
Transactions on Multimedia, vol.1, no.1, pp.53-64, Mar 1999. D.O.I.: 10.1109/6046.748171
Nicolett, M. & Kavanagh, K.M. (2009) 'Magic Quadrant for Security Information and Event
Management', Gartner, pp. 1-22. Pras, A., Schonwalder, J., Burgess, M., Festor, O., Perez, G.M., Stadler, R. & Stiller, B.
(2007) 'Key research challenges in network management', Communications Maga-zine, IEEE, vol. 45, no. 10, pp. 104-110. D.O.I.: 10.1109/MCOM.2007.4342832
Rekimoto, J. & Nagao, K. (1995) 'The world through the computer: computer augmented
interaction with real world environments', in Proceedings of the 8th annual ACM sym-
70
posium on User interface and software technology, ACM New York, NY, USA, Pitts-burgh, Pennsylvania, United States, pp. 29-36.
Srinivasan, S., Fang, Z., Iyer, R., Zhang, S., Epsig, M., Newell, D., Cermak, D., Wu, Y.,
Kozintsev, I. & Haussecker, H. (2009) ‘Performance Characterization and Optimization of Mobile Augmented Reality on Handheld Platforms’, IISWC '09: Proceedings of the 2009 IEEE International Symposium on Workload Characterization (IISWC), pp. 128-137. D.O.I.: http://dx.doi.org/10.1109/IISWC.2009.5306788
Sterritt, R. (2002) 'Towards Autonomic Computing: Effective Event Management', in Pro-
ceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27'02), IEEE Computer Society Washington, DC, USA.
Wagner, D. (2007) 'Handheld Augmented Reality', Graz University of Technology, Aus-
tria. Wang, Y., Langlotz, T., Billinghurst, M. & Bell, T. (n.d.) 'An Authoring Tool for Mobile
Phone AR Environments', Human Interface Technology Laboratory New Zealand [Online]. Available from: http://www.hitlabnz.org/publications/2009-Mobile_phone_AR_environments_final.pdf (Accessed 21st March 2010).
Warrier, U., Besaw, L., LaBarre, L. & Handspicker, B. (1990) 'The Common Management
Information Services and Protocols for the Internet (CMOT and CMIP)', Internet Engi-neering Task Force [Online]. Available from: http://datatracker.ietf.org/doc/rfc1189/ (Accessed 26th March 2010).
Weiser, M. (1993) ‘Ubiquitous Computing’, Computer, vol. 26, no. 10, pp. 71-72, Oct.
1993, D.O.I.:10.1109/2.237456 web|3D (n.d.) 'X3D International Specifications', web|3D Consortium [Online]. Available
from: http://www.web3d.org/x3d/specifications/x3d/ (Accessed 19th March 2010). Yan, H., Maltz, D.A., Ng, T.S.E., Gogineni, H., Zhang, H. & Cai, Z. (2007) 'Tesseract: A
4D Network Control Plane', in Proceedings of USENIX Symposium on Networked Sys-tems Design and Implementation, Carnegie Mellon: School of Computing Science.
71
APPENDICES
Appendix A. PRELIMINARY SURVEY
A.1 Briefing
Your participation in this preliminary survey is entirely voluntarily, and you are free to
withdraw at any time. By completing this survey you are giving consent for the responses
submitted to be used in this research, and only for assisting in the design of a framework
and associated prototype.
Please also be aware that your data will be handled in a secure manner, and no personal
identifiable or confidential information will be included in any of the research. Your E-Mail
address will not be published, and is optionally supplied only if you would like to be noti-
fied when the final dissertation report has been published. Or in the event that an open
evaluation of the prototype is deemed appropriate and you would like to receive notifica-
tion.
This is a brief preliminary survey designed to assist in gathering information detailing the
usage of Network Management and Security Information and Event Management Sys-
tems in relation to physical access to Network Access Devices.
This survey forms a part of my research dissertation, which itself is a part of my study
towards a Master of Science Degree (M.Sc.) in Computer Security. The estimated time to
complete this survey is two to five minutes. Your participation is much appreciated.
If possible, please do encourage your professional Information and Communication
Technology contacts to also participate by using the following link:
http://sgiz.mobi/s3/ba70c61ac949
A.2 Questions
This will be used as a courtesy which may encourage complete responses.
This information can be used to affirm the selection of communication
change to be used by the AR Middleware component.
Figure 40: Question 1
courtesy which may encourage complete responses.
Figure 41: Question 2
This information can be used to affirm the selection of communication and data inte
to be used by the AR Middleware component.
and data inter-
This information can be used to affirm the selection of communication
change to be used by the AR Middleware component.
Responses will assist in understanding the commitment of time spent tendi
erational Support Systems.
Responses will assist in understanding the commitment of time spent tending to physical
NADs.
Figure 42: Question 3
This information can be used to affirm the selection of communication and data inte
to be used by the AR Middleware component.
Figure 43: Question 4
ssist in understanding the commitment of time spent tending to the O
Figure 44: Question 5
understanding the commitment of time spent tending to physical
and data inter-
ng to the Op-
understanding the commitment of time spent tending to physical
This information will be used to categorise the activities which require physical interve
tion in Data Centres or Process networks. This will assist in understanding the scenarios
which may benefit from an AR interface, and drive decision on possible primitives.
A.3 De-Briefing
Thank you for taking this survey. Your response is very important and will provide further
insight for this piece of research. Please do encourage your professional Information and
Communication Technology contacts to also participate by using the followin
http://sgiz.mobi/s3/ba70c61ac949
A.4 Results
Please note E-Mail addresses are not included for privacy reasons.
Preliminary Survey Results.xlsx
Figure 45: Question 6
used to categorise the activities which require physical interve
tion in Data Centres or Process networks. This will assist in understanding the scenarios
which may benefit from an AR interface, and drive decision on possible primitives.
Thank you for taking this survey. Your response is very important and will provide further
insight for this piece of research. Please do encourage your professional Information and
Communication Technology contacts to also participate by using the followin
http://sgiz.mobi/s3/ba70c61ac949
Mail addresses are not included for privacy reasons.
used to categorise the activities which require physical interven-
tion in Data Centres or Process networks. This will assist in understanding the scenarios
Thank you for taking this survey. Your response is very important and will provide further
insight for this piece of research. Please do encourage your professional Information and
Communication Technology contacts to also participate by using the following link:
Appendix B. SET-
B.1 Installation of the AR Middleware
The AR Middleware component was installed to a virtual machine executing under
VMWare Workstation 6.5.4 on GNU/Linux
was the Turnkey Linux LAMP Stack Appliance
http://www.turnkeylinux.org/lamp
Figure 46 illustrates the configuration console of the Turnkey Linux installation. A static IP
address was configured through this console to prevent the used of Dynamic Host Co
figuration Protocol (DHCP).
Figure 46: Turnkey Linux Configuration Console
The Turnkey Linux LAMP Stack Appliance
Apache Web Server, and PHP
PHP’s SNMP libraries, which are not installed by default.
UP OF THE EVALUATION ENVIRONMENT
Installation of the AR Middleware
The AR Middleware component was installed to a virtual machine executing under
VMWare Workstation 6.5.4 on GNU/Linux. The GNU/Linux distribution used for the
urnkey Linux LAMP Stack Appliance, which is available from
http://www.turnkeylinux.org/lamp
illustrates the configuration console of the Turnkey Linux installation. A static IP
ured through this console to prevent the used of Dynamic Host Co
: Turnkey Linux Configuration Console
LAMP Stack Appliance contains a pre-configured installation of
Apache Web Server, and PHP: Hypertext Preprocessor. The AR Middleware utilise
PHP’s SNMP libraries, which are not installed by default. In order to install this required
The AR Middleware component was installed to a virtual machine executing under
. The GNU/Linux distribution used for the OS
, which is available from:
illustrates the configuration console of the Turnkey Linux installation. A static IP
ured through this console to prevent the used of Dynamic Host Con-
configured installation of the
: Hypertext Preprocessor. The AR Middleware utilises
In order to install this required
76
library, a full package update was performed and then the php5-snmp package was in-
stalled, using the following commands:
apt-get update
apt-get install php5-snmp
The Apache Web Server daemon was then restarted using the following command:
/etc/init.d/apache2 restart
Finally the AR Middleware was installed to the Apache Web Server’s Document Root di-
rectory. The resulting file structure was:
/var/www/
ARViewer.swf
ar_middleware.php
includes/
dot1dTpFdbTable-class.php
ipNetToMediaTable-class.php
snmp-include.php
mibs/
BRIDGE-MIB
RFC1213-MIB
templates/
ws-c2924c-xl-class.php
resources/
assets/
vi-material-black-50.png
vi-material-green-50.png
vi-material-red-50.png
77
vi-material-yellow-50.png
flar/
ARViewer_flarConfig.xml
FLARCameraParams.dat
patterns/
pat8/
patt001.pat
B.2 Installation of CactiEZ
A separate virtual machine with a CactiEZ installation was used to provide access to a
freely available Network Management interface, which is accessible via a web browser.
CactiEZ v0.6 was used, and is available from http://cactiez.cactiusers.org/
However, the MAC Track plug-in for CactiEZ which was used to detect the ARP cache
poisoning attack simulations does not function in v0.6 without some adjustments. The
following commands were executed on the CactiEZ virtual machine in order to obtain a
functional plug-in.
First, the database tables relating to the MAC Track plug-in required upgrading. This was
performed by executing the command:
php /var/www/html/plugins/mactrack/database_upgrade.php
Then a new version of the MAC Track plug-in from the project’s Subversion repository
was required. This was obtained and installed using the following commands:
yum install svn subversion
cd ~/
svn co svn://svn.cacti.net/cacti_plugins/mactrack
rm –rf /var/www/html/plugins/mactrack
78
mv mactrack/2.8 /var/www/html/plugins/mactrack
chown –R apache.apache /var/www/html/plugins/mactrack
reboot
Finally in order to facilitate fast data polling required for the attack simulation, the polling
process was executed in a continual loop via the console using the following commands:
for (( ; ; )); do php –q /var/www/html/plugins/mactrack/poller_mactrack.php
–f –d; done
B.3 Client Configuration
The attack simulation clients were booted using a ‘live’ BackTrack 4 DVD. This penetra-
tion testing centric distribution of GNU/Linux includes the Etterpcap tool, which was used
to create Address Resolution Protocol (ARP) cache poisoning attacks. Once each of the
four attack simulation clients were fully booted some additional configuration was re-
quired. First, the windows manager and desktop manager were executed using the com-
mand:
startx
Then the network interface card modules were configured and associated networking
processes were started by using the “Start NETWORK” option, as shown in Figure 47.
Figure 47: BackTrack 4’s Start NETWORK option
In order to remote administer the attack simulation clients from the central server for the
purpose of initiating the attacks. The client required that the Secure Shell Daemon
(SSHD) be configured and started.
tion, which is depicted in Figure
: BackTrack 4’s Start NETWORK option
In order to remote administer the attack simulation clients from the central server for the
purpose of initiating the attacks. The client required that the Secure Shell Daemon
be configured and started. This was performed by using the “Setup SSHD
Figure 48.
In order to remote administer the attack simulation clients from the central server for the
purpose of initiating the attacks. The client required that the Secure Shell Daemon
“Setup SSHD” op-
Figure 48
Finally, in order for the Secure Shell Daemon to authenticate root logins via the network,
root’s authentication tokens must be updated. This can be performed by resetting root’s
password using the passwd command, which is demonstrated in
48: BackTrack 4’s Setup SSHD option
Finally, in order for the Secure Shell Daemon to authenticate root logins via the network,
root’s authentication tokens must be updated. This can be performed by resetting root’s
command, which is demonstrated in Figure 49.
Finally, in order for the Secure Shell Daemon to authenticate root logins via the network,
root’s authentication tokens must be updated. This can be performed by resetting root’s
Figure 49: Resetting root’s password using passwd
The AR Viewer client is a mobile Adobe Flash applet, which will execute on any
supports Adobe Flash. For the evaluation network, the AR Viewer was executing on a
standard Microsoft Windows XP SP3 laptop with the Adobe Flash Player installed, and
executing via Google Chrome.
: Resetting root’s password using passwd
client is a mobile Adobe Flash applet, which will execute on any OS
supports Adobe Flash. For the evaluation network, the AR Viewer was executing on a
standard Microsoft Windows XP SP3 laptop with the Adobe Flash Player installed, and
OS which
supports Adobe Flash. For the evaluation network, the AR Viewer was executing on a
standard Microsoft Windows XP SP3 laptop with the Adobe Flash Player installed, and
82
B.4 Preparing the Environment
The final preparation required was to create a Fiducial marker for the Cisco Ethernet
switch. The stock AR Tag marker pattern depicted in Figure 50 was printed on to hard
card. The Fiducial marker measured 40 millimetres by 40 millimetres, and was then af-
fixed to the Cisco Ethernet switch.
Figure 50: Fiducial marker for Cisco Ethernet switch
Appendix C. F
C.1 Briefing
Your participation in this evaluation survey is entirely voluntary, and you are free to wit
draw at any time. By completing this survey you are giving consent for the responses
submitted to be used in this research.
Please also be aware that your data will be handled in a secure manner, and no personal
identifiable or confidential information will be included in any of the research. Your E
address will not be published, and is optio
fied when the final dissertation report has been published.
This is an evaluation survey designed to assist in measuring the effectiveness of a pr
posed framework for an Augmented Reality (AR) interface for n
security.
This evaluation forms a part of my research dissertation, which itself is a part of my study
towards a Master of Science Degree (M.Sc.) in Computer Security. The estimated time to
complete this evaluation is ten to fifteen
C.2 Questions
This question is used to collect E
tion on the completion of the dissertation project.
FRAMEWORK EVALUATION SURVEY
Your participation in this evaluation survey is entirely voluntary, and you are free to wit
draw at any time. By completing this survey you are giving consent for the responses
d to be used in this research.
Please also be aware that your data will be handled in a secure manner, and no personal
identifiable or confidential information will be included in any of the research. Your E
address will not be published, and is optionally supplied only if you would like to be not
fied when the final dissertation report has been published.
This is an evaluation survey designed to assist in measuring the effectiveness of a pr
posed framework for an Augmented Reality (AR) interface for network management and
forms a part of my research dissertation, which itself is a part of my study
towards a Master of Science Degree (M.Sc.) in Computer Security. The estimated time to
ten to fifteen minutes. Your participation is much appreciated.
Figure 51: Question 1
used to collect E-Mail addresses of those that wished to receive notific
tion on the completion of the dissertation project.
Your participation in this evaluation survey is entirely voluntary, and you are free to with-
draw at any time. By completing this survey you are giving consent for the responses
Please also be aware that your data will be handled in a secure manner, and no personal
identifiable or confidential information will be included in any of the research. Your E-Mail
nally supplied only if you would like to be noti-
This is an evaluation survey designed to assist in measuring the effectiveness of a pro-
etwork management and
forms a part of my research dissertation, which itself is a part of my study
towards a Master of Science Degree (M.Sc.) in Computer Security. The estimated time to
inutes. Your participation is much appreciated.
Mail addresses of those that wished to receive notifica-
This information will assist in determining if respondents have expectations of the fram
work or a set workflow.
This information will assist in determining if
work or a set workflow.
Figure 52: Question 2
assist in determining if respondents have expectations of the fram
Figure 53: Question 3
assist in determining if respondents have expectations of the fram
assist in determining if respondents have expectations of the frame-
respondents have expectations of the frame-
This information will assist in understanding the scenarios which
termine are important, which may affect how they
tal prototype – and by relation –
C.3 Functional Testing
Functional testing consists of a simulated attack being introduced into the evaluation ne
work from a client selected at random. Subjects
network management data to assist them in identify the occurrence of the attack, and to
determine the source of the attack. Question 5
Network Management tool, and questions 6 thru 8
totype.
Figure 54: Question 4
will assist in understanding the scenarios which individual subjects d
termine are important, which may affect how they perceive the single purpose experime
– the framework.
of a simulated attack being introduced into the evaluation ne
work from a client selected at random. Subjects are then provided with an interface to
network management data to assist them in identify the occurrence of the attack, and to
determine the source of the attack. Question 5 is answered whilst using the traditional
Network Management tool, and questions 6 thru 8 are answered whilst using the AR
individual subjects de-
perceive the single purpose experimen-
of a simulated attack being introduced into the evaluation net-
an interface to
network management data to assist them in identify the occurrence of the attack, and to
answered whilst using the traditional
AR pro-
Figure
Timing between initiating the attack and the subject detecting the attack are recorded.
Also, the subject is queried upon identifying the source of the attack.
In total, four interactive tests are
timeliness in the subjects’ responses, and to analyze difference in timings between the
two interface paradigms.
C.4 Response to the Presented Framework
Subjects are surveyed upon their experience in using the prototype of the framework in
order to garner their opinions on the affect the framework had upon diagnosing and ident
fying the attack and source of the attack. Each category of questioning
questions; a Likert scale question, and then
jects can provide additional insight.
Figure 55: Questions 5 thru 8
Timing between initiating the attack and the subject detecting the attack are recorded.
Also, the subject is queried upon identifying the source of the attack.
are used to determine an average degree of accuracy and
ness in the subjects’ responses, and to analyze difference in timings between the
Response to the Presented Framework
surveyed upon their experience in using the prototype of the framework in
order to garner their opinions on the affect the framework had upon diagnosing and ident
fying the attack and source of the attack. Each category of questioning is posed in pairs of
questions; a Likert scale question, and then a free-form comment question where su
provide additional insight.
Timing between initiating the attack and the subject detecting the attack are recorded.
used to determine an average degree of accuracy and
ness in the subjects’ responses, and to analyze difference in timings between the
surveyed upon their experience in using the prototype of the framework in
order to garner their opinions on the affect the framework had upon diagnosing and identi-
posed in pairs of
form comment question where sub-
Figure 56: Question 9
Figure 57: Question 11
Figure 58: Question 13
Figure 59: Question 15
Figure 60: Question 17
C.5 Improvement Feedback
The question captures un-categorised free
tential improvement to the framework.
C.6 De-Briefing
Thank you for taking this survey. Your response is very important and will provide further
insight for this piece of research.
C.7 Results
Please note E-Mail addresses are not included for
Framework Evaluation Results.xlsx
Improvement Feedback
Figure 61: Question 19
categorised free-form feedback from the subject regarding p
tential improvement to the framework.
Thank you for taking this survey. Your response is very important and will provide further
ght for this piece of research.
Mail addresses are not included for privacy reasons.
regarding po-
Thank you for taking this survey. Your response is very important and will provide further
Recommended