C

Consiglio Nazionale delle Ricerche

Current Research Issues on Cyber security in Robotics

G. Lacava, A. Marotta, F. Martinelli, A. Saracino, A. La Marra, E. Gil-Uriarte, V. Mayoral Vilches,

IIT TR-05/2020

Technical Report

Marzo 2020

Iit

Istituto di Informatica e Telematica

Current Research Issues on Cyber security in Robotics

Giovanni Lacava, CNR-IIT

Angelica Marotta, CNR-IITFabio Martinelli, CNR-IITAndrea Saracino, CNR-IIT

Antonio La Marra, Security ForgeEndika Gil-Uriarte, Alias Robotics

Víctor Mayoral Vilches, Alias Robotics

March 25, 2020

Abstract

Cyber Security in Robotics is a rapidly developing area which draws attention frompractitioners and researchers. In this paper we provided an overview of the key issuesarising in the cyber security robotic landscape and the threats affecting this sector. Wealso analyzed the scientific approaches to managing cyber attacks in robotics. Finally, weproposed directions for further advances in this area.

Robotics Security, Cyber-Attacks, Intrusion Detection, Trusted Robot.

1 IntroductionRobots have been introduced massively in the manufacturing industry as well as in everyday lifehas been observed [17]. Due to their potentiality to cover several applications in our lives, thediffusion and development of new robotic systems is expected to increase day by day. In thelast decade, the field of robotics has been pervaded by the emerging technologies like MachineLearning and AI (Artificial Intelligence), IIoT (Industrial Internet of Things), human-machinecollaboration or autonomous mobile systems. Therefore robots have become "intelligent" andrepresented a important resource for digitization in the manufacturing industry1.

As analyzed by IFR (International Federation fo Robotics) the market value for professionalservice robots increased by 32% to US$ 9.2 billion in 2018 (over 2017), driven by a 60% increasein unit sales of logistics systems. Sales of robot vacuum cleaner are also dominating the risein the number of personal/domestic service robots. The majority of these robots are used innon-manufacturing environments, such as warehouses and hospitals, but some are also used infactories or transportation sectors(professional robotics). The rapid increase in sales of logisticssystems is partially due to an expansion in the field of e-commerce; technology advances have,thus, expanded the range of tasks logistics robots, opening avenues to different areas. For ex-ample, logistics robots equipped with sensors can be programmed with the help of data from

1https://ifr.org/img/office/Sales_Flyer_World_Robotics_2019_web.pdf

1

sensors; they can create a map of their environment and elaborate obstacle avoidance strategiesthrough sophisticated algorithms 2.

For instance, sensors and vision-technologies based on machine learning allow robots to iden-tify and select objects. These systems are also applied in fixed industrial robots; in addition torobotic arms, grippers and end-of-arm tools have advanced so significantly, that logistics robotscan now successfully be used to an increasing variety of products, including fragile materials.These technologies also enable robots to enter and move around in narrow spaces. For exam-ple, the adoption of robotic systems in ship inspections improved the accuracy and quality ofmaritime operations.

Through the same market analysis, IFR noticed that medical robots were reported to ranksecond after logistics robots, accounting for 30% of the value of total new sales on professionalrobots in 2018. Another fast growing category of robots is "personal/domestic robots", whichcovers robots used in a home environment, entertainment, and assistance. Additionaly, floor-and window-cleaning robots and robotic lawnmowers, together with robotic toys and games, aredominating sales 3.

However, this process of diffusion needs to meet some critical requirements, such as cost ofproduction, ever-changing market demand, and user safety [3]. In this context manufacturersoften overlook cyber security aspects during the design and production phases. Robotic appli-cations, such as autonomous cars, drones, entertainment robots, medical robots, are among themost exposed to cyber security vulnerabilities [6]. Therefore, it is necessary to have a goodunderstanding of the robotics system to assess security risks and threats. The most criticalchallenges are those relating to the rapidly changing consumer trends, shortage of resources andskilled workers, aging society, demand for local productions and cyber security risks looming overthe dawn of a yet immature industry.

Related works - Although the integration of information technologies (IT) represents animportant step towards obtaining more smart and flexible robotic systems, it introduces somecritical aspects, especially in the context of cyber security. As illustrated by the National Instituteof Science and Technology (NIST) [16], compromised robots can have a digital and physicalimpact on the environment in which they operate. Therefore, it is critical to manage securityand safety in robotic systems. Usually, to asses the strength of a robotic system in terms ofcyber security, it is possible to adopt the following procedures:

• Threat modeling (Identifying attack vectors);

• Vulnerability assessment (Penetration testing);

• Assignment of level risk of the vulnerabilities;

• Identification of cyber-attacks and the vulnerabilities;

• Prioritization and implementation of related countermeasures.

As for the first step, as suggested by Vilches et al. in [1], we can use a specific set of guide-lines to identify vulnerabilities in a system. The author proposed a framework named RoboticSecurity Framework to assess robotics systems, which is helpful to classify attacks vectors in thefollowing categories: Hardware, Network, Firmware/OS (Operating System) and Applicationlevel. Similarly, other cover this topic [3],[10]. Khalil et al. in [6] introduced the Robot AttackTool (RAT). Using this tool, it is possible to implement risk assessment in a robotic platforms.The author use two mobile robots, respectively Mobile Eyes and arnlServer and through the

2https://ifr.org/post/market-for-professional-and-domestic-service-robots-booms-in-201832

2

RAT identified the risks according to CIA -Triad (Confidentiality, Integrity, and Availability).In [3] for example,the authors discuss the impacts of cyber-attacks in different fields of appli-cation of robotic systems, suggesting countermeasures. They classified cyber-attacks in threelevels: Hardware, Fimware/O.S. and Application. However, although they explored the impactof cyber-attacks on robotics, they didn’t analyze the technologies involved in great details.

While new technologies provide the potential for maximizing the capabilities of robots, theyalso increase the need to pay closer attention to the "safety vs security" issue, in particularKirschgens et al. [10] discussed how the lack of security might cause safety repercussions; theyidentified three principal areas of conflict: a) Human loss and injuries b) Data theft and privacyissues and c) Reputation issues. These aspects will be discussed in chapters 3 and 4.

Contribution - As mentioned by Kirschgens in [10], "Robots traditionally employed inindustry are being replaced by collaborative robots. Moreover, robotics is becoming increasinglyintertwined with facets of IT such as the cloud, mobile devices and the Internet of Things (IoT).And, unlike traditional robots, the coming generation of these machines is being envisioned anddesigned to gain more autonomy."

The authors observations highlight the importance of having a substantial understandingof the robotic system parts. To this extent it is crucial assess the related communication vul-nerabilities and the applications used to interact with the robotic systems. When referring toCyber-Physical Systems (CPS), can effectively integrate cyber and physical components usingmodern sensors and computing and network technologies. However the rapid growth of CPSapplications is also leading to several problems relating to security and confidentiality [32]. Cy-ber security is therefore, central to safeguarding the confidentiality, integrity, and availability ofrobotic systems and their related components [105]. Although these considerations open a vastfield of reflection, currently, there is little discussion in the literature on cyber security robotics[17].

Structure of Survey - The purpose of this survey is to highlight future challenges in manag-ing cyber security in robotics and provide an overview of the critical cyber security countermea-sures in robotics. This paper is organized as follows Section 2 discussed the robotic technologiesadopted in this sector. Section 3 shows the current regulatory environment from a safety, securityand privacy perspective. Section 4 we analyzes current threats and attacks in robotic systems.whereas Section 5 defines the current research issues on the topic. Finally, in Section 6, wesummarize the main research findings and recommend future research directions.

2 Robotic System and related technologiesRobotics is traditionally considered as "the art of system integration" of robots. ITS modularnature provides a wide range of usage options. The majority of robots are equipped with the"ability" to sense, process, and act with the world around them. The field of robotics benefit fromcontinued advancements in a variety of disciplines, such as mechanical engineering, computerscience, material science, sensor fabrication, manufacturing techniques, etc. [54]. Robots aredesigned for specific tasks, such as assembling or repairing, which may not be readily adaptablefor other applications. Over the last two decades, several authors have attempted to tackle thisproblem and explain the unusual characteristics of robotic systems. In the literature, robots canbe classified into the following types:

• Humanoid and social robots;

• Unmanned Ground Vehicles (UGVs) and other ground robots;

• Articulated arm robots;

3

• Unmanned Underwater Vehicles (UUVs);

• Unmanned Aerial Vehicles (UAVs).

However, this classification, although relevant for discussion, is very generic and has severallimitations. For example, classifying method robots doesn’t have to be limited to their usage.Instead it is also necessary to identify the general characteristics of robotic systems, their func-tionalities and their components. As shown in Tab n.1, it is worth noting that the majorityof authors discussed autonomous systems. A type of robot that are not controlled by human([3],[66]), others scholars examined robots used in tele-operation mode ([5], [11]). In this re-search, we will extend this classification scheme by focusing on the cyber aspects of robotics.In particular we observed that the following cyber security features are the most common inrobotics:

• Network;

• O.S.;

• Middleware.

Fig.3 shows the types of robots analyzed in the literature.

2.1 Robotic systemAlthough our research doesn’t focus on robotic systems, it is important to clarify the differencebetween robotic systems and Cyber-Physical Systems (CPS). According to Sabaliauskaite etal.[28]"Cyber-Physical System (CPS) is a system that can effectively integrate Cyber and Physi-cal components using the modern sensor, computing, and network technologies". ISO 8373 definea robot as electro-mechanical system composed of a multi-axis manipulator, a control system,an “operator interface,” and its hardware and software communication interface. Other authorsdefine robots as: "It is a complex system integration composed of heterogeneous hardware andsoftware" [7], a mechatronic device which also includes resourcefulness or autonomy 4. One ofthe main differences between a CPS and a robot is that they are equipped with different motionrange, position, and controller tools. A CPS can’t be necessarily designed or structured to movein 2D/3D space. However, it is, necessary to observe that robotics represents a sub-set of CPS.Robots are generally designed to perform the following functions [106]:

• Sensing This function helps robots perceive their environment and share information withthe other modules or systems or their users.

• Actuation This function enables robots to interact physically with the environment.

• Cognition This function(computation and coordination) allows robots to to anticipate theeffects of their actions as well as the activities of the human users around them.

• Energy The purpose of this function is to provide power to their system or subsystems.

• Communication This function allow robot to connect with other modules or interfacesthrough (external) communication channels.

4https://www.galileo.org/robotics/intro.html

4

• User Interface (UI) with This function enables robots and their components to be in-teroperable and visible during human-robot interactions. Examples include tools, such asjoysticks, tactile screens and voice input.

Through the integration and interaction of the functions mentioned above robots can executeseveral tasks , although they have to meet the following requirements [7]:

• Accuracy: Robots need to send actuators commands to perform precise operations withinacceptable error margins;

• Safety: Robots need to make information available to operators. This requirement enablehuman operator to take safe and informed decisions and perform emergency procedures insafely;

• Integrity: Robotic controllers need to minimize the impact of potential incidents involvingphysical parts (e.g. avoiding collisions).

Any violation of these requirements would expose robot to cyber security threats, which couldpotentially compromise the safety and security of the operator and the environment. Vilches etal. in [1] use four levels of analysis to assess the security level of a robotic system a) Physical, b)Network, c) FW/Operating System and d) Application. "For each of these criteria the authorsidentified the following factors: what needs to be assessed (Objective), why it is necessary toperform an assessment (Rationale) and how to systematize an evaluation (Method)." As shownin Fig.n.1 the Robot Security Framework provides a methodology that focuses on four layers,which, in turn, cover several security elements 5.

Each aspect is analyzed according to three points:" 1)Objective or description of the evalua-tion, 2) Rationale or importance of such aspect and 3) Method or systematic action plan."

Figure 1: Robot Security Framework

With the rapidly increasing power of technology, robots have significantly increasing their levelof functionality. In the next section , we will outline the critical steps towards the integrationbetween robots and IT technologies.

5https://github.com/aliasrobotics/RSF

5

2.2 Holistic model in Industry 4.0Today, modern industrial robots are integrated into “smart” industrial devices, which extend andflexibilize their capabilities[5]. In the current industrial context, interconnected robot enableremote operators to control them more efficiently and simplify their maintenance (as mentionedin ISO 10218-2:2011). This advantage is achieved because robots are equipped with Ethernetports to connect via local area networks (LANs). They can be equipped with industrial routersor gateways, or it is possible to connect robot via a VPN (Virtual Private Network) or a cellularnetwork.

An increasing number of companies are implementing the Industry 4.0 paradigm, by providingfactories with plant devices and Internet services. In this connected industrial contexts, managingcyber security issues is one of the most important challenges. Among the benefits derived fromthis interconnectedness reduction of total machine downtime, adoption of predictive maintenance,and remote monitoring of the plant. These aspects also provide the additional capacity to identifythe "muda", a traditional Japanese word for an activity that is useless, and that, in this case,doesn’t add value to the production supply chain.

In this new environment, is necessary to analyze a great amount of data produced by plants(e.g., sensors, actuators and machine connected to computing systems, etc.). For this reasoncritical industrial tools are more exposed to cyber-attacks, which can affect business models[21].Automated manufacturing systems use CPSs to be more flexible and perform smart production.The integration between physical (hardware, sensors network, HMI - Human Machine Interface)and computational systems(cloud computing, computer and so on), makes the monitoring of theplant easier during the state variation of the process parameters. In analyzing smart systems,we used the Computer-Integrated Manufacturing (CIM) model as a reference tool as, it showsthe hierarchical architecture of computer systems and communication connections utilized inmanufacturing automation systems [41].

The CIM is highly integrated, into the manufacturing industry. It is composed of five layers:

• Layer 1: Sensor-Actuator;

• Layer 2: Cell control;

• Layer 3: Supervisory;

• Layer 4: Plant management;

• Layer 5: Enterprise.

As discussed by Tuptuk and Hailes in [41], higher levels represent general purpose networkprotocols. Conversely, lower levels indicate special protocols dedicated to specific applications.Three components distinguish CIM from other manufacturing methodologies:

• Means for data storage, retrieval, manipulation and presentation;

• Mechanisms for sensing state and modifying processes;

• Algorithms for uniting the data processing component with the sensor/modification com-ponent.

However this model is vulnerable to cyber security attacks; for example, the communicationprotocols used to support this infrastructure such as Modbus, PROFIBUS, Industrial Ethernetare not designed with security in mind. For example, they don’t provide authentication orintegrity measures to detect abnormal behaviors. In case of cyber-attack, robots operating at

6

level 2 are exposed to vulnerabilities involving, robot integrity and user safety [6]. Lezzi et al. in[21] stated that "regarding the security risks originated by the cyber threats, it is possible to statethat they depend on the loss of confidentiality (data disclosure risk), integrity (risk of corruptionor modification of records and data loss) and availability of information or information systems(denial of service risk)".

A potential cyber-attack can damage the production itself or cause injuries to workers. Animportant category of robotics is Collaborative Robotics Systems - CRS, which includes systemsthat can interact with human operators. In particular in [20] Khalid et al. analyze the col-laborative and mentioned that these systems are composed of HMI components,which fostersinteractions between the robotic system and the operator (Human Robot Collaboration- HRC)[5].

In this context, the human operator is seen as a vital component of the system. In particularCRSs include the following modules: Human Components (HC), Physical Component (PC) andComputational Component (CC). In this area of research, safety and security are interconnected;for example, potential failures involving CRS sensing components can compromise the safety ofthe operator.

In this regard, it is important consider to cyber security aspects of robots during design phaseof system. Khalid proposed a security framework for CRS that can only be applied if those whouse it are aware of cyber-attacks. In his work, Khalid discussed to machine to machine (M2M)and human to machine (H2M) communication integration.

• As for M2M, the author highlighted that the information produced by the machine (sensoror a physical module) are sent through the network using a gateway and then, are analyzedand processed;

• In H2M communications, it is necessary to establish a real time communication to guaranteesecure communication.

In general, these aspects are extended to all robotics systems, including smart environmentsuch as home-automation, surgery room and military context. In military applications, forexample an unauthorized entity can control and compromise confidential data through the useof drones [80]. In a chapter 4, we will discuss the threats and cyber-attacks concerning roboticsystems.

2.3 IT technology used in RoboticsThe connection between IT systems and operational technology (OT) is useful to guaranteethe safety of plants as well as the integrity of the manufactured product even when faults,human errors, or other abnormal conditions occur. It’s necessary to get cyber security androbotics experts to identify solutions to managing security aspects in robotic systems. Nowadays,manufacturers don’t pay enough attention to security, but the fast-growing robotics market tofocus on risks and threats in this field [10].

Additionally, manufacturers use commercial off-the-shelf (COTS) products, which, on the onehand reduce the costs, but, on the other hand, expose them to vulnerabilities.

Industrial Control Systems (ICS) use a wide variety of insecure communication protocols, suchas Modbus, PROFINET, DNP3, and EtherCAT, which do not have the security mechanisms tosupport authentication or packet integrity [41].

Communications are, therefore, a vital part of a robot’s ecosystem. Mobile application orInternet services/cloud resources use the Internet, Bluetooth or Wi-fi without properly securingcommunication channel. For example, robots need to connect to the Internet to send data tocloud vendors [4]. In the literature, cabled communications in different environments, such as:

7

• Industrial context: smart manufacturing cells;

• Medical context: Surgical and diagnostic applications;

• Other context: operation of inspection and limited operational actions.

Industrial: In industrial field are used FieldBus systems, for example, [41], may have severalproblems regarding the maintenance and integration of devices in complex environment. Theseaspects, as discussed above, exposes the manufacturing systems to security issues; Medical: Med-ical applications are widespread and generally use Machine to Machine (M2M) communicationthrough TCP/IP or UDP/IP link [57], [17]; Other: Teleoperation robotics is used in mobile,civil and military areas, which use of complex transmission procedures [29], [81]. IoT and Cloudtechnologies, as well as, autonomous systems, such as AUVs,UAVs or UGVs (shown in Figure 2),are also widely applied in robotics.

Another family of robots, called co-bots (collaborative robots), operate without protections(like for example AURA Robot6) [5].

Today the combination of IT systems and robots led to an evolution of these systems, whichare now able to share information, such as 3D models, videos and so on. [24]

Figure 2: Types of Autonomous Robots

Given the increasing vulnerabilities affecting communications in robotics, in this work, wire-less telecommunication technologies used for mobile devices and the types of mobile malwarethat generally target them.

Some authors [27] indicated that most of these technologies are used in Autonomous systems(e.g. Wi-fi IEEE 802.11, GSM and GPRS).

For example, through the use of GPRS systems, it is possible to use packets switching mech-anisms (i.e. IP protocol) to enable the exchange of data between users.

Tables n.1, n.2, n.3, n.4 and n.5 show that autonomous systems use the following components:

• Network: TCP/IP, UDP/IP, VPN, Wi-fi communication, GSM and GPRS;

• O.S.: Windows and Linux-based;

• Middleware: ROS (Robotic Operating System).

Additionally the tele-operated systems use the following components:

• Network: TCP/IP, UDP/IP;6https://www.comau.com/it/le-nostre-competenze/robotics/automation-products/

collaborativerobotsaura

8

Table 1: Robotic systemRobot ReferenceAmigot [8],[28],[30]AscTec quadrotor [29]RAVEN [53]RAVEN II [11], [17], [19],

[57], [63], [69]Jacobs [58]Turtle bot [64], [78]Chimera [65]JAV - Lego Mindstorms NXT [66]EZ - Robot JD Humanoid [68]iRobot Create [29]NAO - SoftBank Robotics [4], [68]NPS Arsenil [80]QT [2]Jibo [2]Beam [2], [17]Zenbo [2]Care - O -Bot [3], [15]Ravens, WASPs, Pumas, [3]T-Hawks, Predators,Gray Eagles, Reapers,Shadows, Global Hawks [3]Pepper [4]Alpha 1S, Alpha 2 [4]Robotis OP2, Thorman 3 [4]UR3, UR5, UR10 [4], [5], [24], [108]Sawyer [4]Fanuc CR-35iA [5], [7], [24]PeopleBot-TM [6], [9]ABB’s Yumi [5], [7], [24]WowWee Rovio [10]Robotic Enclave - Kuka youBots [16]Bump, Go robot [17]AntBo [17]Erector Spykee [17]Parrot AR Drone 2.0, [62]Bebop Drone, Phantom 2 Vision,D Robotics Solo4WD [71]ATV (Autonomous Terrestrial [72]Vehicles) swarmKUKA iiwa [5], [23]ABB IRB140 [24]Robot Patrolling swarm [77]Baxter [4], [78]Karen [79]RB-1 [81]

9

Figure 3: Robot types

Table 2: IT Technologies - ProtocolLevel ISO/OSI Protocol

TCP/IP UDP/IP USB CAN VPN TLS SSH Telnet FTP RJ45 HTTP DeviceNet ModBus DeviceNet FieldBus MQTT

Physical Layer [17] [68] [7] [11][24]

[4] [5] [6] [7][9] [16] [69]

Datalink Layer [16] [24]

Network Layer

Transport Layer [5] [15] [17][23] [24] [53][58] [71] [73][77] [78] [79][81]

[15] [17][19] [24][53] [57][63] [73][77]

[5] [7][62] [78]

Session Layer

Presentation Layer [53] [65] [62] [64][68]

Application Layer [64] [68] [24] [62][68]

[5] [77]

Table 3: IT Technologies - WirelessLevel ISO/OSI Wireless

Wi-fi RF GPRS Bluetooth GSM M2M SIM

Physical Layer [58] [66] [3] [5] [7][24] [72]

[4] [7] [24]

Datalink Layer [2] [3] [4] [6] [8][9] [10] [17] [28][29] [30] [62][64] [68] [71][72] [73] [77][80]

[3] [5] [7][24] [72]

[7] [24]

Network Layer [3] [5] [7][24] [72]

[7] [24] [7] [24]

Transport Layer [3] [5] [7][24] [72]

Session Layer [3] [5] [7][24] [72]

Presentation Layer [3] [5] [7][24] [72]

Application Layer [3] [5] [7][24] [72]

10

• O.S.: Linux-based;

• Middleware: ROS (Robotic Operating System).

Table 4: O.S. used in Robotics systemOS

WindowsWindows Windows 7 Windows CE Windows Vista Windows XP[28] [30] [68] [9] [57] [77] [24] [10] [5] [10]

LinuxLinux Trusty Tahr Ubuntu Ubuntu 16.04 Kali Linux Ubuntu 14.04 Raspian Linux Gentoo Linux Ubuntu 12.04[3] [5] [16][19] [62] [63][71] [73] [79]

[79] [77] [19] [64] [69] [80] [81] [66] [68] [9]

OtherOrdroid U-3 NuttX OS ARIA VxWorks Android OpenWRT LTS Operating System[80] [3] [6] [8] [5] [24] [62] [62] [81]

Table 5: Middle-ware used in Robotics systemMiddle-ware

ROS NAOqi YARP ORCA Scapy

[2] [4] [5] [6] [8] [11] [15][16] [17] [19] [23] [28][57] [63] [64] [65] [69][73] [77] [78] [79] [80][81]

[15] [17] [17] [68] [73] [57]

Finally tables n. 2 and n. 3,n.4 , n.5 summarize the studies correlating robotics with Network,O.S. and Middle-ware issues.

In particular, the Middle-ware a layer of software connecting client and back-end systems, isone of the most vulnerable to cyber attacks. The Robotic Operating System (ROS), is one ofthe most common [78].

Potential attacks to ROS applications may cause authenticity and confidentiality issues. Gen-erally, in a graph structure, ROS processes are represented as nodes connected by edges calledtopics. ROS nodes can send (publish) and receive (subscribe) messages to one another. This typeof communication protocol is also known as the publish/subscribe model. the publish/subscribemodel.However, although efficient, this model is also subject to several vulnerabilities:

• No authentication (Man in the Middle - MITM attacks);

• No encryption/confidentiality [80];

• Easy to footprint [109].

One possible solution to these issues is adopting the Open Platform Communication UnifiedArchitecture (OPCA) protocol, a security model that guarantees an appropriate level of securitythrough cryptographic methods. According to Breiling et al. [75] this security model can bemodelled as a centralized client-server communication. By doing so, it is possible to leverage someaspects of the protocol better to secure connections between nodes. However, although exposedto threats, open-sources systems, such as ROS and Linux-based operating systems, are the mostused in robotics. They provide flexible opportunities to standardize procedures and developefficient solutions. ROS, for example has also been extended to Android systems. Understandingthe strengths and weaknesses of these systems is crucial to make informed decisions regarding

11

cyber security and navigate a smart environment more efficiently. Nowadays, robots are becomingmore ubiquitous; for example they can be used to supervises "intelligent" sensors or report systemchanges to the user [33], [41].

Example of data processed by robots include the following: a) maps , b) temperature value,c) images of the surrounding environment, and so on.

In this context cloud technologies based robotic systems can contribute to extending the datastorage robots [44], [46]. Potential benefits derived from the use of cloud technologies includethe following:

• Big Data: access to remote libraries of images, maps, trajectories, and object data;

• Cloud Computing: access to parallel grid computing for statistical analysis, learning, andmotion planning;

• Collective Robot Learning: robots sharing trajectories, control policies, and outcomes;

• Human computation: crowd-sourcing access to analyze images and perform classification,learning,and error recovery.

For example in [46] the authors discuss the CloudThink architecture, an open-standard forself-reporting sensing devices (e.g. sensors on-board). This infrastructure provides collaborativedata sharing procedures for traffic routing and obstacle avoidance strategies. This framework isa good example of integration among different types of communication Figure n.4.

The architecture implemented in this application includes two components [52]: 1) cloudinfrastructure and 2) bottom storage facility.

Figure 4: CloudThink architecture

As disruptive technologies such as cloud, mobile, social, and cognitive computing becomesignificantly integrated into robotics, there is increasing interest in understanding how to developrobots that better respond to cyber security needs [10]. In chapter 3, we will discuss the challengesrelated to this aspect from regulatory perspective.

3 Regulatory FrameworkRobotics continues to open new opportunities and benefits in terms of efficiency and economicconvenience. Not only do these advantages encourage improvements in manufacturing and trade,but also in sectors, such as transportation, medical assistance, education, and agriculture [84].

12

However, despite these advantages, the development of robotics can also lead to severe problemsin the legal sphere. For example, some issues may include civil or criminal liability connectedto the use of robotic systems. The effort to regulate such a complex subject is, therefore, notexempt, among others, from a part dedicated to the regulation of safety, security, and privacyaspects of robotics.

3.1 Safety and Security in RoboticsAs robots continue to become more sophisticated and widespread, the need for complete robotsafety standards increases exponentially. Robots can be challenging to operate, especially whensafety controls are not applied. A robot safety standard is a set of guidelines for specificationsand controls concerning robots and their safe operations. Some of the common topics in this areainclude manufacturing, sales and use of robots [94] and are often created by a diverse group ofindustry experts to ensure that they provide benefits in different sectors. For example, in someareas, safety has already been addressed, particularly with regards to robotics systems adaptedto structured and unstructured environments [94]. The type of environment in which robotsoperate in may significantly impacts the safety characteristics and capabilities of a robot.

• Structured environments - A structured environment is a space that is visibly and accuratelydefined. Working in this type of environment means that a robot has a defined navigationprocedure and a clear perception of potential obstacles or impediments within a space. Anexample of standards within this category involves industrial robots, which, in Europe, arecovered within the scope of the Machinery Directive 2006/42/EC 7;

• Unstructured environments - An unstructured environment is a space that is chaotic andundefined. Unstructured environments may be more challenging for a robot to navigatebecause they must be equipped with advanced capabilities. These may include featuresaimed at identifying and adapting to unpredictable changes and variables (e.g. people,lighting, humidity, temperature, etc.). Some standards within this category include theGeneral Product Safety Directive 2001/95/EC (GPSD) and the Consumer Protection –Directive 1999/44/EC.

However, with the advancement of new technological systems, lawmakers are making adjust-ments and updates to these standards. In particular, much of the work of organizations involvedin improving robotics safety, such as the American National Standards Institute (ANSI), theInternational Organization for Standardization (ISO), and the International Electro-technicalCommission (IEC) includes harmonizing and creating international robot standards. For exam-ple, the following figure shows the connection between standards and the manufacturing system[101], Fig.n.5.

3.2 Privacy in RoboticsAccording to Rueben (2018) [85], privacy is defined as “the effective setting of boundaries betweenoneself and other people.” The author [94] states that these boundaries define the limits of per-sonal information, personal space, territory, social interaction, relationships, thoughts, feelings,opinions, and decisions. Robots play a crucial role in the establishment of these boundaries asthey are capable of collecting and sharing a significant amount of information, moving throughpersonal spaces and distances, and interacting with people. In particular, the social aspect ofrobots is one of the most controversial issues. Some scholars argue that humans often interact

7https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2006:157:0024:0086:EN:PDF

13

Figure 5: Smart Manufacturing System Pyramid

socially with machines, and this phenomenon, also known as “Computers Are Social Actors”(CASA), represents a significant threat to privacy [90], [102], [91]. Also, a recent study of theGhent U has shown that social robots are able to manipulate and trick people into unsafe actions8. 8;

This theory suggests that robots are perceived as social actors and, therefore, are included ininteractions that typically exist only among people. Calo (2010) [86] and other authors [92] 9,[103] observe that robots are quickly trending towards ubiquity, which involves several privacyimplications. Current generations of robots are equipped with microphones, connected sensors,cameras, GPS, rangefinders, accelerometers, etc. (Calo, 2010). The majority of robots collectinformation about the everyday life of users (Calo, 2010) and their sensitive characteristics, suchas emotional, medical, and mental states. For example, the development of robotics applicationsin the context of healthcare is one of the most discussed in this field. The use of robots tomonitoring the clinical and medical parameters of older adults and their transmission to hospitalsor doctors in real-time can bring concerns from a data protection perspective. One answer toprivacy threats identified in the literature is the concept of “privacy by design” (Calo, 2010; Lutzand Tamò [87], 2015; Sanfeliu, Llácer (Schafer, B., Edwards)[88]).

These authors argue that privacy protection needs to be taken into account from the verybeginning of the development process of robotics systems. However, not only is the “privacy bydesign” concept considered to be effective in the literature, but it is also an essential principlein the regulatory environment surrounding robotics [89]. To this extent, in Europe, the rela-tionship between robotics and privacy has been receiving particular attention over the last fewyears. For example, on 16 February 2017, the European Parliament passed a resolution withrecommendations to the European Commission on civil law rules on robotics (2015/2103(INL))10; following this initiative, the European Parliament (EP) has proposed many principles andrequirements for the development of a comprehensive regulatory framework on robotics. Exam-ples of these principles include the concept of reversibility, the inclusion of a protective stop, andthe possibility of attributing liability to robots.

The EU’s General Data Protection Regulation (GDPR) also includes requirements relatedto automated decision-making processes, fundamental rights concerning data subjects, and data

8https://www.pieterwolfert.com/files/lbr1162-wolfertA_accepted.pdf9https://watermark.silverchair.com/

10http://www.europarl.europa.eu/doceo/document/TA-8-2017-0051_EN.html

14

protection by design, which have a significant impact on the distribution of robotics systems.Additionally, GDPR includes specific requirements regarding software medical devices, which arekey to addressing challenges such as regulating the liability of all players involved in the roboticsproduction chain (e.g., producers, doctors, users, healthcare centers)[93]. However, despite theseefforts, the processing of personal data within robotics is still a regulatory goal because, to moveforward with privacy regulations, it is necessary for regulators to ti unify regulatory approachesto address privacy concerns in robotics.

4 Current and future threats and vulnerabilities

Figure 6: CIA-Triad

The methods used to implemented the bias in sensors data are the following:

• Bias through addition (Injection);

• Bias through multiplication (Scaling);

• Data can be manipulated by utilizing knowledge of the system’s model and the parametersof detection method (Stealth)

These attacks may cause failure to measure the distance from the robot to the surroundingobstacles and the consequent crash.

Availability and ConfidentialityIn [29] Gil et all. implemented a Sibyl attack involving swarm-drones. In particular this

attack compromised the drones capabilities and cooperation abilities. This type of attack used afake member to send a high number of requests to the server node, resulting in drone, degradationor unavailability of the servers.

Another important application field is Medical Robotics. Example include heterogeneousrobotic platforms used in surgery procedures (e.g. Da Vinci - Intuitive Surgical Ltd, Mako -Stryker Corporation, NAVIO - Smith+Nephew). In [19]the authors the effects of a DoS (Denialof Service) attack on the RAVEN II robotic system, which uses a master-slave communicationbetween the surgical console and the manipulator. Raven II is based on Linux O.S. and ROSmiddle-ware, and uses the ITP protocol(Inter-operable Telesurgery Protocol) to control inputand robot feedback.

In particular, this platform can be used in tele-operation and human operator-robot commu-nication. For example, during hijacking attacks, an attacker may induces the robot to completelyignore the intentions of a surgeon; packets may end up being forwarded towards the wrong partof the network, enter an endless loop, and potentially perform harmful actions.

15

Integrity and ConfidentialitySometimes, Hardware (HW) Trojan, a malicious addition and/or modification to ICs (Inte-

grated Circuits), may cause damages to robots [83]. Malicious users may discover the encryptionkey of the system and compromise the system (i.e. Malicious off-chip leakage enabled by sidechannels). Examples of attacks may include the following:

• During the fabrication phase, a malicious user can implement a backdoor in general-purposeprocessors bypass memory range protection using buffer overflow attacks, or gain access toprivileged assets by bypassing control protection mechanisms (confidentiality);

• Through HW Trojans, attackers could implement the stealth attack by modifying theoutput values of sensors (integrity).

In this context there are two ways to access robotic system: a) through the network or b)through physical components. An attacker needs network or physical access to a robot controlleror robotic set up to implement the attack (e.g access through industrial routers and compromise)robot functionalities, such as sensors reading, executing control logic, making precise movementsand ensuring human safety.

Specific types of attacks to industrial robots are listed below:

• Alteration of control-loop parameters;

• Tamper through modification of calibration parameters;

• Tamper through modification of the Production logic;

• Alteration of the user-perceived Robot state;

• Alteration of Robot state.

These attacks can alter the interaction between the robot and the surrounding physical en-vironment. For example in the case of production tampering, an attacker can use a file systemor an authentication-bypass vulnerability to compromise the manufacturing process, modify awork-piece or cause the robot to perform wrong task.

In [7] the authors the most common ways in which a robotic system can be compromised:

• Information disclosure: technical materials available on manufacturers website, includingsoftware images;

• Outdated software: custom patches applied by manufacturers to update the software, createopportunities for attackers leverage software vulnerabilities;

• Default authentication: remote connections enable attackers to compromise devices throughnull or "admin" default password;

• Poor transport encryption: for example, symmetric keys for VPNs or web-based adminis-tration are not available on HTTPS;

• Poor software protection: attackers can manipulate software images (e.g. debug informa-tion) that are available on manufacturers website.

• Security by obscurity: Poor information about robots may lead to unclear security.

16

In [5], the authors identified two principal attack vector for robotic system: USB physical[108] port (posed on or teach-pendant or robot controller), remote access through the network.

To make a robotic system safe, it is, therefore, necessary to implement the CIA Triad, whichprovides a core foundation for cyber security into robotics. For instance, one of the commonprinciples of the CIA Triad is data protection. While lost data can generally be restored frombackups, in robotics, this procedure may not be as effective. Dieber et al. [23] provide an exampleof a drone under attack. Simply shutting the drone’s system down would not be a good strategyas its basic functionality must be available until it reaches a safe state (e.g., it has landed).According to Clark et al. [3] most of the cyber security issues related to robots derive from thefact the design and manufacture of robots are generally not designed to include cyber security.In fact, development costs and delivering functionality to consumers are the real priorities whenbuilding robotics systems.

The fact that the manufacturer doesn’t consider security aspects in the design phase can leadto safety issues [108], [2]. However, robots are also moving from factories and general industrialareas into spaces occupied by individuals. The following areas are those most likely to needparticular attention in relation to this aspect:

• Collaborative robotics;

• Autonomous vehicles;

• Social robotics:

– eldercare robotics;– educative robotics.

These fields require continuous feedback from the operator/user (or feedback generatedthrough AI - Artificial Intelligence algorithm) and need to be equipped with sensors that en-able the operator to interact with them. Today big data-sets and software, cooperative learningcapabilities, knowledge sharing, and human knowledge can greatly contribute to maximizing thepotential of robots in these areas [46, 52].

For example in [12] Morimoto et al. discussed how ECU (Electronics Control Unit) installedon AUVs (autonomous underwater vehicles) can help improve energy efficiency and reduce un-necessary energy lost. The type of data coming from the car and the communication networkused to and from the autonomous car define the surface attack for a malicious user.

In this context, the use of AI might generate incorrect commands. For example, certain safetyconditions may be wrongly classified as low risk, therefore, a malicious users can manipulate intravehicle data or alter intra-vehicle communications to perform a cyber-attack on autonomousvehicles.

Most of the authors [4, 6, 7, 10], addressing this issue argue that the lack of security by designmay generate the following categories of threats:

• Insecure communications: issues between users and robotics systems may encouragea verity of cyber security risks. Kumar et al. [96] specifically addressed this issue insurgical robotics. They argued that there are several ways in which intruders may hackinto insecure communication links, especially if robots are connected to public networks.Additionally, some authors claim that plain or poorly encrypted text may enable attackersto obtain a significant amount of data from robotics systems. In particular [1, 4, 6, 7,10, 17, 20], they argue that the majority of threats related to communications may becaused by the use of libraries or applications connected to the Internet. Another roboticsarea, which is vulnerable to communication issues is firmware [1], [3], [4], [6],[7], [8], [9],

17

[11], [12], [14], [15], [16], [17], [18], [19], [20], [23], [24], [25], [26], [27]. For example,upgrading firmware online provides ample opportunities for attacks. Finally, since mostrobots are connected to the Internet via networks, attackers could gain full control ofrobots by exploiting communication vulnerabilities [1], [4], [5],[7],[10], [11], [12], [15], [16],[17], [23], [24], [25], [26], [27];

• Authentication issues: One of the most underestimated threats in robotics is that involv-ing authentication. Some robot applications are designed without the need for usernameand passwords, allowing anyone to access them remotely. However, even when these ser-vices use authentication features, attackers may bypass them [3], [4], [6],[7],[10], [17], [20].Similarly, most of the networks to which robots are connected networks are not passwordprotected and, therefore, vulnerable. Conversely, when they are protected, authenticationmechanisms may not be up-to-date, and unauthenticated users may exploit this vulnera-bility and access the network [1], [4], [5],[7],[10], [11], [12], [15], [16], [17], [23], [24], [25],[26], [27], [30]. The lack of authentication also means having no verification of whetherthe physical components of the robot are accessed or not. For this reason, attackers couldeasily interact with or tamper any components [3], [4], [5],[7],[10], [11], [13], [15], [17], [24],[25];

• Missing authorization: Only authorized users should have access to robotic devices andtheir resources. Failing to manage unauthorized access properly may enable attackers toeasily and remotely use certain robotic features and control the robot. At the applicationlevel, [3], [4], [5],[7],[10], [11], [13], [15], [17], [24], [25] most threats involve the ability toaccess robotic remotely by Internet services, software, mobile applications, etc. Addition-ally, because these applications communicate via networks, which may be the weakest linksduring an attack [1], [4], [5],[7],[10], [11], [12], [15], [16], [17], [23], [24], [25], [26], [27].Anyone within the same network can gain access to the robot and send commands. In caseof failed authentication, robots could also be attacked during the maintenance process ofits firmware [3], [4], [6],[7], [8], [9], [11], [12], [14], [15], [16], [17], [20], [23], [24], [25], [26],[27]. For example, some robot manufacturers make firmware available online for updates,leaving the device vulnerable. However, making firmware available to the public becomesan issue only if the firmware is modifiable. Finally, unauthorized physical access to a robotmay lead to availability issues. The intruder may attack the device hardware and use it tomanipulate its data or change its behavior [3], [4], [5],[7],[10], [11], [13], [15], [17], [24], [25];

• Privacy issues: Some researchers are concerned that robots could raise privacy concerns,giving companies tremendous access into people’s life. For example, robots’ mobile appli-cations can send private information to remote servers without user consent [2], [4], [6],[7],[10], [17]. At the firmware level, one of the major risks is that attacker get into therobot through firmware and then steal information, such as sensitive IP, logs, and othercontent [2], [4], [6], [7], [8],[9], [11], [12], [15], [16], [17], [18], [20], [23], [24], [25], [26], [27].Similarly, attacks that are performed at the network level may provide a vehicle for threatsto users’ privacy [1], [2], [4], [5],[7],[10], [11], [12], [15], [16], [17], [23], [24], [25], [26], [27];

• Weak default configuration: When robots include insecure features in their origi-nal configuration, they may easily be disabled or accessed. Generally, attacks exploit-ing these features operate at the hardware [3], [4], [5],[7],[10], [11], [13], [108],[15], [17],[24], [25]] and network level [4], [5],[7],[10], [11], [12], [15], [16], [17], [23], [24], [25],[26], https://arxiv.org/abs/1912.07714 [27], but there may also be applications accessi-ble through default passwords or built using vulnerable open source code and libraries [4],[6], [7], [10], [17], [20]. Additionally, attacks may also be performed to corrupt firmware

18

that is not properly configured or has an outdated configuration [4], [6], [7], [8],[9], [11],[12], [14], [15], [17], [18], [19], [20], [23], [24], [25], [26], [27].

Table n.6 summarizes the types of issues affecting robots and the related areas of vulnerability.

Table 6: ATTACK IN ROBOTIC SYSTEMCyber security Problems in Robotics Common Attacks

Hardware Attack Network Attack Firmware/OS Attack Application Attack

Insecure communications

[32], [37], [68], [83] [1], [4], [5], [7], [10], [1], [3],[4], [6], [7], [1], [4], [6], [7],[11], [12], [15], [16], [17], [8], [9], [11], [12], [14], [10], [17], [20], [32],[23], [24], [25], [26], [27], [15], [16], [17], [18], [19], [41], [43], [46], [48],[32], [43], [46], [47], [48], [20], [23], [24], [25], [26], [51], [57], [63], [69],

Insecure communications [49], [50], [51], [52], [53], [27], [40], [41], [43], [46], [77], [78][54], [55], [56], [57], [58], [47], [48], [50], [51], [59],[59], [60], [61],[63], [64], [61], [64], [65], [66], [67],[65], [66], [67], [68], [69], [68], [69], [71], [73], [74],[70], [71], [72], [73], [75], [75], [76], [78], [79], [81],[78], [79], [81] [82]

Authentication issues

[3], [4], [5], [7], [1], [4], [5], [7], [39], [49], [80] [3], [4], [6], [7],[10], [11], [13], [15], [10], [11], [12], [15], [10], [17], [20], [67]

Authentication issues [17], [24], [25], [31], [16], [17], [23], [24],[40] [25], [26], [27], [30],

[62], [77]

Missing authorization

[3], [4], [5], [7], [1], [4], [5], [7], [3], [4], [6], [7], [8], [3], [4], [6], [7],[10], [11], [13], [15], [10],[11], [12], [15], [9], [11], [12], [14],[15], [10], [17], [20], [37]

Missing authorization [17], [24], [25], [41] [16],[17], [23], [24], [16], [17], [18],[20],[23],[25], [26], [27], [41], [24], [25], [26],[27], [52],[73] [57], [75], [76]

Privacy issues

[39] [1], [2], [4], [5], [2], [4], [6], [7], [8], [2], [4], [6], [7],[7], [10], [11], [12], [9], [11], [12], [15],[16], [10],[17]

Privacy issues [15], [16], [17], [23], [17], [18], [20], [23], [24],[24], [25], [26], [27], [25], [26], [27], [32], [37],[39] [73], [77]

Weak default configuration

[3], [4], [5], [7], [4], [5], [7], [10], [4], [6], [7], [8], [4], [6], [7], [10],[10], [11], [13], [15], [11], [12], [15], [16], [9], [11], [12], [14], [17], [20], [71]

Weak default configuration [17], [24], [25], [47], [17], [23], [24], [25], [15], [16], [17], [18],[51], [83] [26], [27] [19], [20], [23], [24],

[25], [26], [27], [63]

However, when it comes to cyber security in the field of robotics, there is no single issuethat needs to be analyzed to ensure full protection. Given the increasing interconnectedness ofrobotic devices, attackers have found ways to perform multiple attacks and overcome traditionalbarriers. One of the best cyber security practices lies in creating a comprehensive architectureto mitigate attacks.

5 Current research issuesThe increasing dependence of businesses and customers on robotics devices and applicationsis leading to an exponential growth in terms of cyber risk. Cyber-attacks exploit any type ofvulnerabilities concerning robotics systems, whether they are come in the form of software orhardware, or are dependent on the person who uses them. Thus, because cyber-attacks areon the increase in this field, several scholars and experts are bringing cyber security into muchprominent focus when trying to find methods to mitigate cyber threats in robotics. Severalresearch areas should be further investigated. Below we give some examples.

• Security by design - Implementing security by design means reducing vulnerabilities insoftware / hardware. This procedure requires a proper consideration of security propertiesfrom the very beginning in the requirements phase of the development. In particular, it is

19

Table 7: AttacksAttack ReferenceDoS [3], [6], [7], [9], [11]

[12], [14], [15], [17][18], [19], [20], [23][25], [26], [27], [28][32], [41], [43], [50][57], [59], [61], [63][67], [69], [71], [73][75], [76], [79], [81][82]

Stuxnet [3], [40], [47], [57]MitM [6], [7], [16], [17]

[41], [43], [46], [57][69], [77], [80]

Eavesdropping Attack [3], [15], [17], [18][20], [41], [57], [63][67], [77], [79]

Fault Injection [3], [28], [41], [76]Phishing [63]HW Backdoor [3], [15], [41], [83]Sthealty Attack [28], [30]Tampering Attack [7], [24], [41], [73]Ransomware [37]Spoofing Attack [15], [18], [25], [32]

[41], [46], [47], [48][51], [62], [67], [79]

Teardrop Attack [18]Sybil Attack [26], [29], [32]Homing Attack [26]Jamming Attack [11], [18], [20]Jacking Attack [11]Brute FOrce Attack [31]RAT Attack [39], [41]Replay Attack [41], [57], [67], [71], [73], [79]Surge Attack [51]Surface Attack [57]Masquarade Attack [67]

20

necessary to consider security requirements engineering for robotics applications, includingprivacy and safety aspects. Also, the evolvement of requirements through the whole SDLC(systems development life cycle ) and the socio-economic impact of this evolvement shouldbe taken into account. Similarly, it is necessary to develop security support in programmingenvironments. This research area covers new programming platforms that deliver develop-ment and runtime environments for trustworthy application that is executed in complexrobotics scenarios. The purpose of this discipline is to implement language based security,as well as to secure coding principles and practices. Code signatures and instrumentations,are also an important component of this area;

• Security and safety co-engineering - Developing a system that is safe and secure isone of the headrest challenges. Depending on the context, these two concepts could becontrasting and potential solutions need to meet specific risk factors related to both fields;

• Monitoring - Monitoring and tracking robotic activities, accesses, and the use of privilegedaccounts can be an effective way to detect and mitigate the impact of some attacks preven-tively. According to Alemzadeh et al.[98], detection mechanisms can dynamically estimatethe consequence of the attacks before their effect manifests in the systems. Intrusion detec-tion and prevention systems should be adopted. Specific anomaly detection mechanisms,able to behaviourally fingerprinting robots behaviour could be also investigated;

• Data usage control - sensing is one of the main activities of robotics systems. Thoseoften collaborate with humans and the collected data should be controlled, where sharedand disseminated to other digital systems;

• Identity management - robotics systems are often composed of several devices thathave their own input output capabilities. Considering how to identify the robotic systemsis paramount for the consideration of trust issues related, for instance, to collaborativeaspects;

• Trustworthiness - Trust is an essential concept in human-robot interaction and their“secure relationship.” Trust, defined as "an attitude involving beliefs and expectations ofa trustee’s trustworthiness," [97] is often connected to vulnerability since the "trustor isdependent on the trustee," and there is always a certain degree of uncertainty about relyingon the trustee. Recent studies have proved that trust in social/professional robots is lostwhen functionalities and operation misbehaves and does not meet expectations. Trustis, therefore, a critical factor to consider when the goal is trusting that robotics systemsbehave securely (e.g., trusting that the information users get from the robotic device issecure and reliable). One example of developing trust in robotics is implementing securityby design and default when building robotics systems. Knowing that robot manufacturersand developers apply cyber security considerations throughout the design and developmentstages could help create a more operating framework for robots and their users as a warrantfor the whole robotics industry;

• Robustness - Organizations operating in the robotics field, especially those that havesuffered from the effects of cyber attacks, have strengthened perimeter security controls,adopted firewalls, and other protection systems. Although necessary, such security methodsare still not enough to protect companies from large-scale cyber threats. For this reason,several authors [57] discussed the importance of strengthening the robustness of communi-cations rather than focusing on enhancing other areas. According to Priyadarshini,[17] thetransmission medium is one of the most vulnerable components. The author argues that

21

ensuring that adding a layer of security to reinforce communications would reduce probableinsecurities.

More generally, other authors [100] argue that it is critical to have extensive knowledge andbe aware of the contemporary and existing cyber-attacks and countermeasures that are specificto robotics systems. The growing shortage of trained cyber security employees and the constantpressure to manage and reduce costs make it harder for companies to maintain robotics systemssecure or improve their cyber security posture more efficiently.

6 ConclusionIn this work, we reviewed the literature concerning the following topics: Robotics, IT technologiesused in Robotics and related fields. Firstly, we discussed the current cyber security scenario inrobotics; then, we classified and summarize the modules composing the robotic systems with theaim to analyze them in relation to their vulnerabilities.

In particular, we examined the connection between issues in robotics and other domains,such as security and safety. We provided an overview of the regulatory environment surroundingrobotics, which helped us frame the current situation in robotics systems.

Secondly, we discussed the problems derived from the interconnection between Robotics andIT technologies and the cyber security vulnerabilities affecting robotic systems in industrialcontexts and other sectors (i.e. house, autonomous vehicles, unstructured environment). Thirdly,we analyzed potential and actual cyber-attacks, provided a classification according to the CIAtriad concept, and divided them into categories of threats. The outcome of this analysis suggeststhat Robotics faces prominent challenges on security in the following areas:

• Collaborative Robotics;

• Autonomous vehicles;

• Autonomous Robotic platform;

• Regulation and regulatory frameworks.

Finally, we noticed that, in the last decades, the research and development in the roboticsfield shifted from a focus on industrial robots to a focus on intelligent robotics. This shiftcreated methods of easier integration to create robotics systems, which are capable of providingpromising results in different areas of robotic research, such as artificial intelligence, cognitiverobotics, human-robot interaction, multi-agent systems for mobile robot collaboration, etc.[33].In particular, the use of AI and ML algorithms led to new security and safety challenges. Theintroduction of mandatory regulatory requirements will probably slow down the pace of progressin robotics, but the current advanced robotics systems have enormous potential to transformmany aspects of people’s lives.

References[1] Víctor Mayoral Vilches, Laura Alzola Kirschgens, Asier Bilbao Calvo, Alejandro Hernán-

dez Cordero, Rodrigo Izquierdo Pisón, David Mayoral Vilches, Aday Muñiz Rosas, GorkaOlalde Mendia, Lander Usategi San Juan, Irati Zamalloa Ugarte, Endika Gil-Uriarte,Erik Tews, Andreas Peter - Introducing the Robot Security Framework (RSF), a standard-ized methodology to perform security assessments in robotics. Ledger Journal, 2019 -(10.5195/LEDGER.201X.X)

22

[2] Francisco J. Rodríguez Lera, Camino Fernández Llamas, Ángel Manuel Guerrero and VicenteMatellán Olivera - Cybersecurity of Robotics and Autonomous Systems: Privacy and Safety- Robotics – V Chapter of Legal, Ethical and Socioeconomic Impacts. Edited by GeorgeDekoulis – 2017

[3] J.George W. Clark Jr., Michael V. Doran, Todd R. Andel -Cybersecurity Issues in Robotics.2017 IEEE Conference on Cognitive and Computational Aspects of Situation Management(CogSIMA)

[4] Cesar Cerrudo, Lucas Apa - Hacking Robots Before Skynet. https://ioactive.com/hacking-robots-before-skynet/ 2017 IOActive, Inc. All Rights Reserved

[5] Marcello Pogliani, Davide Quarta, Mario Polino, Martino Vittone, Federico Maggi, StefanoZanero -Security of controlled manufacturing systems in the connected factory: the case ofindustrial robots. Journal of Computer Virology and Hacking Techniques (2019) 15:161–175

[6] Khalil M. Ahmad Yousef, Anas AlMajali, Salah Abu Ghalyon, Waleed Dweik and Bassam J.Mohd - Analyzing Cyber-Physical Threats on Robotic Platforms. Sensors 2018, 18(5), 1643

[7] Davide Quarta, Marcello Pogliani, Mario Polino, Andrea M. Zanchettin, and Stefano Zanero- Rogue Robots: Testing the Limits of an Industrial Robot’s Security. 2017 by Trend Micro,Incorporated. All rights reserved - A TrendLabs Research Paper (March 2017)

[8] Vicente Matellán, Tamara Bonaci, Giedre Sabaliauskaite - Cyber-security in robotics andautonomous systems. 0921-8890/ c© 2017 Elsevier B.V. All rights reserved.

[9] ] Anas AlMajali, Khalil M. Ahmad Yousef, Bassam J. Mohd, Waleed Dweik, Salah AbuGhalyon and Roa’a Hasan - SEMI-QUANTITATIVE SECURITY RISK ASSESSMENTOF ROBOTIC SYSTEMS. Jordanian Journal of Computers and Information Technology(JJCIT), Vol. 04, No. 03, December 2018.

[10] Laura Alzola Kirschgens, Irati Zamalloa Ugarte, Endika Gil Uriarte, Aday Muniz Rosas ,Vıctor Mayoral Vilches -ROBOT HAZARDS: FROM SAFETY TO SECURITY. arXiv R© isa registered trademark of Cornell University (https://arxiv.org/abs/1806.06681v3) Septem-ber 2019

[11] Ryan Shah -Security Landscape for Robotics. arXiv R© is a registered trademark of CornellUniversity (https://arxiv.org/abs/1904.03033v1) April 2019

[12] Shusuke Morimoto, Fang Wang, Ranchao Zhang, Jinghui Zhu -Cybersecurity in AutonomousVehicles. INTRODUCTION TO APPLIED INFORMATICS, UNIVERSITY OF HYOGO,MAY 2017

[13] Theodoros Theodoridis and Huosheng Hu -Toward Intelligent Security Robots: A Survey.IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLI-CATIONS AND REVIEWS, VOL. 42, NO. 6, NOVEMBER 2012

[14] Bernhard Dieber, Benjamin Breiling, Sebastian Taurer, Severin Kacianka, Stefan Rass,Peter Schartner -State-of-the-art in robot security. JOANNEUM RESEARCH Institutefor Robotics and Mechatronics Klagenfurt, Austria https://bernharddieber.com/talk/erf2019security/erf2019security.pdf

[15] Santiago Morante, Juan G. Victores and Carlos Balaguer -Cryptobotics: why robots needcyber safety. OPINION ARTICLE Front. Robot. AI, 29 September 2015

23

[16] Timothy A. Zimmerman -Metrics and Key Performance Indicators for Robotic CybersecurityPerformance Analysis. NIST Published: May 22, 2019

[17] Ishaani Priyadarshini -Cybersecurity risks in Robotics. Available on: https://www.researchgate.net/publication/326414211_Cyber_security_risks_in_robotics

[18] Delia Ioana Dogaru, Ioan Dumitrache -Cyber Security in Healthcare Networks The 6 thIEEE International Conference on E-Health and Bioengineering - EHB 2017 Grigore T. PopaUniversity of Medicine and Pharmacy, Sinaia, Romania, June 22-24, 2017

[19] Tamara Bonaci, Jeffrey Herron, Tariq Yusuf, Junjie Yan, Tadayoshi Kohno, Howard JayChizeck - Experimental Analysis of Denial-of-Service Attacks on Teleoperated Robotic Sys-tems. ICCPS ’15 Proceedings of the ACM/IEEE Sixth International Conference on Cyber-Physical Systems Pages 11-20 (2015)

[20] Azfar Khalid, Pierre Kirisci, Zeashan Hameed Khand, Zied Ghrairi, Klaus-Dieter Thoben,Jürgen Pannek -Security framework for industrial collaborative robotic cyber-physical systems.Computers in Industry 97 (2018) 132–145 c© 2018 Elsevier B.V. All rights reserved

[21] Marianna Lezzi, Mariangela Lazoi, Angelo Corallo -Cybersecurity for Industry 4.0 in thecurrent literature: A reference framework. Science Direct Computers in Industry Volume103, December 2018, Pages 97-110

[22] Dazhong Wu, Anqi Ren, Wenhui Zhang, Feifei Fan, Peng Liu, Xinwen Fu, Janis Terpenny -Cybersecurity for digital manufacturing. 0278-6125/ c© 2018 The Society of ManufacturingEngineers. Published by Elsevier Ltd. All rights reserved.

[23] Bernhard Dieber, Benjamin Breiling, Sebastian Taurer, Severin Kacianka, Stefan Rass, PeterSchartner -Security for the Robot Operating System. 0921-8890/ c© 2017 Elsevier B.V. Allrights reserved. Robotics and Autonomous Systems 98 (2017) 192–203

[24] Davide Quarta, Marcello Pogliani, Mario Polino, Federico Maggi, Andrea Maria Zanchettinand Stefano Zanero -An Experimental Security Analysis of an Industrial Robot Controller.2017 IEEE Symposium on Security and Privacy

[25] Gianluca Dini and Angelica Lo Duca -A Secure Communication Suite for Underwater Acous-tic Sensor Networks. Sensors 2012, 12, 15133-15158

[26] Guang Yang, Lie Dai and Zhiqiang Wei -Challenges, Threats, Security Issues and NewTrends of Underwater Wireless Sensor Networks. Sensors 2018, 18, 3907

[27] Mariantonietta La Polla, Fabio Martinelli, and Daniele Sgandurra -A Survey on Securityfor Mobile Devices. IEEE COMMUNICATIONS SURVEYS, TUTORIALS, VOL. 15, NO.1, FIRST QUARTER 2013

[28] Giedre Sabaliauskaite, Geok See Ng, Justin Ruths, Aditya Mathur -A comprehensive ap-proach, and a case study, for conducting attack detection experiments in Cyber–Physical Sys-tems. Robotics and Autonomous Systems 98 (2017) 174–191

[29] Stephanie Gil, Swarun Kumar, Mark Mazumder, Dina Katabi, Daniela Rus -GuaranteeingSpoof-Resilient Multi-Robot Networks. Autonomous Robots. 2017 ; Vol. 41, No. 6. pp. 1383-1400.

24

[30] Giedre Sabaliauskaite, Geok See Ng, Justin Ruths and Aditya Mathur -Experimental Eval-uation of Stealthy Attack Detection in a Robot. 2015 IEEE 21st Pacific Rim InternationalSymposium on Dependable Computing

[31] Manal Abd Al-Jabbar Ahmad Mizher, Riza Sulaiman -Robotic Movement Encryption UsingGuaranteed Cellular Automata. 2018 Cyber Resilience Conference (CRC), 1-3.

[32] Alguliyev, Rasim M., Yadigar Imamverdiyev and Lyudmila Sukhostat -Cyber-physical sys-tems and their security issues. Computers in Industry 100 (2018): 212-223.

[33] Ioan Stefan Sacala, Mihnea Alexandru Moisescu, Ioan Dumitrache Calin Aurel Munteanu,Simona Iuliana Caramihai -Cyber Physical Systems oriented Robot Development Platform.Procedia Computer Science 65 ( 2015 ) 203 – 209.

[34] Gunes, Volkan Peter, Steffen Givargis, Tony Vahid, Frank - A Survey on Concepts, Ap-plications, and Challenges in Cyber-Physical Systems. KSII Transactions on Internet andInformation Systems. 8. 4242-4268. 10.3837/tiis.2014.12.001.

[35] Shi, Jianhua, Jiafu Wan, Hehua Yan and Hui Suo -A survey of Cyber-Physical Systems.2011 International Conference on Wireless Communications and Signal Processing (WCSP)(2011): 1-6.

[36] Nazarenko, Artem A. and Luis M. Camarinha-Matos -Towards collaborative Cyber-PhysicalSystems. 2017 International Young Engineers Forum (YEF-ECE) (2017): 12-17.

[37] Reddy, Yenumula -Security and Design Challenges in Cyber-Physical Systems. Proceedings- 12th International Conference on Information Technology: New Generations, ITNG 2015.200-205. 10.1109/ITNG.2015.38.

[38] Baheti, Radhakisan and Helen Gill -Cyber-physical Systems. 2011

[39] Allison Nellis -Hello, Friend: Cybersecurity Issues in Season One of Mr. Robot. (2016)The Serials Librarian, 71:3-4, 203-211

[40] Wang, Lihui, Martin Törngren and Mauro Onori -Current status and advancement of cyber-physical systems in manufacturing. 2015

[41] Tuptuk, Nilufer and Stephen Hailes -Security of smart manufacturing systems. 2018

[42] Monostori, L. - Cyber-physical production systems: Roots from manufacturing science andtechnology. at Automatisierungstechnik, Vol. 63, No. 10, 2015, pp. 766-776. (ISSN: 0178-2312)

[43] Keith A. Stouffer, Victoria Y. Pillitteri, Suzanne Lightman, Marshall Abrams, Adam Hahn- Guide to Industrial Control Systems (ICS) Security. Special Publication (NIST SP) -800-82 Rev 2 (June 2015)

[44] Dutta, Vibekananda and Teresa Zielinska -Networking technologies for robotic applications.ArXiv abs/1505.07593 (2015)

[45] R. S. H. Piggin -Development of industrial cyber security standards: IEC 62443 for SCADAand Industrial Control System security. IET Conference on Control and Automation 2013:Uniting Problems and Solutions, Birmingham, 2013, pp. 1-6.

25

[46] Kehoe, Ben, Sachin Patil, Pieter Abbeel and Kenneth Y. Goldberg -A Survey of Research onCloud Robotics and Automation. IEEE Transactions on Automation Science and Engineering12 (2015): 398-409.

[47] Khorrami, Farshad, Prashanth Krishnamurthy and Ramesh Karri -Cybersecurity for ControlSystems: A Process-Aware Perspective. IEEE Design e Test 33 (2016): 75-83.

[48] Candell, Richard, Dhananjay M. Anand and Keith Stouffer -A Cybersecurity Testbed forIndustrial Control Systems. 2014

[49] Candell, Richard Stouufer, Keith -Measuring Impact of Cybersecurity on the Performanceof Industrial Control Systems. ASME Dynamic Systems and Control Magazine. 136.10.1115/1.2014-Dec-5.

[50] Byres, Eric, Pin-Kwang Eng and Justin Peter Lowe -The Myths and Facts behind CyberSecurity Risks for Industrial Control Systems. 2004

[51] Candell, Richard, Timothy A. Zimmerman and Keith A. Stouffer -An Industrial ControlSystem Cybersecurity Performance Testbed. 2015

[52] Saha, Olimpiya and Prithviraj Dasgupta -A Comprehensive Survey of Recent Trends inCloud Robotics Architectures and Applications. Robotics 7 (2018): 47.

[53] Lee, Gregory S. and Bhavani M. Thuraisingham -Cyberphysical systems security applied totelesurgical robotics. Computer Standards and Interfaces 34 (2012): 225-229.

[54] Michniewicz, Joachim Reinhart, Gunther -Cyber-Physical-Robotics – Modelling of modu-lar robot cells for automated planning and execution of assembly tasks. Mechatronics. 34.10.1016/j.mechatronics.2015.04.012.

[55] Michniewicz, Joachim and Gunther Reinhart -Cyber-physical Robotics – Automated Analy-sis, Programming and Configuration of Robot Cells based on Cyber-physical-systems. 2014

[56] Yogesh Kumar Sharma, Ashish Bagla -SECURITY CHALLENGES FOR SWARMROBOTICS. International Journal of Information Technology and Knowledge ManagementJanuary June 2009, Volume 2, No. 1, pp. 45-48.

[57] Bonaci, Tamara, Jeffrey Herron, Tariq Yusuf, Junjie Yan, Tadayoshi Kohno and HowardJay Chizeck -To Make a Robot Secure: An Experimental Analysis of Cyber Security ThreatsAgainst Teleoperated Surgical Robots. ArXiv abs/1504.04339 (2015).

[58] Birk, Andreas, Sören Schwertfeger and Kaustubh Pathak - A networking framework forteleoperation in safety, security, and rescue robotics. IEEE Wireless Communications 16(2009): 6-13.

[59] Higgins, Fiona, Allan Tomlinson and Keith M. Martin -Survey on Security Challenges forSwarm Robotics. 2009 Fifth International Conference on Autonomic and Autonomous Sys-tems (2009): 307-312.

[60] Fink, Jonathan, Alejandro Ribeiro and Vijay Kumar -Robust Control for Mobility and Wire-less Communication in Cyber–Physical Systems With Application to Robot Teams. Proceed-ings of the IEEE 100 (2012): 164-178.

26

[61] Kalam, Anas Abou El, Antoine Ferreira and Frédéric Kratz -Bilateral Teleoperation Sys-tem Using QoS and Secure Communication Networks for Telemedicine Applications. IEEESystems Journal 10 (2016): 709-720.

[62] Vattapparamban, Edwin, Ismail Güvenç, Ali Ihsan Yurekli, Kemal Akkaya and A. SelcukUluagac -Drones for smart cities: Issues in cybersecurity, privacy, and public safety. 2016International Wireless Communications and Mobile Computing Conference (IWCMC) (2016):216-221.

[63] Alemzadeh, Homa, Daniel Chen, Xiao Li, Thenkurussi Kesavadas, Zbigniew T. Kalbarczykand Ravishankar K. Iyer -Targeted Attacks on Teleoperated Surgical Robots: Dynamic Model-Based Detection and Mitigation. 2016 46th Annual IEEE/IFIP International Conference onDependable Systems and Networks (DSN) (2016): 395-406.

[64] Horton, Michael, Lei Chen and Biswanath Samanta -Enhancing the security of IoT enabledrobotics: Protecting TurtleBot file system and communication. 2017 International Conferenceon Computing, Networking and Communications (ICNC) (2017): 662-666.

[65] Dieber, Bernhard and Benjamin Breiling -Security Considerations in Modular Mobile Ma-nipulation. 2019 Third IEEE International Conference on Robotic Computing (IRC) (2019):70-77.

[66] Clark, George, Andel, Todd, Doran, Michael -Simulation-Based Reductionof Operational and Cybersecurity Risks in Autonomous Vehicles. 140-146.10.1109/COGSIMA.2019.8724160.

[67] Yağdereli, Eray, Cemal Gemci and A. Ziya Aktaş -A study on cyber-security of autonomousand unmanned vehicles. 2015

[68] Kinzler, Matt, Justin Miller, Zhou Wu, Andrew Williams, and Debbie Perouli -CybersecurityVulnerabilities in Two Artificially Intelligent Humanoids on the Market. Workshop on Tech-nology and Consumer Protection (ConPro ‘19), Held in Conjunction with the 40th IEEESymposium on Security and Privacy, April. http://par.nsf.gov/biblio/10099177

[69] Munteanu, Andrei, Riccardo Muradore, Massimo Merro and Paolo Fiorini -On cyber-physicalattacks in bilateral teleoperation systems: An experimental analysis. 2018 IEEE IndustrialCyber-Physical Systems (ICPS) (2018): 159-166.

[70] Miller, Justin andWilliams, Andrew and Perouli, Debbie -A Case Study on the Cybersecurityof Social Robots. 195-196. 10.1145/3173386.3177078.

[71] Vuong, Tuan, Avgoustinos Filippoupolitis, George Loukas and Diane Gan -Physical indica-tors of cyber attacks against a rescue robot. 2014 IEEE International Conference on PervasiveComputing and Communication Workshops (PERCOM WORKSHOPS) (2014): 338-343.

[72] Fraisse, Philippe and Zapata, Rene and Zarrad, Walid and Andreu, David -Remote Se-cure Decentralized Control Strategy for Mobile Robots. Advanced Robotics 2005 - 19.10.1163/156855305774307040.

[73] Martín, Francisco, Enrique Soriano and José M. Cañas -Quantitative analysis of security indistributed robotic frameworks. Robotics and Autonomous Systems 100 (2018): 95-107.

[74] Elkady, Ayssam Yehia and Tarek M. Sobh -Robotics Middleware: A Comprehensive Litera-ture Survey and Attribute-Based Bibliography. Robotics 2012 (2012): 959013:1-959013:15.

27

[75] Breiling, Benjamin, Bernhard Dieber and Peter Schartner -Secure communication for therobot operating system. 2017 Annual IEEE International Systems Conference (SysCon)(2017): 1-6.

[76] Rivera, Sean, Sofiane Lagraa and Radu State -ROSploit: Cybersecurity Tool for ROS. 2019Third IEEE International Conference on Robotic Computing (IRC) (2019): 415-416.

[77] Mukhandi, Munkenyi, David Portugal, Samuel Pereira and Micael S. Couceiro -A novelsolution for securing robot communications based on the MQTT protocol and ROS. 2019IEEE/SICE International Symposium on System Integration (SII) (2019): 608-613.

[78] DeMarinis, Nicholas, Stefanie Tellex, Vasileios P. Kemerlis, George Konidaris and RodrigoFonseca -Scanning the Internet for ROS: A View of Security in Robotics Research. 2019International Conference on Robotics and Automation (ICRA) (2018): 8514-8521.

[79] Lera, Francisco J. Rodríguez, Vicente Matellán Olivera, Jesús Balsa-Comerón, Ángel ManuelGuerrero Higueras and Camino Fernández Llamas -Message Encryption in Robot OperatingSystem: Collateral Effects of Hardening Mobile Robots. Front. ICT 2018 (2018).

[80] Thulasiraman, Preetha -Study of Security Primitives for the Robot Operating System (ROS)of UAV Swarms. 2017 https://calhoun.nps.edu/handle/10945/53348 Calhoun.

[81] Lera, Francisco J. Rodríguez, Jesús Balsa, Fernández Casado, Camino Fernández Llamas,Francisco Martín Rico and Vicente Matellán Olivera -Cybersecurity in Autonomous Systems:Evaluating the performance of hardening ROS. 2016

[82] Marotta, Angelica and Martinelli, Fabio and Nanni, Stefano and Orlando, Albina and Yaut-siukhin, Artsiom -Cyber-insurance survey. Computer Science Review (2017) pag.35-61,Vol.24.

[83] Bhunia, Swarup, Michael S. Hsiao, Mainak Banga and Seetharam Narasimhan -HardwareTrojan Attacks: Threat Analysis and Countermeasures. Proceedings of the IEEE 102 (2014):1229-1247.

[84] Borenstein, J. Pearson, Y. (2010)- Robot caregivers: harbingers of expanded freedom for all?- Ethics and Information Technology, 12(3): 277-288.

[85] Rueben, M., Smart, W. D., Grimm, C. M., Cakmak, M. (2017, March)-Privacy-SensitiveRobotics. In Proceedings of the Companion of the 2017 ACM/IEEE International Conferenceon Human-Robot Interaction (pp. 425-426). ACM.

[86] Calo, R. (2010)-Robots and privacy. In G. B. Patrick Lin K. Abney (Eds.). Robot ethics:The ethical and social implications of robotics. Cambridge: MIT Press.

[87] Lutz, C., Tamò, A. (2015) -RoboCode-ethicists: Privacy-friendly robots, an ethical respon-sibility of engineers? In Proceedings of the ACM Web Science Conference. New York, NY:ACM. doi:10.1145/2786451.2786465

[88] Schafer, B., Edwards, L. (2017)-"I spy, with my little sensor": Fair data handling practicesfor robots between privacy, copyright, and security. Connection Science, 29(3), 200–209.

[89] Holder, C., Khurana, V., Harrison, F., Jacobs, L. (2016) -Robotics and law: Key legal andregulatory implications of the robotics age (Part I of II). Computer Law and Security Review,32(3), 383-402.

28

[90] Rueben, Matthew, Cindy Grimm, Frank J. Bernieri and William D. Smart -A Taxonomy ofPrivacy Constructs for Privacy-Sensitive Robotics. ArXiv abs/1701.00841 (2017)

[91] Katagiri, Y., Nass, C. and Takeuchi, Y. -Cross-Cultural Studies of the Computers are SocialActors Paradigm: The Case of Reciprocity. In: Proceedings of the Ninth InternationalConference on Human-Computer Interaction 2001. (2001),1558-1562.

[92] Rueben, Matthew -Privacy in Human-Robot Interaction : Survey and Future Work. (2016).

[93] Dolic, Z. Castro, R. Moarcas, A. -Robots in Healthcare: A Solution Or a Problem? :Workshop Proceedings European Parliament Book 2019 - SN 9789284647705, https://books.google.it/books?id=YGrexQEACAAJ

[94] Marty A., Hartmut R.-Legal and ethical considerations in the era of autonomous robots TheExecutive M.B.L.-HSG University of St.Gallen’s (HSG)15. Februar 2018

[95] Baumann, Rainer & Schmid, Stefan. (2019)- Voice Over IP - Security and SPIT SwissArmy, FU Br 41, KryptDet Report.

[96] Kumar, Raghavendra, Pattnaik, Prasant Kumar, Pandey, Priyanka - Detecting and Miti-gating Robotic Cyber Security Risks SN - 9781522521556, Advances in Information Secu-rity, Privacy, and Ethics, IGI Global, 2017, 336-348 https://books.google.it/books?id=NWtxDgAAQBAJ

[97] Allison Langer, Ronit Feingold-Polak, Oliver Mueller, Philipp Kellmeyer, Shelly Levy-Tzedek (2019) - Trust in socially assistive robots: Considerations for use in rehabilitationNeuroscience and Biobehavioral Reviews 104 (2019) 231–239.

[98] Homa Alemzadeh, Daniel Chen, Xiao Li, Thenkurussi Kesavadas, Zbigniew T. Kalbarczyk,Ravishankar K. Iyer - Targeted Attacks on Teleoperated Surgical Robots: Dynamic Model-based Detection and Mitigation 2016 46th Annual IEEE/IFIP International Conference onDependable Systems and Networks.

[99] EY Report -How do you protect the robots from cyber attack? https://www.ey.com/Publication/vwLUAssets/ey-how-do-you-protect-robots-from-cyber-attack/\protect\T1\textdollarFILE/ey-how-do-you-protect-robots-from-cyber-attack.pdf

[100] Abdullahi Chowdhury, Gour Karmakar, Joarder Kamruzzaman - Survey of Recent CyberSecurity Attacks on Robotic Systems and Their Mitigation Approaches Detecting andMitigating Robotic Cyber Security Risks. IGI Global, 2017. pp. 284-299.

[101] Yan Lu, KC Morris, Simon Frechette -Current Standards Landscape for Smart Manufac-turing Systems National Institute of Standards and Technology - NISTIR 8107

[102] Nass, Clifford & Steuer, Jonathan & Siminoff, Ellen - Computer are social ac-tors Conference on Human Factors in Computing Systems (1994) - Proceedings. 204.10.1145/259963.260288.

[103] Chibani, Abdelghani & Amirat, Yacine & Mohammed, Samer & Matson, Eric & Hagita,Norihiro & Barreto, Marcos - Ubiquitous Robotics: Recent Challenges and Future TrendsRobotics and Autonomous Systems (2013). 61. 1162-1172. 10.1016/j.robot.2013.04.003.

29

[104] Agarwal, Ashish & Agarwal, Aparna - The Security Risks Associated with Cloud ComputingInternational Journal of Computer Applications in Engineering Sciences (2011). [VOL I,SPECIAL ISSUE ON CNS]

[105] R. Davies - The Internet of Things Opportunities and Challenges, European ParliamentaryResearch Service. 2015 PE 557.012 http://www.europarl.europa.eu/RegData/etudes/BRIE/2015/557012/EPRS_BRI(2015)557012_EN.pdf ).

[106] Zamalloa, Irati & Muguruza, Iñigo & Hernandez, Alejandro & Kojcev, Risto & Mayoral-Vilches, Víctor - An information model for modular robots: the Hardware Robot InformationModel (HRIM). 2018

[107] Sattarova Feruza Y. and Prof.Tao-hoon Kim - IT Security Review: Privacy, Protection,Access Control, Assurance and System Security. International Journal of Multimedia andUbiquitous Engineering Vol. 2, No. 2, April, 2007

[108] Mayoral-Vilches, Víctor and Juan, Lander and Carbajo, Unai and Campo, Rubén andCámara, Xabier and Urzelai, Oxel and García, Nuria and Gil-Uriarte, Endika - Industrialrobot ransomware: Akerbeltz - arXiv:1912.07714v1 [cs.CR] 16 Dec 2019

[109] Mayoral-Vilches, Víctor & Olalde, Gorka & Baskaran, Xabier & Cordero, Alejandro &Juan, Lander & Gil-Uriarte, Endika & Urabain, Odei & Kirschgens, Laura. - Aztarna, afootprinting tool for robots - 2018

30