Embed Size (px)
Citation preview
sensors
Article
Multiple Participants’ Discrete Activity Recognition in aWell-Controlled Environment Using Universal Software RadioPeripheral Wireless SensingUmer Saeed 1,* , Syed Yaseen Shah 2 , Syed Aziz Shah 1 , Haipeng Liu 1 , Abdullah Alhumaidi Alotaibi 3 ,Turke Althobaiti 4 , Naeem Ramzan 5 , Sana Ullah Jan 6 , Jawad Ahmad 6 and Qammer H. Abbasi 7
�����������������
Citation: Saeed, U.; Yaseen Shah, S.;
Aziz Shah, S.; Liu, H.; Alhumaidi
Alotaibi, A.; Althobaiti, T.; Ramzan,
N.; Ullah Jan, S.; Ahmad, J.; Abbasi,
Q.H. Multiple Participants’ Discrete
Activity Recognition in a
Well-Controlled Environment Using
Universal Software Radio Peripheral
Wireless Sensing. Sensors 2022, 22, 809.
https://doi.org/10.3390/s22030809
Academic Editors: Paolo Barsocchi,
Filippo Palumbo, Victor Hugo C. De
Albuquerque and Akash Kumar Bhoi
Received: 12 November 2021
Accepted: 17 January 2022
Published: 21 January 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
1 Research Centre for Intelligent Healthcare, Coventry University, Coventry CV1 5FB, UK;[email protected] (S.A.S.); [email protected] (H.L.)
2 School of Computing, Engineering and Built Environment, Glasgow Caledonian University,Glasgow G4 0BA, UK; [email protected]
3 Department of Science and Technology, College of Ranyah, Taif University, P.O. Box 11099,Taif 21944, Saudi Arabia; [email protected]
4 Faculty of Science, Northern Border University, Arar 91431, Saudi Arabia; [email protected] School of Computing, Engineering and Physical Sciences, University of the West of Scotland,
Paisely PA1 2BE, UK; [email protected] School of Computing, Edinburgh Napier University, Edinburgh EH10 5DT, UK; [email protected] (S.U.J.);
[email protected] (J.A.)7 James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, UK;
[email protected]* Correspondence: [email protected]
Abstract: Wireless sensing is the utmost cutting-edge way of monitoring different health-relatedactivities and, concurrently, preserving most of the privacy of individuals. To meet future needs,multi-subject activity monitoring is in demand, whether it is for smart care centres or homes. In thispaper, a smart monitoring system for different human activities is proposed based on radio-frequencysensing integrated with ensemble machine learning models. The ensemble technique can recognise awide range of activity based on alterations in the wireless signal’s Channel State Information (CSI).The proposed system operates at 3.75 GHz, and up to four subjects participated in the experimentalstudy in order to acquire data on sixteen distinct daily living activities: sitting, standing, and walking.The proposed methodology merges subject count and performed activities, resulting in occupancycount and activity performed being recognised at the same time. To capture alterations owing toconcurrent multi-subject motions, the CSI amplitudes collected from 51 subcarriers of the wirelesssignals were processed and merged. To distinguish multi-subject activity, a machine learning modelbased on an ensemble learning technique was designed and trained using the acquired CSI data. Formaximum activity classes, the proposed approach attained a high average accuracy of up to 98%.The presented system has the ability to fulfil prospective health activity monitoring demands and isa viable solution towards well-being tracking.
Keywords: USRP; RF sensing; software-defined radio; multi-subject monitoring; smart healthcare;ensemble learning
1. Introduction
Due to a wide variety of applications, human activity sensing (detection or monitoring)has received immense attention in recent years. The goal is to identify how humans reactand behave especially in enclosed environments. Some of the applications where humanactivity sensing is highly considered are: smart home systems, context-aware systems,Internet-of-Things (IoT) systems, and healthcare systems, in particular for the monitoringof elderly people [1,2]. Lately, non-invasive human activity sensing schemes have beenproposed based on Software-Defined Radio (SDR) [3], WiFi [4], and Radio Detection and
Sensors 2022, 22, 809. https://doi.org/10.3390/s22030809 https://www.mdpi.com/journal/sensors
Sensors 2022, 22, 809 2 of 16
Ranging (RADAR) technology [5]. The key characteristic of the human activity sensingsystem is the human body, which is primarily water (up to 60%) and capable of reflectingradio signals. As a result, the adjacent radio system’s received signal properties changeand indicate distinct human activities.
In most conventional human activity detection systems, either wearable devices orcamera-based technologies are employed [6,7]. Although these technologies are highlyaccurate, nevertheless, certain limitations are associated with them. For instance, camera-based technology possesses the risk of privacy, and wearable devices may cause discomfortwhile being attached to a body for long periods. Therefore, a non-invasive (or contactless)technology is required such as Radio-Frequency (RF) sensing, which has limited privacyconcerns and does not require being attached to a body. Over the years, research onRF sensing technology has provided several advantages over conventional methods [8].For instance, RF-based methods do not require the placement of any sensors on the humanbody. The reflection of the wireless signals from the human body is used to assess differentactions of humans. As a result, patients such as pregnant women, children, and the elderlywill find it easier to be monitored by contactless technology [9,10].
The majority of current research on human activity sensing has focused on singlesubjects, whereas research on multiple subjects is still in its early stages [11,12]. With theadvancement of microelectronics and IoT technologies, sensors are increasingly becomingprevalent in everyday life. According to estimates from the Information Handling Servicescompany, the IoT industry will increase from 15 billion devices deployed in 2015 to 30 bil-lion in 2020 and about 75 billion in 2025 [13]. A vast range of information can be obtainedwith the help of these smart devices, which can be useful in a variety of fields, includingpublic safety, businesses, and healthcare.
In this paper, we focused on multi-subject activity sensing using Universal SoftwareRadio Peripheral (USRP) devices. The USRP devices are based on SDR technology em-ployed for distinct RF applications. To the best of our knowledge, this research work isthe first of its kind to utilise non-invasive technology for multi-subject activity sensingconsidering a realistic environment. In general, multi-subject activities can be classifiedinto three categories as per the tasks that subjects perform [14]:
• Group activity: In a group activity, two or more subjects perform a common tasksimultaneously, for example two subjects performing a walking activity;
• Multi-individual activity: In a multi-individual activity, two or more subjects per-form tasks that are not related. For example, one subject performs the sitting activity,while another subject performs the standing activity;
• Mixed activity: In a mixed activity, both above-mentioned activities are considered,for example two subjects at the same time performing the walking activity, while onesubject performs the sitting activity.
The rest of this paper is organised in the following manner: Section 2 provides up-to-date information about wireless sensing technologies. Section 3 presents the proposedscheme based on SDR sensing. Section 4 presents experimental outcomes, and lastly,Section 5 provides concluding remarks and recommendations for further study.
2. Related Work
This section compares existing wireless sensing techniques, which were effectivelyutilised in the past for various applications, including human activity sensing. Thesetechnologies are based on the Channel State Information (CSI) approach, the ReceivedSignal Strength Indicator (RSSI) approach, the RADAR approach, and the SDR approach.
The CSI-based approach exploiting WiFi technology has recently become prevalent forfeature extraction in human-related activity recognition [15–17]. Several research works havefocused their efforts on creating CSI-based applications such as human presence detection [18],human crowd reckoning [19], indoor localisation [20], and a fall/collapse detection schemefor the elderly and young [21]. According to some of the current literature, WiFi signals are
Sensors 2022, 22, 809 3 of 16
capable of detecting and distinguishing even tiny movements of the human body, for instancemouth motions [22], keystrokes [23], heart rate [24], and respiratory rate [25].
The RSSI-based approach for human activity sensing is primarily dependent on thereceived signal strength variations induced by distinct human actions [26]. In comparisonto SDR and CSI, the RSSI-based system has limited detecting ability and precision. The SDR-based method improves the identification accuracy by up to 72%, allowing for a betterresolution of RSSI capture [27]. With the lack of frequency diversity, which is present inCSI-based systems, the precision and coverage region of RSSI-based systems are lower.The RSSI is recorded by a single value per packet, while the CSI is assessed per OrthogonalFrequency-Division Multiplexing (OFDM) from each packet. This makes the CSI approachmore stable and provides further information compared to RSSI. As a result, CSI is morerobust towards challenging situations.
The RADAR-based approach with a significantly greater bandwidth is also employedfor human activity recognition [28,29]. In contrast to WiFi-based technology, Frequency-Modulated Continuous Wave (FMCW) RADAR exploits a bandwidth up to 1.79 GHz,while WiFi technology only uses a bandwidth up to 20 MHz [30]. The RADAR-basedmethods are used for micro-Doppler information extraction and have a greater distanceresolution of around 20 cm [31,32]. Nevertheless, RADAR-based solutions need specialisedprocessing units and hardware.
The SDR-based approach is a specifically built hardware that can be employed tosense several human activities [33,34]. WiSee utilises USRP to detect the Doppler shifts inwireless signals and accomplish activity identification with an accuracy of up to 94% [35].Using specialised circuit hardware, Allsee technologies introduced a short-range detectiontechnique for gesture recognition that is less than 2.5 ft [36,37]. The sole method forextracting wireless CSI from WiFi signals without upgrading or modifying the hardware isto employ a platform based on SDR [38].
3. Methodology
This section presents a complete methodology adopted to carry out this research.It begins with a paragraph on the experimental scheme and proceeds on to a thoroughoverview of the hardware design stage, data acquisition, data wrangling, and proposedsystem model training based on the Machine Learning (ML) ensemble approach. Figure 1depicts the proposed system’s overall concept and primary components. The conceptualdesign, as depicted, is based on a non-invasive sensing system that can detect the presenceand activity of several individuals in the same space. Moreover, integrating a wireless sens-ing system with advanced Artificial Intelligence (AI) algorithms can assist in identifying avariety of human activities in real time.
3.1. Experimental Scheme
To confirm the efficacy of the proposed methodology, a number of experiments wereconducted in a rectangular activity area of 2.8 × 3 m2, as shown in Figure 2. In theopposite corners, two X-Series USRP devices for CSI signal transmission and receivingwere placed. During multiple experiments, the subjects (or participants) changed positionsarbitrarily in the specified activity area while maintaining a one-meter distance betweenthemselves to capture maximal intraclass variance for all activities, including sitting,standing, and walking. This scenario was performed to mimic a diminutive setting as in anelderly care facility with a small group of individuals. The experiment’s current focus wason accommodating four individuals; however, this number will be increased in the futureresearch work. Furthermore, the proposed ML-based classification approach to recognisemultiple participants’ activities is made up of two primary modules: model trainingand model testing. The model training module uses an offline approach to train the MLalgorithm using previously obtained and preprocessed CSI samples data. The model testingtook place in an online environment, where an input CSI data sample was categorisedas one of the human activities after all essential preprocessing was completed. The twomodules are described in detail in Section 4.
Sensors 2022, 22, 809 4 of 16
Figure 1. Proposed system concept design based on non-contact sensing technology and AI formultiple individuals’ activity recognition.
Figure 2. Setup for recording distinct activities utilising the 5G frequency in an experimental setting [39].
In one of the previous works [39], a similar CSI dataset was adopted, and experimentswere carried out in multiple phases to identify different human activities. The informationregarding the dataset is provided in Table 1 (see Figure 3 for the data samples). Each datasample of the activity consists of approximately 50× 1200 data points. For instance, 100 datasamples are equivalent to 100× 50× 1200 data points. To recognise multi-subject activities,a deep-learning-based solution called the “convolutional neural network” was designedand trained in the afore-mentioned paper. The classification accuracy and confusion matrixwere used as evaluation metrics. The employed approach attained an average accuracy of91.25% for single-subject activity and an overall 83% accuracy for four subjects’ activitiessegregated into sixteen distinct classes. In order to improve the performance of recognisingmultiple activities based on CSI data, we performed certain enhancements, which aredescribed as follows.
1. Instead of deep learning, we adopted an ensemble-learning-based solution in thispaper. The ML ensemble technique is explained in Section 3.5, while Section 4 revealsthe experimental outcomes based on an ensemble approach for the recognition ofmultiple activity classes;
Sensors 2022, 22, 809 5 of 16
2. In order to design a lightweight scheme that can be effectively utilised for the real-timesystem, we exceedingly reduced the data points from the original dataset. The de-tails regarding the employed data for training and testing are provided in the firstparagraph of Section 4;
3. Instead of assessing a classifier’s efficiency based on accuracy and a confusion matrix,we exploited six distinct criteria to evaluate the performance of the trained ML clas-sifiers, including accuracy, precision, recall, F1-score, confusion matrix, and modeltraining duration. The reason for selecting multiple evaluation criteria is that classi-fiers can fail to correctly identify some classes, even if the overall accuracy is higher(e.g., due to excessive false positives and lower true positives). However, the precisionwill decrease significantly.
Table 1. Number of participants and performed activities (see Figure 3 for the data samples).
No. of Participants No. of Classes Activities Performed No. of Data Samples
0 1 Empty Room 117
12 1 Sitting 1403 1 Standing 1404 1 Walking 140
2
5 1 Sitting + 1 Standing 1006 1 Walking + 1 Sitting 1007 2 Sitting 1008 2 Standing 100
3
9 1 Sitting + 2 Standing 12010 1 Walking + 2 Sitting 12011 2 Sitting + 1 Standing 10012 3 Sitting 10013 3 Standing 100
414 4 Sitting 10015 4 Standing 10016 2 Sitting + 2 Standing 100
(a)
(b)
Figure 3. Cont.
Sensors 2022, 22, 809 6 of 16
(c)
(d)
(e)
Figure 3. Samples of obtained CSI data from different activities: (a) no participant (empty room),(b) 1 participant, (c) 2 participants, (d) 3 participants, and (e) 4 participants (see Table 1).
Sensors 2022, 22, 809 7 of 16
3.2. Hardware Design
The experiment carried out in this research study made use of two USRP devices, eachoutfitted with the VERT2450 omnidirectional antenna. One USRP served as the transmitter,while the other served as the receiver. Each USRP was linked to a separate PC with 16 GBRAM and an Intel Core i7-3.60 GHz CPU. To provide the Ubuntu 16.04 operating system,the system made use of a Virtual Machine (VM). GNU Radio was employed to interact withthe USRPs on the Ubuntu VM. The USRP function can be carried out using flow diagramsgenerated using GNU Radio. Following that, the flow diagrams can be transformed toPython scripts. A single Python script was used to send data to the transmitter, whileanother received data from it. The transmitter used OFDM to send random integers from0–255. The transmitter sent a signal to the receiver, which was configured to receive it.The CSI complex number was then output to the terminal by the script, which ran on thereceiver side. Following that, the amplitude readings from the CSI complex numbers weretaken from this output. Table 2 lists the system’s primary configuration parameters.
Table 2. Parameter selection and configuration of the software.
Parameter Value
Platform USRP X300/310OFDM Subcarriers 51
Operating Frequency 3.75 GHzTransmitter Gain 70 dB
Receiver Gain 50 dB
3.3. Data Acquisition
The acquisition of data for the proposed scheme’s model training comprised fivephases, with four participants performing three distinct activities (sitting, standing, walk-ing) in a laboratory setting, as illustrated in Figure 2. The setup was duplicated in twodistinct laboratory settings to introduce varying amounts of complexity, increasing thevolatility of the data and strengthening the system model. Nonetheless, the data for aspecific activity class gathered in both laboratory settings were handled as a single dataset,indicating that the amount of complexity caused by the surroundings was not a measurablevariable in the experiments carried out.
The same as any other experiment, there were both fixed and variable characteristics inthis experimental study. The fixed characteristics for the experiment presented in this studywere: the hardware and its setup; the data wrangling and ML approaches; the experimentalsetting. The variable characteristics were: the number of participants; the participantidentity; the position of the performed activity (e.g., one participant performing the sittingactivity in various chair positions, as shown in Figure 2). The first variable was assessed,and the outcomes are emphasised in the rest of this section and in the findings, whereasthe second and third variables were used to establish the utmost intraclass variance inthe obtained data. To ensure data reproducibility across different days, all data werecollected over the course of a week, with a randomised amount of data taken for eachof the 16 classes on each day of the week. The activities sitting and standing depict theaction of conducting these tasks rather than the posture/position of an individual in thesitting or standing condition. The participants were not compelled to maintain their upperbody motionless while recording the activity data; therefore, both the sitting and standingactivity data contained minor shifts of the upper body. The details regarding each activityemployed in this paper are listed in Table 1 and explained in the following paragraph.
In the first data acquisition stage, a single participant’s CSI data were gathered sepa-rately for the sitting, standing, and walking activities, where 420 samples were obtained intotal. Three distinct participants took part in the data acquisition phase to ensure maximalvariance. Each participant contributed evenly to the data acquisition, i.e., each participanttook part in the acquisition of one-hundred forty samples, which were split among thethree activities classes. In 3 s, 1200 packets were transmitted for each CSI data sample.
Sensors 2022, 22, 809 8 of 16
In the second data acquisition stage, two participants performed the three activities indi-cated above, for a total of 400 samples. This data acquisition step featured the same threeindividuals with equal contributions, i.e., from each participant, a minimum of thirty-threesamples were acquired per the four classes chosen for this phase of data acquisition. Threeand four volunteers were enrolled in the third and fourth data acquisition stages to assistin the collecting of data on multiple activities simultaneously. The individuals recruited forthis data acquisition phase remained the same throughout. In this phase, 540 and 300 datasamples were obtained in total. Additionally, 117 samples of data were acquired for theclass “Empty”, which depicts the state of the space when the participants are not present.Each of the 16 classes are listed in Table 1. Figure 3 shows the data samples from the distinctactivities. The interclass variance in the data samples from various activity classes wasevident, and it can be used in the classification procedure to improve the outcomes.
3.4. Data Wrangling
As stated in Equation (1), the CSI for a single transmitter and a receiver antennaproduces a matrix, including frequency reactions for all N = 51 subcarriers.
H = [H1( f ), H2( f ), . . . , HN( f )]T (1)
The frequency here per subcarrier Hi can be written as:
Hi( f ) = |Hi( f )|ej∠Hi( f ) (2)
The ith subcarrier amplitude and phase responses are represented by |Hi( f )| and∠Hi( f ). Each of these subcarrier responses relates to the system input and output, as pro-vided in Equation (3).
Hi( f ) =Xi( f )Yi( f )
(3)
where Xi( f ) and Yi( f ) are the Fourier transforms of the system’s input and output,respectively.
The obtained CSI data were generally obscured as a result of high-frequency ambientnoise and multi-path CSI signal propagation. As a result, the data were sent through thesubsequent data processing or wrangling phases in order to denoise them and preparethem for the ML training process:
1. In the initial phase, as shown in Equation (4), each sample of CSI data was aver-aged over all 51 subcarriers to obtain a single averaged data sample for consequentprocessing.
xi =1J
J
∑j=1
yij (4)
where xi is the ith data sample, which portrays an average across equivalent subcarri-ers yij for (j = 1, 2, 3, ..., 51);
2. After that, each averaged data sample xi was smoothed and minor variations wereremoved using the Butterworth lowpass filter of order n = 4;
3. Subsequently, the approximation coefficients Ai for every smooth data sample si wereobtained utilising the discrete wavelet transform with a Haar basis function at Level3. Since the approximation coefficients reflect the turnout of the lowpass filter in thediscrete wavelet transform, it further assisted the noise reduction. The downsamplingand convolution processes entailed in the wavelet decomposition across all threeextents are expressed mathematically as follows:
A0i [m] =
M−1
∑k=0
si[k]× g[2m− k], for m = 1, 2, . . . , M (5)
Sensors 2022, 22, 809 9 of 16
A1i [t] =
T−1
∑k=0
A0i [k]× g[2t− k], for t = 1, 2, . . . , T =
M2
(6)
A2i [u] =
U−1
∑k=0
A1i [k]× g[2u− k], for u = 1, 2, . . . , U =
M4
(7)
where g[k] for k = 1, 2, 3, ..., K represents the lowpass filter of length K per decomposi-tion level, si[m] for m = 1, 2, 3, ..., M represents the smooth signal of length M afterimplementing the Butterworth lowpass filter, and Al
i for levels l = 0, 1, 2 representsthe approximation coefficient of three levels of the discrete wavelet transform.
3.5. Ensemble-Technique-Based Training
The purpose of the ML-based ensemble technique is to aggregate the predictions ofseveral base estimators created using a specific learning algorithm to increase generalisationand robustness over a single estimator. Averaging is one of the methods used in theensemble technique, where the idea is to build numerous estimators independently andthen average their estimates. Since its variance is decreased by this approach, the mergedestimator is generally enhanced compared to any of the single base estimators [40].
The Extremely Randomised Trees or Extra Trees (ETs) and the Random Forest (RFo)algorithm are two averaging approaches based on randomised Decision Trees (DTs). Bothapproaches are perturb and merge methods created particularly for trees. This showsthat by inserting randomness into the ML classifier building, a varied group of classifiersis created. The averaged prediction of the individual classifiers was used to obtain theensemble prediction. Figure 4 illustrates the ensemble method, which consisted of multipleDTs. An individual tree is composed of a root node, child nodes, and leaf nodes.
Figure 4. Ensemble-technique-based classification approach.
In RFo, each tree in the ensemble is generated using a sample selected with replace-ment from the training data. The optimal split is selected from all input features or arandom subset of size max features when dividing each node during tree building. On theother hand, randomness is taken a step further in the case of ETs. A random subset offeatures is utilised in the same way as RFo; however, rather than looking for the mostdiscriminating thresholds, thresholds are produced at random for each feature. The bestof these randomly produced thresholds is chosen as the splitting criteria. This generally
Sensors 2022, 22, 809 10 of 16
allows for a small reduction in model variance at the risk of a slight increase in bias. In thispaper, we used both of the ensemble techniques (ETs/RFo) for classification purposes andcompared their performance with DTs.
4. Experimental Findings
All simulations for this work were performed on a MacBook Air with a processor1.6 GHz Dual-Core Intel Core i5 and memory 8 GB 1600 MHz DDR3. The Python pro-gramming language was utilised for the ML part, primarily using the scikit-learn, NumPy,and pandas libraries. As described earlier in this paper, the data were acquired based on16 different activity classes (see Table 1). Out of the total acquired data for each activity,we exploited 3600× 1200 data points or observations. Considering all 16 activity classes(16× 3600), a total of 57600× 1200 data points were used for experimental purposes. Weconsiderably reduced the data from the original datasets in order to design a lightweightsolution that can be efficiently used for real-time schemes. From the total data points,70% were used for training the ML models and 30% for testing. The hyperparametersfor training the ML algorithms were obtained through the grid-search technique. Thesehyperparameters are revealed in Table 3.
Using a single performance-examining criterion for the algorithms is typically nota good practise in the ML world. If the classifier somehow does not accurately identifysome classes, the accuracy may still be greater, for instance high false positives and lowtrue positives. Nevertheless, the precision will decline significantly. As a result, evaluatinga classifier’s effectiveness only based on accuracy is inadequate. Therefore, to examine theperformance of the trained ML classifiers in this work, we used six different criteria, suchas accuracy, precision, recall, F1-score, confusion matrix, and training time of the models.
In Figure 5a, the ET performance can be noted for the precision, recall, and F1-score.For all 16 activity classes, the ET scored more than 92%. From Classes 9–16, the ET attainedup to a 100% score for all evaluation metrics. The minimum score noted was 93% precisionfor the first class, that is an empty room. In Figure 5b, the performance of RFo can be noted.Although many variations can be seen for all 16 activity classes, nevertheless, RFo scoredmore than 90% for each distinct class. The minimum score noted was 91% recall for thethird class, that is one person standing. The maximum score noted with all three evaluationmetrics was 100% for the twelfth and thirteenth classes, which were three persons sittingand three persons standing. In Figure 5c, the DT performance can be noted. The minimum(second and third class) and maximum (eleventh, twelfth, and thirteenth class) scoresattained by the DT were 84% and 96%, respectively. This shows how the DT can performclassification tasks well; nevertheless, an ensemble of DTs (ET/RFo) can perform evenbetter by the combination of multiple DTs.
In Figure 6a, the ET performance can be seen in the form of a confusion matrix.From Classes 4–16, the ET accomplished a maximum prediction score of about 99%. Someof the classes such as the first, second, and third had a few misclassifications that resultedin a prediction score of 92%, 96%, and 97%, respectively. In Figure 6b, the performance ofRFo can be observed. As shown, RFo accomplished up to 99% correct prediction scores forthe activity classes: ninth (1 sitting + 2 standing), tenth (1 walking + 2 sitting), eleventh(2 sitting + 1 standing), twelfth (3 sitting), thirteenth (3 standing), fourteenth (4 sitting),and sixteenth (2 sitting + 2 standing). The class with the maximum misclassification andminimum prediction score was the first one (empty room). The possible reason for that isthe data features of the first class and how it was different from rest of the classes, whichwere entirely based on some activity happening. In Figure 6c, the DT performance can benoted. For most of the activity classes, the DT obtained more than a 90% prediction scoreexcept a few activity classes: second (1 sitting), third (1 standing), and fourth (1 walking).As can be noted, the DT held more misclassifications compared to the ET and RFo.
Sensors 2022, 22, 809 11 of 16
Table 3. Machine learning algorithms’ hyperparameters adopted by the grid-search methodfor training.
Classifier Hyperparameters
Extra Tree
bootstrap = Falseccp − alpha = 0
class − weight = Nonecriterion = gini
max − depth = Nonemax − features = auto
max − leaf − nodes = Nonemax − samples = None
min − impurity − decrease = 0min − impurity − split = None
min − samples − leaf = 1min − samples − split = 2
min − weight − fraction − leaf = 0n − estimators = 5n − jobs = None
oob − score = Falserandom − state = None
verbose = 0warm − start = False
Random Forest
bootstrap = Trueccp − alpha = 0
class − weight = Nonecriterion = gini
max − depth = Nonemax − features = auto
max − leaf − nodes = Nonemax − samples = None
min − impurity − decrease = 0min − impurity − split = None
min − samples − leaf = 1min − samples − split = 2
min − weight − fraction − leaf = 0n − estimators = 5n − jobs = None
oob − score = Falserandom − state = None
verbose = 0warm − start = False
Decision Tree
ccp − alpha = 0class − weight = None
criterion = ginimax − depth = None
max − features = Nonemax − leaf − nodes = None
min − impurity − decrease = 0min − impurity − split = None
min − samples − leaf = 1min − samples − split = 2
min − weight − fraction − leaf = 0presort = deprecated
random − state = Nonesplitter = best
Sensors 2022, 22, 809 12 of 16
(a) (b)
(c)
Figure 5. Precision, recall, and F1-score comparison for individual activity classes on: (a) extra tree;(b) random forest; (c) decision tree.
Furthermore, Table 4 presents the overall classification accuracy of ML classifiers forall 16 activity classes and the average training time of the models in terms of seconds. Weexploited the cross-validation method using the parameter cv = 5 to check the overall accu-racy of the algorithms for all activity classes. The cross-validation approach randomly splitsa set of data observations into k groups, or folds of roughly comparable size. The techniquewas applied to the remaining k− 1 folds, with the first fold functioning as a validation set.Cross-validation is an approach utilised in applied ML to evaluate the model’s competenceon unknown data. It is a well-known strategy since it produces a less biased assessment ofthe ML model compared to other approaches.
Table 4. Classifiers’ average training time and overall accuracy score by cross-validation using theparameter cv = 5.
Classifier Training Time Accuracy
Extra Tree 2.24 s 98%
Random Forest 10.52 s 97%
Decision Tree 120.61 s 90%
Sensors 2022, 22, 809 13 of 16
(a)
(b)
Figure 6. Cont.
Sensors 2022, 22, 809 14 of 16
(c)
Figure 6. Confusion matrix report of sixteen distinct activity classes by trained model: (a) extra tree;(b) random forest; (c) decision tree.
As shown in Table 4, the ET accomplished the highest accuracy up to 98% with aminimal training time of 2.24 s. RFo attained up to 97% accuracy with a training time ofabout 10 s. The DT reached the utmost accuracy rate of 90% with the highest training time ofaround 120 s. These results reveal how an ensemble-based ML approach can be effectivelyused for classification tasks. The randomisation technique in ensemble approaches (i.e., ET)improves the tree’s diversity and makes it easier to reduce the correlation when developingthe trees. Ensemble learning approaches can be described as divide-and-conquer techniquesor the wisdom of the crowd. Stable and more robust ML models can be obtained withaccurate predictions utilising an ensemble method since it highly reduces the bias andvariance errors. As a result, the ET approach can be emphasised for classification andregression tasks. The quick training time of the ET demonstrates how it can be utilised fora vast number of applications, especially real-time lightweight systems for hospitals andcare centres.
5. Conclusions and Future Work
The objective of this study was to propose a contactless RF sensing system for detectingand monitoring human presence and simultaneously activities (such as sitting, standing,and walking) utilising CSI signals. The system operated at 3.75 GHz, that is within the 5Gfrequency range of 3.4–3.8 GHz. The primary aim was to propose a non-invasive sensingsystem that could identify the presence and activity of multiple humans in the same space.The findings of this paper reveal that integrating RF sensing technology with state-of-the-art ensemble-based machine learning algorithms (such as extra trees) can efficientlyrecognise diverse human activities of daily living. The system was put to the test to seehow well it could recognise simultaneous activities among a variety of subjects, rangingfrom 0–4. Variations were added to train the system on 16 various activities, simulatinga realistic real-life scenario. The activity recognition experiments revealed that all test
Sensors 2022, 22, 809 15 of 16
participants’ activities were recognised with up to 98% accuracy. The findings of this studyare intriguing and have much potential for real-world applications.
The goal for future research is to expand the experiments to cover the majority of thehuman activity spectrum in different spaces with the maximum number of subjects. Exper-iments will be carried out to see how the number of spaces affects the performance of theproposed scheme. Additionally, various heights and placements of the transmitter/receiverantennas will be investigated.
Author Contributions: Conceptualisation, U.S., S.A.S., H.L., S.U.J. and Q.H.A.; formal analysis,U.S., S.Y.S., S.A.S., J.A., A.A.A., T.A., N.R. and Q.H.A.; funding acquisition, S.A.S., A.A.A. and T.A.;investigation, S.A.S., J.A., A.A.A., T.A., N.R. and Q.H.A.; methodology, U.S.; project administration,S.A.S.; resources, S.Y.S. and S.U.J.; software, U.S.; supervision, S.A.S.; validation, J.A., N.R. and S.U.J.;writing—original draft preparation, U.S.; writing—review and editing, U.S. All authors have readand agreed to the published version of the manuscript.
Funding: This work is supported in parts by the Engineering and Physical Sciences ResearchCouncil (EPSRC): EP/R511705/1 and EP/T021063/1. The authors extend their appreciation to theDeputyship for Research Innovation, Ministry of Education in Saudi Arabia for funding this researchwork through the project number IF-2020-NBU-201 and in part by the Taif University, Taif, SaudiArabia, through the Taif University Research Grant under Project TURSP-2020/277.
Institutional Review Board Statement: The study was conducted according to the guidelines andapproved by the Institutional Review Board (or Ethics Committee) of University of Glasgow, UK(Application No: 300190109—10 February 2020).
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.
References1. Shah, S.A.; Fioranelli, F. RF sensing technologies for assisted daily living in healthcare: A comprehensive review. IEEE Aerosp.
Electron. Syst. Mag. 2019, 34, 26–44. [CrossRef]2. Taylor, W.; Shah, S.A.; Dashtipour, K.; Zahid, A.; Abbasi, Q.H.; Imran, M.A. An intelligent non-invasive real-time human activity
recognition system for next-generation healthcare. Sensors 2020, 20, 2653. [CrossRef] [PubMed]3. Estrela, V.V.; KHELASSI, A.; Monteiro, A.C.B.; Iano, Y.; Razmjooy, N.; Martins, D.; Rocha, D.T. Why software-defined radio (SDR)
matters in healthcare? Med. Technol. J. 2019, 3, 421–429. [CrossRef]4. Wang, Y.; Liu, J.; Chen, Y.; Gruteser, M.; Yang, J.; Liu, H. E-eyes: Device-free location-oriented activity identification using
fine-grained wifi signatures. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking,Maui, HI, USA, 7–11 September 2014; pp. 617–628.
5. Fioranelli, F.; Le Kernec, J.; Shah, S.A. Radar for health care: Recognizing human activities and monitoring vital signs. IEEEPotentials 2019, 38, 16–23. [CrossRef]
6. Yatani, K.; Truong, K.N. Bodyscope: A wearable acoustic sensor for activity recognition. In Proceedings of the 2012 ACMConference on Ubiquitous Computing, Pittsburgh, PA, USA, 5–8 September 2012; pp. 341–350.
7. Dehbandi, B.; Barachant, A.; Smeragliuolo, A.H.; Long, J.D.; Bumanlag, S.J.; He, V.; Lampe, A.; Putrino, D. Using data from theMicrosoft Kinect 2 to determine postural stability in healthy subjects: A feasibility trial. PLoS ONE 2017, 12, e0170890. [CrossRef][PubMed]
8. Latif, S.; Qadir, J.; Farooq, S.; Imran, M.A. How 5g wireless (and concomitant technologies) will revolutionize healthcare? FutureInternet 2017, 9, 93. [CrossRef]
9. Saeed, U.; Shah, S.Y.; Zahid, A.; Ahmad, J.; Imran, M.A.; Abbasi, Q.H.; Shah, S.A. Wireless Channel Modelling for Identifying SixTypes of Respiratory Patterns with SDR Sensing and Deep Multilayer Perceptron. IEEE Sens. J. 2021, 21, 20833–20840. [CrossRef]
10. Malasinghe, L.P.; Ramzan, N.; Dahal, K. Remote patient monitoring: A comprehensive study. J. Ambient Intell. Humaniz. Comput.2019, 10, 57–76. [CrossRef]
11. Gravina, R.; Ma, C.; Pace, P.; Aloi, G.; Russo, W.; Li, W.; Fortino, G. Cloud-based Activity-aaService cyber–physical framework forhuman activity monitoring in mobility. Future Gener. Comput. Syst. 2017, 75, 158–171. [CrossRef]
12. Fortino, G.; Parisi, D.; Pirrone, V.; Di Fatta, G. BodyCloud: A SaaS approach for community body sensor networks. Future Gener.Comput. Syst. 2014, 35, 62–79. [CrossRef]
13. Lucero, S. IoT platforms: Enabling the Internet of Things. 2016. Available online: https://cdn.ihs.com/www/pdf/enabling-IOT.pdf (accessed on 5 November 2021).
Sensors 2022, 22, 809 16 of 16
14. Li, Q.; Gravina, R.; Li, Y.; Alsamhi, S.H.; Sun, F.; Fortino, G. Multi-user activity recognition: Challenges and opportunities. Inf.Fusion 2020, 63, 121–135. [CrossRef]
15. Halperin, D.; Hu, W.; Sheth, A.; Wetherall, D. Tool release: Gathering 802.11 n traces with channel state information. ACMSIGCOMM Comput. Commun. Rev. 2011, 41, 53–53. [CrossRef]
16. Zhou, Z.; Yang, Z.; Wu, C.; Shangguan, L.; Liu, Y. Towards omnidirectional passive human detection. In Proceedings of the 2013Proceedings IEEE INFOCOM, Turin, Italy, 14–19 April 2013; pp. 3057–3065.
17. Shah, S.A.; Tahir, A.; Ahmad, J.; Zahid, A.; Pervaiz, H.; Shah, S.Y.; Ashleibta, A.M.A.; Hasanali, A.; Khattak, S.; Abbasi, Q.H.Sensor fusion for identification of freezing of gait episodes using WiFi and radar imaging. IEEE Sens. J. 2020, 20, 14410–14422.[CrossRef]
18. Zhou, R.; Lu, X.; Zhao, P.; Chen, J. Device-free presence detection and localization with SVM and CSI fingerprinting. IEEE Sens. J.2017, 17, 7990–7999. [CrossRef]
19. Xi, W.; Zhao, J.; Li, X.Y.; Zhao, K.; Tang, S.; Liu, X.; Jiang, Z. Electronic frog eye: Counting crowd using WiFi. In Proceedings of theIEEE INFOCOM 2014-IEEE Conference on Computer Communications, Toronto, ON, Canada, 27April–2 May 2014; pp. 361–369.
20. Yang, Z.; Zhou, Z.; Liu, Y. From RSSI to CSI: Indoor localization via channel response. ACM Comput. Surv. (CSUR) 2013, 46, 1–32.[CrossRef]
21. Wang, Y.; Wu, K.; Ni, L.M. Wifall: Device-free fall detection by wireless networks. IEEE Trans. Mob. Comput. 2016, 16, 581–594.[CrossRef]
22. Wang, G.; Zou, Y.; Zhou, Z.; Wu, K.; Ni, L.M. We can hear you with WiFi! IEEE Trans. Mob. Comput. 2016, 15, 2907–2920.[CrossRef]
23. Ali, K.; Liu, A.X.; Wang, W.; Shahzad, M. Keystroke recognition using wifi signals. In Proceedings of the 21st Annual InternationalConference on Mobile Computing and Networking, Paris, France, 7–11 September 2015; pp. 90–102.
24. Liu, J.; Chen, Y.; Wang, Y.; Chen, X.; Cheng, J.; Yang, J. Monitoring vital signs and postures during sleep using WiFi signals. IEEEInternet Things J. 2018, 5, 2071–2084. [CrossRef]
25. Gao, Q.; Tong, J.; Wang, J.; Ran, Z.; Pan, M. Device-Free Multi-Person Respiration Monitoring Using WiFi. IEEE Trans. Veh.Technol. 2020, 69, 14083–14087. [CrossRef]
26. Gu, Y.; Ren, F.; Li, J. Paws: Passive human activity recognition based on wifi ambient signals. IEEE Internet Things J. 2015,3, 796–805. [CrossRef]
27. Sigg, S.; Scholz, M.; Shi, S.; Ji, Y.; Beigl, M. RF-sensing of activities from non-cooperative subjects in device-free recognitionsystems using ambient and local signals. IEEE Trans. Mob. Comput. 2013, 13, 907–920. [CrossRef]
28. Van Dorp, P.; Groen, F. Feature-based human motion parameter estimation with radar. IET Radar, Sonar Navig. 2008, 2, 135–145.[CrossRef]
29. Seyfioglu, M.S.; Özbayoglu, A.M.; Gürbüz, S.Z. Deep convolutional autoencoder for radar-based classification of similar aidedand unaided human activities. IEEE Trans. Aerosp. Electron. Syst. 2018, 54, 1709–1723. [CrossRef]
30. Wang, W.; Liu, A.X.; Shahzad, M.; Ling, K.; Lu, S. Understanding and modeling of wifi signal based human activity recognition.In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, Paris, France, 7–11 September2015; pp. 65–76.
31. Du, L.; Li, L.; Wang, B.; Xiao, J. Micro-Doppler feature extraction based on time-frequency spectrogram for ground movingtargets classification with low-resolution radar. IEEE Sens. J. 2016, 16, 3756–3763. [CrossRef]
32. Saeed, U.; Shah, S.Y.; Alotaibi, A.A.; Althobaiti, T.; Ramzan, N.; Abbasi, Q.H.; Shah, S.A. Portable UWB RADAR sensing systemfor transforming subtle chest movement into actionable micro-doppler signatures to extract respiratory rate exploiting ResNetalgorithm. IEEE Sens. J. 2021, 21, 23518–23526. [CrossRef]
33. Ashleibta, A.M.; Zahid, A.; Shah, S.A.; Imran, M.A.; Abbasi, Q.H. Software Defined Radio Based Testbed for Large Scale BodyMovements. In Proceedings of the 2020 IEEE International Symposium on Antennas and Propagation and North AmericanRadio Science Meeting, Montreal, QC, Canada, 4–11 July 2020; pp. 2079–2080.
34. Zhang, T.; Song, T.; Chen, D.; Zhang, T.; Zhuang, J. WiGrus: A WiFi-based gesture recognition system using software-definedradio. IEEE Access 2019, 7, 131102–131113. [CrossRef]
35. Pu, Q.; Gupta, S.; Gollakota, S.; Patel, S. Whole-home gesture recognition using wireless signals. In Proceedings of the 19thAnnual International Conference on Mobile Computing & Networking, New York, NY, USA, 30 September–4 October 2013;pp. 27–38.
36. Wang, W.; Liu, A.X.; Shahzad, M.; Ling, K.; Lu, S. Device-free human activity recognition using commercial WiFi devices. IEEE J.Sel. Areas Commun. 2017, 35, 1118–1131. [CrossRef]
37. Kellogg, B.; Talla, V.; Gollakota, S. Bringing gesture recognition to all devices. In Proceedings of the 11th {USENIX} Symposiumon Networked Systems Design and Implementation ({NSDI} 14), Seattle, WA, USA, 2–4 April 2014; pp. 303–316.
38. Ashleibta, A.M.; Abbasi, Q.H.; Shah, S.A.; Khalid, M.A.; AbuAli, N.A.; Imran, M.A. Non-Invasive RF Sensing for DetectingBreathing Abnormalities using Software Defined Radios. IEEE Sens. J. 2020, 21, 5111–5118. [CrossRef]
39. Ashleibta, A.M.; Taha, A.; Khan, M.A.; Taylor, W.; Tahir, A.; Zoha, A.; Abbasi, Q.H.; Imran, M.A. 5g-enabled contactless multi-userpresence and activity detection for independent assisted living. Sci. Rep. 2021, 11, 1–15.
40. Saeed, U.; Jan, S.U.; Lee, Y.D.; Koo, I. Fault diagnosis based on extremely randomized trees in wireless sensor networks. Reliab.Eng. Syst. Saf. 2021, 205, 107284. [CrossRef]
