1st International Conference of Recent Trends in Information and Communication Technologies
*Corresponding author: [email protected]
Comparison of Hidden Markov Model and Naïve Bayes Algorithms among
Events in Smart Home Environment
Abba Babakura*, Md Nasir Sulaiman, Norwati Mustapha, Khairul A. Kasmiran
Faculty of Computer Science and Information Technology, UPM Serdang, Malaysia
Abstract
The smart home environment consists of numerous subsystems which are heterogeneous in
nature. Smart home environment are configured in such a way that it comfort driven as well as
achieving optimized security and task-oriented without human intervention inside the home. The subsystems, due to their diversified nature develop difficulties as the events communicate
making the smart home uncomfortable. The complexity of decision making in handling events
stands at the bottleneck in ensuring various tasks executed jointly among diversified systems in smart home environment. In this paper, we propose Hidden Markov Model (HMM) and Naïve
Bayes (NB) to test the accuracy and response time of the home data and to compare between
the two algorithms. The result experimented shows that the HMM algorithm stands at higher
accuracy and better response time than the NB. The implementation has been carried out in
such a way that quality information is acquired among the systems to demonstrate the
effectiveness of decision making among events in the smart home environment.
Keywords: Smart home; HMM; Naïve Bayes; Decision making; Feature selection
1 Introduction
Generally, smart home is seen as an entity integrated with diversified service function of
automation, communication and control of its environment, and performing them in unified
manner via intelligent tasks [1]. In smart home, there is high interest and priority for low cost
solutions with high performance technologies consolidated together. These technologies
include the rise of high-speed communication structure and assured rapid increment of
diversified systems in home environment. In this context, many researchers try to develop
smart home technologies to provide assistance in handling the events in the environment by
utilizing machine learning algorithms. The identification of the ongoing inhabitant (events) is
IRICT 2014 Proceeding
12th -14th September, 2014, Universiti Teknologi Malaysia, Johor, Malaysia
Abba Babakura et. al. /IRICT (2014) 1-11 2
one of the main issues in smart homes. In most cases, the events are inferred with a plan context
that consists of basic activities predefined by an expert [2], [3]. The smart home environment
consists of numerous subsystems which are heterogeneous in nature. The subsystems, due to
their diversified nature develop difficulties as the events communicate making the smart home
uncomfortable. Heterogeneous systems in the smart home consists of the building automation
system (BAS), energy management system, fire alarm system, digital surveillance system and
other network based systems. Due to the high number of events and their complexity, the static
decision making algorithms could not handle the problem accurately and efficiently. Therefore,
another option is to build an automatic learning algorithm that can accurately and efficiently
handle or enumerate the sequence of events occurring from the subsystems in the smart home
environment. Symbolically, it is difficult to enumerate all the possible events occurring in a
home because of the large dataset. Nevertheless, new ways allow us to explore this paradigm
with supervised learning methods, which eliminate the requirement of a human intervention in
the learning process.
To do so, one avenue is to utilize the machine learning algorithms to handle the decision
making among the events and to accurately learn and adapt to changes as events are occurring
in sequence. We propose, in this paper, machine learning algorithms namely- Hidden Markov
Model (HMM) and Naïve Bayesian (NB) to address the problem of learning the sequence of
events in the smart home and to compare the accuracy and response time of the algorithms.
The paper begins with describing related works in Section 2. This is preceded with
methodological development elaborated in Section 3 and accompanied by experimental result
and evaluation in Section 4. The conclusion of the research is preceded in Section 5.
2 Related work
The necessity to ensure learning of events in other to make decision among subsystems in smart
home environment can be seen from diverse outlook. In recent times, literatures as well work-in
progress on smart home research suggested the importance of using the machine learning
algorithms in terms of decision making due to the fact that it provide effective and efficient
results. Available literatures on smart home research works highlighted the importance of
applying the HMM and NB in different tasks before achieving adaptability using decision
models. An important piece of work done by Rashidi et al., on automated approach for activity
tracking that naturally occurs in a smart home user lifestyle. Their work focuses on tracking the
Abba Babakura et. al. /IRICT (2014) 1-11 3
incident of regular tasks with intention to monitor health and identifying changes in a user
patterns and lifestyle [4].
On the other hand, another important work introduces the use of passive sensors and a Hidden
Markov Model as a means to identify individuals [5]. The result is a passive, low profile means
to attribute individual events to unique residents. Here, HMM is deployed to compare against a
prior naïve Bayes solution on the same data sets. One significant work to focus on is the work
done by Perumal et al. [6]. In this study, Perumal et al. discussed the importance of providing a
decision support among heterogeneous systems using the ECA based interoperability
framework. The work also highlighted how overall interoperability is obtained using decision
models benchmarked using SOAP protocols. This shows that there is need to use the decision
making technique among heterogeneous systems. Furthermore, one important piece of work
done by Freitag et.al. discusses a method, called “shrinkage” that improves the sampling as
well as computing HMM architecture probabilities for training data [7], [8]. In this work, they
deployed an optimization procedure for feasible selection of HMM architecture corresponding
bespoke training data requirement for smart home. A significant work presented by Cheng et al.
provides a dominant inference engine based on HMM, known as ALHMM, combining the
Viterbi and Baum-Welch algorithms measuring accuracy purposes and learning ability
enhancement [9]. Another significant work was conducted by Aaron et al., in their study
attributing events to individuals in multi-inhabitant environments, where their aim is to identify
the individuals and their activities in an intelligent environment using passive sensors,
deploying the use of naïve Bayesian algorithm resulted in obtaining an average accuracy of
96% [10]. Finally, another significant work was conducted, using Hidden Markov Model for
resident identification in smart home environment [5]. In their course of study they aim to
recognize inhabitants or individuals correctly using the Hidden Markov Model technique
providing average accuracies of 94.0% and 90.2% using two different data sets.
Therefore the proposed model in this research is designed to overcome the disadvantage of
other used algorithms in making decision among subsystem events and to provide a clear
methodology by utilizing the two aforementioned algorithms HMM and NB, which has the
properties of stochastic and mathematical functions that makes it have more advantage over
other algorithms leading to higher accuracy and improved response time.
Abba Babakura et. al. /IRICT (2014) 1-11 4
3 Framework Development
A first step to understand a phenomenon is to collect long term data (observations) then to build
a behavioral model of this system. The mathematical model can then be played with the added
flexibility to modify, adjust and play various scenarios. Here we propose two algorithms HMM
and NB, they are both a powerful and robust tools for stochastic processes as they lead to
effective and efficient build up of model achieving high accuracy with relatively low model
complexity. The two algorithms used in this study are namely: Hidden Markov Model (HMM)
and Naive Bayes (NB). These two algorithms have the flexibility of dealing with sequential
data which often makes them to be unique and draws attention of using them. The popularity of
this models and their ability to learn fast and adapt to changes in the occurrence of series of
data makes them to be scalable and reliable in the field of smart home. The methodological
framework can be seen in Figure 1 below. The framework design consists of four components
as displayed:
Model Data,
HMM and NB classifiers,
Automated Classification,
Prediction and System Evaluation.
It solely describes the role of the architecture to perform cooperative execution of tasks among
subsystems. The large arrows in between the phases indicate the system flow among
components. In the Model Data components, a set of data and rules are generated and stored in
the database. The rules are stored as raw data in .csv format in which a particular tool is used to
run it.
Abba Babakura et. al. /IRICT (2014) 1-11 5
Figure 1: Framework Development
Phase 1: Model Data
Phase 3: Automated Classification
Test Data Set
HMM & NB-Based System
System Evaluation
Surveillance
subsystem
Audio subsystem Building
Automation
Energy
management
DATA SELECTION
Feature Selection
Training and Test Data Set
(Baum-Welch)
Phase 2: Classifiers (HMM &NB)
Training Data Set
System Implementation
Implementation:
Model training
Integration:
Building system Rule based in .csv
Data processing
Data reduction
Abba Babakura et. al. /IRICT (2014) 1-11 6
The data is often splitted into training and testing set using cross validation technique. They are
generated as an output used to build up and test the model. The two arrows starting from this
component indicate where the data sets are used. The HMM and NB classifiers module
provides the model design’s full process including training models, based on the training data
set. Then, a complete interoperation system is constructed or implemented by putting the
trained models together, which is shown by the arrow connecting the Classifiers and
Classification components. Once the system is developed, the system performance is measured
by test data. The experimental results are analyzed in the Evaluation component. Since the
preparation of the classified events is a highly time-consuming and cost-expensive process,
such a logical mechanism might be viewed as a means of generating classified data (supervised
data) form unclassified data (unsupervised data).
3.1 Hidden Markov Models
HMMs are a powerful and robust tool for modeling stochastic random processes as they are
able to model a large variety of processes achieving high accuracy with relatively low model
complexity. They have been extensively used in a myriad of signal processing applications
during the last 20 years, mainly for fitting experimental data onto a parametric model which can
be used for real-time pattern recognition, and to make short-term predictions based on the
available prior knowledge [11].
A Hidden Markov Model is defined as a doubly embedded process with an underlying
stochastic process that is not observable. This hidden process (state) can only be evaluated
through another set of processes that produce sequences that actually can be observed [11]. An
HMM for discrete symbol observations is defined by the following elements:
A = {aij}, the state transition probability distribution.
aij = P(qi+1 = j | qt = i)
aij denotes the probability of being in state j at time t+1 given that it was in
state i at time t. it is assumed that aij are independent of time
B = {bj(Ot)}j=1N , the observation symbol probability distribution. Ot represents the
observation outcome at time t. bj(Ot ) is defined as:
bj(Ot ) = P{Ot = vk | qt = j} where
1 ; 1
, the initial state distribution.
{ i}, i = P{q1 = i}
Abba Babakura et. al. /IRICT (2014) 1-11 7
N, the number of states in the model. Although the states are hidden, for many
practical applications there is often some physical significance attached to the states or
to the set of states in the model. The set of states is denoted by:
S = {S1, S2, …,SN} where Si is state i, i {1,…,N} and the state at time t as qt. A fixed
state sequence can be represented by Q = {q1, q2,…,qT}.
M, the number of distinct observation symbols per state. The observation symbol
corresponds to the physical output of the system being modeled. Individual symbols are
denoted as V = {v1, v2,…,vM}.
Thus, a complete definition of an HMM involves: two model parameters (N and M), the
specification of the observations symbols and the specification of A, B and . In practice the
compact notation used for an HMM will be
λ = (A, B, )
Regarding the structure of the transition matrix A, HMMs can be classified into different
groups. In an ergodic or fully-connected HMM every state of the model can be reached from
any other state in the model after a finite number of steps. However, transitions between
HMM’s states are most frequently limited. An example of these models is the left-right or
Bakis HMM which has the property that as time increases the state index either increases or
stays the same.
3.2 Naïve Bayes Model
Naïve Bayes is based on Bayes’ theorem and an attribute independence assumption [12], [13].
Its competitive performance in classification is surprising, because the conditional
independence assumption on which it is based is rarely true in real world applications. A
Bayesian network consists of a structural model and a set of conditional probabilities. The
structural model is a directed acyclic graph in which nodes represent attributes and arcs
represents attribute dependencies. Attribute dependencies are quantified by conditional
probabilities for each node given its parents. Bayesian network are often used for classification
problems, in which a learner attempts to construct a classifier from a given set of training
instances with class labels. Naive Bayesian Classification (i.e. Simple Bayesian Classifier) is
the most known and used classification method. It is not only easy to implement on various
kinds of datasets, but also it is quite efficient. A naive Bayes classifier is a simple
probabilistic classifier based on applying Bayes' theorem with strong
(naive) independence assumptions. Bayesian classifiers adopt a supervised learning approach.
Naive Bayes classifiers can be trained very efficiently in a supervised learning setting. They
Abba Babakura et. al. /IRICT (2014) 1-11 8
have the ability to predict the probability that a given tuple belongs to a particular class [14].
The strength of Naïve Bayesian classifier, as a powerful probabilistic model has been proven
for solving classification tasks effectively [15]. For any given instance, X= (X1,X2……Xn ,)
where,
X1 : is the value of attribute X1,
P(C|X) is calculated by Bayesian classifier for all possible class values C and
predicts:
C* =argmaxc p(x|c)
As the class value for instance X.
Hence, estimating P(X|C) which is proportional to P(X|C) P(C) is the main step of a Bayesian
classifier. The two algorithms as defined mathematically are experimented to test for the
accuracy and response time in order to solve the problem of decision making among event
occurring in sequence from the subsystems in the smart home environment.
4 Experimental result and evaluation
To test the accuracy and response time of the developed models, we utilize the data set obtained
for some events occurring in the smart home environment as sampled. A single dataset is used
for the purpose of comparison between the two algorithms so as to check the accuracy and
response time. The description of the tests’ dataset is depicted below:
Table 1. Dataset showing occurrences of sequence of events
Events
#
Timing Sequence
<Time & Date>
Subsystems identifier
with status <ON & OFF>
Action occurrences
1 2012-12-18 01:28:39 6 <= Off=> Alarm
2 2012-12-18 01:28:40 7 <= On=> Video
3 2012-12-18 03:20:58 12 <= On=> Light
4 2012-12-18 03:22:18 24 <= Off=> Alarm
5 2012-12-18 03:55:52 24 <= On=> Audio
As seen from the table above, the data set has five features which are Date, Time, Unique
identifier (subsystems), status and action. This example above shows single sensors sequence
corresponds to the action with concrete date, time, unique identifier, status as well as the label
parameters (action). The preliminary fields are established by the data gathering setup. The
Abba Babakura et. al. /IRICT (2014) 1-11 9
field named action describes the relationship between systems with the events occurred and the
transition between them. In this study we utilize the use of MatLabTM which provides
convenience in performing this experiment. With the ability of also providing good result, it has
proven that it is a good tool for dealing with problems related to sequential data in the smart
home environment.
According to the 3-fold cross validation, which makes the splitting of the data into accurate sets
of training and testing shows that three pair of training and test sets was prepared which
depends on the selection of different group for a test set. We conducted the experiment using
Hidden Markov Model and Naïve Bayes method for the test’s dataset. The results for both
algorithms are shown as follows:
Table 2. Comparison between Hidden Markov Model and Naïve Bayes
HMM NB
Accuracy 95.7% 90.8%
Error rate 4.3% 9.2%
Response time 0.008ms 0.012ms
From table 2 above, the HMM and NB accuracy and response time results for each actions
performed are promising. The initial hypothesis that the subsystems events can be learned using
the HMM and NB algorithms as better options are verified by the obtained overall accuracies
and response times results. It is evident that the test results shows a high accuracies (95.7% and
90.8%), given the complexity of home data together with their respective service performed.
The results also prove better reliability based on the condition of no given structure to their
respective behavior. These contribute towards higher accuracies from the algorithms
deployment and also verify its robustness. Similarly, classification errors occurred during
HMM and NB selection, but there was no substantial transition based action performed. Hence,
the total error rates for HMM and NB are 4.3% and 9.2%. For the purpose of comparison, we
can see that HMM algorithm has higher accuracy and less response time as compared to NB,
which clearly shows that the HMM algorithm is more flexible and robust in terms of handling
sequential data in the smart home environment.
Abba Babakura et. al. /IRICT (2014) 1-11 10
5 Conclusion
In this paper, we have proposed new algorithms namely Hidden Markov Model and Naïve
Bayes model for decision making among event in the smart home as a potential solution to the
problem of handling sequential data occurring from subsystems. The techniques presented in
this paper shows robustness and efficiency in terms of handling sequential data. The test result
for the HMM algorithm shows 95.7% accuracy and 0.008ms response time. Whereas, the test
result for the NB algorithm shows accuracy of 90.8% and 0.012ms response time. The
Experimental results for both the algorithms manifest that, Hidden Markov Model performed
better than the Naïve Bayes for the smart home dataset. A direction for future work is to use
other techniques for feature selection, and study their effect on the prediction performance of
different algorithms.
ACKNOWLEDGEMENT
The authors would like to extend their utmost appreciation to the staffs of Faculty of Computer
Science and Information Technology (FCSIT, UPM) for their support and encouragement.
References
1. Wang, S., Xu, Z., Li, H., Hong J. and Shi, W. Z. “Investigation on intelligent building
standard communication protocols and application of IT technologies,” Automation in
Construction, vol. 13, no. 5, pp. 607-619, 2004.
2. Capezio, F., Giuni, A., Mastrogiovanni, F., Sgorbissa, A., Vernazza, P., Vernazza, T.,
and Zaccaria, R. “Sweet Home! Perspectives of Ambient Intelligence,” In Journal of
the Italian AEIT Association, pp. 42 – 49, 2007.
3. Geib C. and Goldman, R. “Partial Observability and Probabilistic Plan/Goal
Recognition,”
International Joint Conference on Artificial Intelligence, pp 418-425, 2005.
4. Rashidi, P., Cook, D. J., Holder L. B. and Schmitter-Edgecombe, M. “Discovering
Activities to Recognize and Track in a Smart Environment,” IEEE Transactions on
Knowledge and Data Engineering, 23(4), 527-539, 2011.
Abba Babakura et. al. /IRICT (2014) 1-11 11
5. Crandall, A. S. and Cook. D.J. “Using a Hidden Markov Model for resident
identification,” Intelligent Environments (IE), 2010 Sixth International Conference
on, pp. 74-79, 2010.
6. Perumal, T., Sulaiman, M. N. and Leong, C. Y. “ECA-based interoperability
framework for intelligent building,” Automation in Construction, 31, 274-280, 2013.
7. Freitag, D. and McCallum, A. “Information extraction with HMMs and shrinkage,”
In: Proceedings of the AAAI- 99 workshop on machine learning for information
extraction, 1999.
8. Freitag, D. and McCallum, A. “Information extraction with HMM structures learned
by stochastic optimization,” In: AAAI/IAAI 2000, pp. 584–589, 2000.
9. Cheng, B., Tsai, Y. and Liao, G. “HMM machine learning and inference for Activities
of Daily Living Recognition,” J Supercomputer, 54: 29–42, 2010.
10. Crandall, A. S. and Cook, D. Attributing events to individuals in multi-inhabitant
environments, 2008.
11. Rabiner, L. and Juang, B. H. Fundamentals of Speech Recognition, Prentice-Hall,
1993.
12. Duda R. O and Hart, P. E. Pattern Classification and Scene Analysis. New York: John
Wiley, 1973.
13. Lewis, D. D. Naive (Bayes) at forty: The independence assumption in information
retrieval. Machine Learning: ECML-98, Tenth European Conference on Machine
Learning, 4-15, 1998.
14. Batchu, V. and Aravindhar D. J. et al., A classification based dependent approach for
suppressing data, IJCA Proceedings on Wireless Information Networks & Business
Information System (WINBIS 2012), Foundation of Computer Science (FCS), 2011.
15. Geenen, P.L. and Van der Gaag L. C. et al., Constructing naive bayesian classifier for
veterinary medicine: A case study in the clinical diagnosis of classical swine fever,
Research in Veterinary Science, vol 91(1), 64–70, 2011.