Comparison of Hidden Markov Model and Naïve Bayes ... · Comparison of Hidden Markov Model and Naïve Bayes Algorithms among ... and Naive Bayes (NB). These two algorithms have the

1st International Conference of Recent Trends in Information and Communication Technologies

*Corresponding author: [email protected]

Comparison of Hidden Markov Model and Naïve Bayes Algorithms among

Events in Smart Home Environment

Abba Babakura*, Md Nasir Sulaiman, Norwati Mustapha, Khairul A. Kasmiran

Faculty of Computer Science and Information Technology, UPM Serdang, Malaysia

Abstract

The smart home environment consists of numerous subsystems which are heterogeneous in

nature. Smart home environment are configured in such a way that it comfort driven as well as

achieving optimized security and task-oriented without human intervention inside the home. The subsystems, due to their diversified nature develop difficulties as the events communicate

making the smart home uncomfortable. The complexity of decision making in handling events

stands at the bottleneck in ensuring various tasks executed jointly among diversified systems in smart home environment. In this paper, we propose Hidden Markov Model (HMM) and Naïve

Bayes (NB) to test the accuracy and response time of the home data and to compare between

the two algorithms. The result experimented shows that the HMM algorithm stands at higher

accuracy and better response time than the NB. The implementation has been carried out in

such a way that quality information is acquired among the systems to demonstrate the

effectiveness of decision making among events in the smart home environment.

Keywords: Smart home; HMM; Naïve Bayes; Decision making; Feature selection

1 Introduction

Generally, smart home is seen as an entity integrated with diversified service function of

automation, communication and control of its environment, and performing them in unified

manner via intelligent tasks [1]. In smart home, there is high interest and priority for low cost

solutions with high performance technologies consolidated together. These technologies

include the rise of high-speed communication structure and assured rapid increment of

diversified systems in home environment. In this context, many researchers try to develop

smart home technologies to provide assistance in handling the events in the environment by

utilizing machine learning algorithms. The identification of the ongoing inhabitant (events) is

IRICT 2014 Proceeding

12th -14th September, 2014, Universiti Teknologi Malaysia, Johor, Malaysia

Abba Babakura et. al. /IRICT (2014) 1-11 2

one of the main issues in smart homes. In most cases, the events are inferred with a plan context

that consists of basic activities predefined by an expert [2], [3]. The smart home environment

consists of numerous subsystems which are heterogeneous in nature. The subsystems, due to

their diversified nature develop difficulties as the events communicate making the smart home

uncomfortable. Heterogeneous systems in the smart home consists of the building automation

system (BAS), energy management system, fire alarm system, digital surveillance system and

other network based systems. Due to the high number of events and their complexity, the static

decision making algorithms could not handle the problem accurately and efficiently. Therefore,

another option is to build an automatic learning algorithm that can accurately and efficiently

handle or enumerate the sequence of events occurring from the subsystems in the smart home

environment. Symbolically, it is difficult to enumerate all the possible events occurring in a

home because of the large dataset. Nevertheless, new ways allow us to explore this paradigm

with supervised learning methods, which eliminate the requirement of a human intervention in

the learning process.

To do so, one avenue is to utilize the machine learning algorithms to handle the decision

making among the events and to accurately learn and adapt to changes as events are occurring

in sequence. We propose, in this paper, machine learning algorithms namely- Hidden Markov

Model (HMM) and Naïve Bayesian (NB) to address the problem of learning the sequence of

events in the smart home and to compare the accuracy and response time of the algorithms.

The paper begins with describing related works in Section 2. This is preceded with

methodological development elaborated in Section 3 and accompanied by experimental result

and evaluation in Section 4. The conclusion of the research is preceded in Section 5.

2 Related work

The necessity to ensure learning of events in other to make decision among subsystems in smart

home environment can be seen from diverse outlook. In recent times, literatures as well work-in

progress on smart home research suggested the importance of using the machine learning

algorithms in terms of decision making due to the fact that it provide effective and efficient

results. Available literatures on smart home research works highlighted the importance of

applying the HMM and NB in different tasks before achieving adaptability using decision

models. An important piece of work done by Rashidi et al., on automated approach for activity

tracking that naturally occurs in a smart home user lifestyle. Their work focuses on tracking the


incident of regular tasks with intention to monitor health and identifying changes in a user

patterns and lifestyle [4].

On the other hand, another important work introduces the use of passive sensors and a Hidden

Markov Model as a means to identify individuals [5]. The result is a passive, low profile means

to attribute individual events to unique residents. Here, HMM is deployed to compare against a

prior naïve Bayes solution on the same data sets. One significant work to focus on is the work

done by Perumal et al. [6]. In this study, Perumal et al. discussed the importance of providing a

decision support among heterogeneous systems using the ECA based interoperability

framework. The work also highlighted how overall interoperability is obtained using decision

models benchmarked using SOAP protocols. This shows that there is need to use the decision

making technique among heterogeneous systems. Furthermore, one important piece of work

done by Freitag et.al. discusses a method, called “shrinkage” that improves the sampling as

well as computing HMM architecture probabilities for training data [7], [8]. In this work, they

deployed an optimization procedure for feasible selection of HMM architecture corresponding

bespoke training data requirement for smart home. A significant work presented by Cheng et al.

provides a dominant inference engine based on HMM, known as ALHMM, combining the

Viterbi and Baum-Welch algorithms measuring accuracy purposes and learning ability

enhancement [9]. Another significant work was conducted by Aaron et al., in their study

attributing events to individuals in multi-inhabitant environments, where their aim is to identify

the individuals and their activities in an intelligent environment using passive sensors,

deploying the use of naïve Bayesian algorithm resulted in obtaining an average accuracy of

96% [10]. Finally, another significant work was conducted, using Hidden Markov Model for

resident identification in smart home environment [5]. In their course of study they aim to

recognize inhabitants or individuals correctly using the Hidden Markov Model technique

providing average accuracies of 94.0% and 90.2% using two different data sets.

Therefore the proposed model in this research is designed to overcome the disadvantage of

other used algorithms in making decision among subsystem events and to provide a clear

methodology by utilizing the two aforementioned algorithms HMM and NB, which has the

properties of stochastic and mathematical functions that makes it have more advantage over

other algorithms leading to higher accuracy and improved response time.


3 Framework Development

A first step to understand a phenomenon is to collect long term data (observations) then to build

a behavioral model of this system. The mathematical model can then be played with the added

flexibility to modify, adjust and play various scenarios. Here we propose two algorithms HMM

and NB, they are both a powerful and robust tools for stochastic processes as they lead to

effective and efficient build up of model achieving high accuracy with relatively low model

complexity. The two algorithms used in this study are namely: Hidden Markov Model (HMM)

and Naive Bayes (NB). These two algorithms have the flexibility of dealing with sequential

data which often makes them to be unique and draws attention of using them. The popularity of

this models and their ability to learn fast and adapt to changes in the occurrence of series of

data makes them to be scalable and reliable in the field of smart home. The methodological

framework can be seen in Figure 1 below. The framework design consists of four components

as displayed:

Model Data,

HMM and NB classifiers,

Automated Classification,

Prediction and System Evaluation.

It solely describes the role of the architecture to perform cooperative execution of tasks among

subsystems. The large arrows in between the phases indicate the system flow among

components. In the Model Data components, a set of data and rules are generated and stored in

the database. The rules are stored as raw data in .csv format in which a particular tool is used to

run it.


Figure 1: Framework Development

Phase 1: Model Data

Phase 3: Automated Classification

Test Data Set

HMM & NB-Based System

System Evaluation

Surveillance

subsystem

Audio subsystem Building

Automation

Energy

management

DATA SELECTION

Feature Selection

Training and Test Data Set

(Baum-Welch)

Phase 2: Classifiers (HMM &NB)

Training Data Set

System Implementation

Implementation:

Model training

Integration:

Building system Rule based in .csv

Data processing

Data reduction


The data is often splitted into training and testing set using cross validation technique. They are

generated as an output used to build up and test the model. The two arrows starting from this

component indicate where the data sets are used. The HMM and NB classifiers module

provides the model design’s full process including training models, based on the training data

set. Then, a complete interoperation system is constructed or implemented by putting the

trained models together, which is shown by the arrow connecting the Classifiers and

Classification components. Once the system is developed, the system performance is measured

by test data. The experimental results are analyzed in the Evaluation component. Since the

preparation of the classified events is a highly time-consuming and cost-expensive process,

such a logical mechanism might be viewed as a means of generating classified data (supervised

data) form unclassified data (unsupervised data).

3.1 Hidden Markov Models

HMMs are a powerful and robust tool for modeling stochastic random processes as they are

able to model a large variety of processes achieving high accuracy with relatively low model

complexity. They have been extensively used in a myriad of signal processing applications

during the last 20 years, mainly for fitting experimental data onto a parametric model which can

be used for real-time pattern recognition, and to make short-term predictions based on the

available prior knowledge [11].

A Hidden Markov Model is defined as a doubly embedded process with an underlying

stochastic process that is not observable. This hidden process (state) can only be evaluated

through another set of processes that produce sequences that actually can be observed [11]. An

HMM for discrete symbol observations is defined by the following elements:

A = {aij}, the state transition probability distribution.

aij = P(qi+1 = j | qt = i)

aij denotes the probability of being in state j at time t+1 given that it was in

state i at time t. it is assumed that aij are independent of time

B = {bj(Ot)}j=1N , the observation symbol probability distribution. Ot represents the

observation outcome at time t. bj(Ot ) is defined as:

bj(Ot ) = P{Ot = vk | qt = j} where

1 ; 1

, the initial state distribution.

{ i}, i = P{q1 = i}


N, the number of states in the model. Although the states are hidden, for many

practical applications there is often some physical significance attached to the states or

to the set of states in the model. The set of states is denoted by:

S = {S1, S2, …,SN} where Si is state i, i {1,…,N} and the state at time t as qt. A fixed

state sequence can be represented by Q = {q1, q2,…,qT}.

M, the number of distinct observation symbols per state. The observation symbol

corresponds to the physical output of the system being modeled. Individual symbols are

denoted as V = {v1, v2,…,vM}.

Thus, a complete definition of an HMM involves: two model parameters (N and M), the

specification of the observations symbols and the specification of A, B and . In practice the

compact notation used for an HMM will be

λ = (A, B, )

Regarding the structure of the transition matrix A, HMMs can be classified into different

groups. In an ergodic or fully-connected HMM every state of the model can be reached from

any other state in the model after a finite number of steps. However, transitions between

HMM’s states are most frequently limited. An example of these models is the left-right or

Bakis HMM which has the property that as time increases the state index either increases or

stays the same.

3.2 Naïve Bayes Model

Naïve Bayes is based on Bayes’ theorem and an attribute independence assumption [12], [13].

Its competitive performance in classification is surprising, because the conditional

independence assumption on which it is based is rarely true in real world applications. A

Bayesian network consists of a structural model and a set of conditional probabilities. The

structural model is a directed acyclic graph in which nodes represent attributes and arcs

represents attribute dependencies. Attribute dependencies are quantified by conditional

probabilities for each node given its parents. Bayesian network are often used for classification

problems, in which a learner attempts to construct a classifier from a given set of training

instances with class labels. Naive Bayesian Classification (i.e. Simple Bayesian Classifier) is

the most known and used classification method. It is not only easy to implement on various

kinds of datasets, but also it is quite efficient. A naive Bayes classifier is a simple

probabilistic classifier based on applying Bayes' theorem with strong

(naive) independence assumptions. Bayesian classifiers adopt a supervised learning approach.

Naive Bayes classifiers can be trained very efficiently in a supervised learning setting. They


have the ability to predict the probability that a given tuple belongs to a particular class [14].

The strength of Naïve Bayesian classifier, as a powerful probabilistic model has been proven

for solving classification tasks effectively [15]. For any given instance, X= (X1,X2……Xn ,)

where,

X1 : is the value of attribute X1,

P(C|X) is calculated by Bayesian classifier for all possible class values C and

predicts:

C* =argmaxc p(x|c)

As the class value for instance X.

Hence, estimating P(X|C) which is proportional to P(X|C) P(C) is the main step of a Bayesian

classifier. The two algorithms as defined mathematically are experimented to test for the

accuracy and response time in order to solve the problem of decision making among event

occurring in sequence from the subsystems in the smart home environment.

4 Experimental result and evaluation

To test the accuracy and response time of the developed models, we utilize the data set obtained

for some events occurring in the smart home environment as sampled. A single dataset is used

for the purpose of comparison between the two algorithms so as to check the accuracy and

response time. The description of the tests’ dataset is depicted below:

Table 1. Dataset showing occurrences of sequence of events

Events

#

Timing Sequence

<Time & Date>

Subsystems identifier

with status <ON & OFF>

Action occurrences

1 2012-12-18 01:28:39 6 <= Off=> Alarm

2 2012-12-18 01:28:40 7 <= On=> Video

3 2012-12-18 03:20:58 12 <= On=> Light

4 2012-12-18 03:22:18 24 <= Off=> Alarm

5 2012-12-18 03:55:52 24 <= On=> Audio

As seen from the table above, the data set has five features which are Date, Time, Unique

identifier (subsystems), status and action. This example above shows single sensors sequence

corresponds to the action with concrete date, time, unique identifier, status as well as the label

parameters (action). The preliminary fields are established by the data gathering setup. The


field named action describes the relationship between systems with the events occurred and the

transition between them. In this study we utilize the use of MatLabTM which provides

convenience in performing this experiment. With the ability of also providing good result, it has

proven that it is a good tool for dealing with problems related to sequential data in the smart

home environment.

According to the 3-fold cross validation, which makes the splitting of the data into accurate sets

of training and testing shows that three pair of training and test sets was prepared which

depends on the selection of different group for a test set. We conducted the experiment using

Hidden Markov Model and Naïve Bayes method for the test’s dataset. The results for both

algorithms are shown as follows:

Table 2. Comparison between Hidden Markov Model and Naïve Bayes

HMM NB

Accuracy 95.7% 90.8%

Error rate 4.3% 9.2%

Response time 0.008ms 0.012ms

From table 2 above, the HMM and NB accuracy and response time results for each actions

performed are promising. The initial hypothesis that the subsystems events can be learned using

the HMM and NB algorithms as better options are verified by the obtained overall accuracies

and response times results. It is evident that the test results shows a high accuracies (95.7% and

90.8%), given the complexity of home data together with their respective service performed.

The results also prove better reliability based on the condition of no given structure to their

respective behavior. These contribute towards higher accuracies from the algorithms

deployment and also verify its robustness. Similarly, classification errors occurred during

HMM and NB selection, but there was no substantial transition based action performed. Hence,

the total error rates for HMM and NB are 4.3% and 9.2%. For the purpose of comparison, we

can see that HMM algorithm has higher accuracy and less response time as compared to NB,

which clearly shows that the HMM algorithm is more flexible and robust in terms of handling

sequential data in the smart home environment.


5 Conclusion

In this paper, we have proposed new algorithms namely Hidden Markov Model and Naïve

Bayes model for decision making among event in the smart home as a potential solution to the

problem of handling sequential data occurring from subsystems. The techniques presented in

this paper shows robustness and efficiency in terms of handling sequential data. The test result

for the HMM algorithm shows 95.7% accuracy and 0.008ms response time. Whereas, the test

result for the NB algorithm shows accuracy of 90.8% and 0.012ms response time. The

Experimental results for both the algorithms manifest that, Hidden Markov Model performed

better than the Naïve Bayes for the smart home dataset. A direction for future work is to use

other techniques for feature selection, and study their effect on the prediction performance of

different algorithms.

ACKNOWLEDGEMENT

The authors would like to extend their utmost appreciation to the staffs of Faculty of Computer

Science and Information Technology (FCSIT, UPM) for their support and encouragement.

References

1. Wang, S., Xu, Z., Li, H., Hong J. and Shi, W. Z. “Investigation on intelligent building

standard communication protocols and application of IT technologies,” Automation in

Construction, vol. 13, no. 5, pp. 607-619, 2004.

2. Capezio, F., Giuni, A., Mastrogiovanni, F., Sgorbissa, A., Vernazza, P., Vernazza, T.,

and Zaccaria, R. “Sweet Home! Perspectives of Ambient Intelligence,” In Journal of

the Italian AEIT Association, pp. 42 – 49, 2007.

3. Geib C. and Goldman, R. “Partial Observability and Probabilistic Plan/Goal

Recognition,”

International Joint Conference on Artificial Intelligence, pp 418-425, 2005.

4. Rashidi, P., Cook, D. J., Holder L. B. and Schmitter-Edgecombe, M. “Discovering

Activities to Recognize and Track in a Smart Environment,” IEEE Transactions on

Knowledge and Data Engineering, 23(4), 527-539, 2011.


5. Crandall, A. S. and Cook. D.J. “Using a Hidden Markov Model for resident

identification,” Intelligent Environments (IE), 2010 Sixth International Conference

on, pp. 74-79, 2010.

6. Perumal, T., Sulaiman, M. N. and Leong, C. Y. “ECA-based interoperability

framework for intelligent building,” Automation in Construction, 31, 274-280, 2013.

7. Freitag, D. and McCallum, A. “Information extraction with HMMs and shrinkage,”

In: Proceedings of the AAAI- 99 workshop on machine learning for information

extraction, 1999.

8. Freitag, D. and McCallum, A. “Information extraction with HMM structures learned

by stochastic optimization,” In: AAAI/IAAI 2000, pp. 584–589, 2000.

9. Cheng, B., Tsai, Y. and Liao, G. “HMM machine learning and inference for Activities

of Daily Living Recognition,” J Supercomputer, 54: 29–42, 2010.

10. Crandall, A. S. and Cook, D. Attributing events to individuals in multi-inhabitant

environments, 2008.

11. Rabiner, L. and Juang, B. H. Fundamentals of Speech Recognition, Prentice-Hall,

1993.

12. Duda R. O and Hart, P. E. Pattern Classification and Scene Analysis. New York: John

Wiley, 1973.

13. Lewis, D. D. Naive (Bayes) at forty: The independence assumption in information

retrieval. Machine Learning: ECML-98, Tenth European Conference on Machine

Learning, 4-15, 1998.

14. Batchu, V. and Aravindhar D. J. et al., A classification based dependent approach for

suppressing data, IJCA Proceedings on Wireless Information Networks & Business

Information System (WINBIS 2012), Foundation of Computer Science (FCS), 2011.

15. Geenen, P.L. and Van der Gaag L. C. et al., Constructing naive bayesian classifier for

veterinary medicine: A case study in the clinical diagnosis of classical swine fever,

Research in Veterinary Science, vol 91(1), 64–70, 2011.

Documents

Comparison of Hidden Markov Model and Naïve Bayes ... · Comparison of Hidden Markov Model and Naïve Bayes Algorithms among ... and Naive Bayes (NB). These two algorithms have the