iJADE web-miner: an intelligent agent framework …gkmc.utah.edu/7910F/papers/IEEE TKDE iJade web miner for...iJADE Web-Miner: An Intelligent Agent Framework for Internet Shopping

iJADE Web-Miner: An Intelligent AgentFramework for Internet ShoppingRaymond S.T. Lee, Member, IEEE, and James N.K. Liu, Member, IEEE

Abstract—There is growing interest in using intelligent software agents for a variety of tasks, including navigating and retrieving

information from the Internet and from databases, online shopping activities, user authentication, negotiation for resources, and

decision making. This paper proposes an integrated framework for information retrieval and information filtering in the context of

Internet shopping. The work focuses on applying agent technology, together with Web mining technology, to automate a series of

product search and selection activities. It is based on a multiagent development platform, namely, iJADE (Intelligent Java Agent

Development Environment), which supports various e-commerce applications. The framework comprises an automatic facial

authentication utility and six other modules, namely, customer requirements definition, a requirement-fuzzification scheme, a fuzzy

agents-negotiation scheme, a fuzzy product-selection scheme, a product-defuzzification scheme, and a product-evaluation scheme. A

series of experiments were carried out and favorable results were produced in executing the framework. From an experimental point of

view, we used a database of 1,020 facial images that were obtained under various conditions of facial expression, viewing perspective

and size. An overall correct recognition rate of over 85 percent was attained. For the product selection test of our fuzzy shopper

system, an average matching rate of more than 81 percent was achieved.

Index Terms—iJADE Web miner, Web mining, visual data mining, e-commerce, intelligent agents.

�

1 INTRODUCTION

DUE to the rapid developments in e-commerce, rangingfrom C2C (Consumer-to-Consumer) e-commerce appli-

cations such as an e-auction to sophisticated B2B (Business-to-Business) e-commerce activities such as e-Supply ChainManagement (eSCM), the Internet is becoming a commonvirtual marketplace in which we can do business, search forinformation, and communicate with each other. Because ofthe ever-increasing amount of information in cyberspace,knowledge discovery and Web mining are becoming criticalfor successfully conducting business in the cyber world.

With the advance of PC computing technology in termsof its computational speed and its popularity for Webbrowsing, intelligent software applications known asagents—with their autonomous properties, automatic dele-gation of jobs, and highly mobile and adaptive behavior inthe Internet environment—are becoming a potential area ofdevelopment for e-business in the new millennium [18].

In a typical e-shopping scenario, there are two funda-mental areas of functionality where Web mining and visualdata mining might be able to help. The first area is customerauthentication. Traditional authentication is based on ausername and password transmitted using a secure trans-port protocol such as the Secure Socket Layer (SSL).Although this provides a secure user authenticationscheme, it requires proactive login from a customer beforeaccess rights are granted, which may discourage somecustomers from shopping. Other authentication schemes

that are based on digital certificates with smart cardtechnology [34], or biometric authentication techniquesbased on iris or palmprint recognition, might provide somefeasible alternatives. However, those approaches needspecial authentication equipment that limits their use inan e-commerce environment, not to mention the legalimplications of accessing personal and private data, such asiris and palmprint patterns. In contrast, automatic authen-tication based on human-face recognition can help toovercome these limitations. In terms of the visual proces-sing equipment required, a standard Web camera is alreadysufficient for facial pattern extraction and, nowadays, this iscommon equipment for Web browsing. This kind ofauthentication scheme can provide a truly automaticscheme in which a customer does not need to provide anyspecial identification information and, more importantly, itdoes not exploit any “confidential” or “sensitive” data, suchas fingerprints or iris patterns.

The second area of functionality is the automation of theonline shopping process via agent technology. Traditionalshopping models include consumer buying behavior mod-els, such as the Blackwell model [11], the Bettman model [6],and the Howard-Sheth model [16], which all share a similarlist of six fundamental stages in consumer buying behavior.The six stages are:

1. consumer requirements definition,2. product brokering,3. merchant brokering,4. negotiation,5. purchase and delivery, and6. after-sales service and evaluation.

The first three stages involve a wide range of uncertaintyand many possibilities, what we call “fuzziness,” rangingfrom the setting of purchasing criteria (and the supplying of

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004 461

. The authors are with the Department of Computing, Hong KongPolytechnic University, Hung Hom, Kowloon, Hong Kong, China.E-mail: {csstlee, csnkliu}@comp.polyu.edu.hk.

Manuscript received 13 Dec. 2000; revised 15 Aug. 2002; accepted 7 Feb.2003.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference IEEECS Log Number 113292.

1041-4347/04/$20.00 � 2004 IEEE Published by the IEEE Computer Society

products by a merchant) to selecting goods. So far, these areall “gray areas” that need to be thoroughly explored toapply agent technology to an e-commerce environment.

In this paper, we propose an integrated, intelligentagent-based framework, known as iJADE (pronounced as“IJ”)—the intelligent Java Agent-based Development En-vironment. The aim of this framework is to overcome thedeficiency of contemporary agent software platforms, suchas IBM Aglets [2] and ObjectSpace Voyager Agents [38],which mainly focus on multiagent mobility and commu-nication. iJADE provides an important layer, called the“Intelligent Layer,” that supports the implementation ofdifferent AI functionalities for multiagent applications.From an implementation point of view, we describe oneof the most important applications of iJADE in thee-commerce environment: iJADE Web miner. The iJADEWeb miner is an intelligent agent-based Web miningapplication that consists of two major modules: 1) Anagent-based facial-pattern authentication scheme, calledFAgent, which uses an innovative, visual data mining andvisualization scheme that is based on the EGDLM (ElasticGraph Dynamic Link Model) technique [22]. 2) A Webmining tool that is based on fuzzy shopping agents forproduct selection and brokering, called FShopper.

The rest of this paper is organized as follows: Section 2presents an overview of Web mining, which is an extensionof data mining to cyberspace. Section 3 gives a generaldescription of agent systems for e-commerce applications.Section 4 presents the model framework of iJADE and thetwo major components of iJADE Web miner: FAgent andFShopper. The implementation of the system is discussed inSection 5, which is followed by a brief conclusion.

2 WEB MINING—A PERSPECTIVE

As an important extension of data mining [12], Web miningis a technology that integrates various research fields,including computational linguistics, statistics, informatics,artificial intelligence (AI), and knowledge discovery [39]. Itcan also be interpreted as the discovery and analysis ofuseful information from the Web. Although there is nodefinite principle in a Web mining model, Web mining canbe categorized into two main areas: content mining andstructural mining [9], [39]. A taxonomy of Web mining isdepicted in Fig. 1.

Content mining focuses on the extraction of Web contentand knowledge discovery (mining) from that information,ranging from HTML and XML documents found in Webservers to data sources (e.g., databases) attached to theback-end of Web systems. Structural mining focuses onknowledge discovery from the structure of the Websystems. This includes the mining of user preferences whilethey browse the Web (Web usage mining), examining theusage of different URLs in a particular Web site (URLmining), mining external structure (hyperlinks betweendifferent Web pages), and mining internal structure(hyperlinks within a particular Web page). Active researchincludes Spertus [36] on mining internal structure andPitkow [33] on mining Web usage data.

In content mining, popular search engines (such asLycos, WebCrawler, Infoseek, and Alta Vista) provide some

Web searching functionality. However, they fail to provideconcrete and structural information [27]. In recent years,interest has been focused on how to provide a higher levelof organization for semistructured (and even unstructured)data on the Web using Web mining techniques. Within thistopic, there are two main areas of research: the databaseapproach and the agent-based approach. The databaseapproach to Web mining tries to focus on techniques fororganizing the semistructured and unstructured data (onthe Web) into more structured information and associatedresources, so that traditional query tools and data miningcan be applied for data analysis and knowledge discovery.Typical examples can be found in the ARANEUS system[3], which uses a multilevel database for Web mining, andChe et al. [8] in which a complex Web query system is usedfor Web mining of G-Protein Coupled Receptors (GPCRs).The agent-based approach focuses on the provision of“intelligent” and “autonomous” Web mining tools that arebased on agent technology. Typical examples can be foundin the FAQFinder system [14] for intelligent search enginesand the Firefly system [35] for personalized Web agents.

3 AGENT TECHNOLOGY IN E-COMMERCE

3.1 Background

The current World Wide Web system is catalyzing thedevelopment of e-commerce using the Internet. The currentInternet commerce systems are primarily based on theclient-server architecture. All transactions are carried out bymany request/response interactions over the Internet. Asthe Internet is a best-effort network, sometimes a user mayexperience a long response time. Another approach is to usea mobile agent-based system. This involves sending amobile software agent to a remote system using varioustechnologies (such as IBM Aglets [2], ObjectSpace VoyagerAgents [38], the FTP Software Agent [13], the General MagicOdyssey Agent System [32], and the Agent BuilderEnvironment from IBM [1]). The agent can conduct multipleinteractions with the software resident on a remote system.The output of the interactions is then sent back to the user.An agent can also interact with other agents on the Internet

462 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 16, NO. 4, APRIL 2004

Fig. 1. A taxonomy of Web mining.

before returning to the originating system. It is expectedthat this type of agent-based system will complement theexisting client-server Internet commerce systems by provid-ing more advanced services, including agent cloningservices, multiagent collaboration, and messaging services.Currently, there are many different e-commerce systems inuse around the world, ranging from simple online shops tomore complex systems that provide different types ofservices. Some examples include the following:

. BargainFinder—a database search engine for search-ing online music stores [5],

. AuctionBot—a generic auction server that allowssuppliers to auction products [4],

. Metabroker—a generic framework for creating elec-tronic brokers [7], and

. MAGNET—a system for networked electronictrading [10].

3.2 Advantages of Using Mobile AgentTechnologies for E-Commerce Applications

3.2.1 Reduce Network Load

Distributed systems often rely on communications protocolsthat involve multiple interactions to accomplish a given task.Mobile agents allow one to package and dispatch tasks to adestination host where the interactions can take place locally.

3.2.2 Overcome Network Latency

E-commerce applications often involve real-time interac-tions, such as online shopping and e-auctions. They all needto respond in real time to changes in their environment.Controlling such systems through a factory network of asubstantial size involves significant latencies, which is notacceptable for these real-time operations. Mobile agentsoffer a solution because they can be dispatched from acentral controller to act locally and to directly execute acontroller’s directions.

3.2.3 Execute Asynchronously and Autonomously

Very often, mobile devices must rely on expensive or fragilenetwork connections. Tasks that require a continuouslyopen connection between a mobile device and a fixednetwork will probably not be economically or technicallyfeasible. Using mobile agent technologies, tasks can beembedded into mobile agents, which can then be dis-patched into a network and operated asynchronously andautonomously.

3.2.4 Adapt Dynamically

One major characteristic of using agent technologies is theability to “sense” the execution environment (e.g., themarketplace in shopping operations) and react autono-mously to changes. Multiple mobile agents possess a uniqueability to distribute themselves among the hosts in a networkso that they can maintain an optimal configuration for anye-commerce operations, such as “e-window-shopping.”

3.2.5 Naturally Heterogeneous

Network computing is fundamentally heterogeneous,often from the perspective of both the hardware andsoftware. Because mobile agents are generally computer

and transport-layer independent, they depend only ontheir execution environment. Thus, they possess optimalcapabilities for seamless system integration, especiallyunder circumstances where other efforts have failed.

3.2.6 Robust and Fault-Tolerant

Mobile agents react dynamically to unfavorable situationsand events, making it easier to build robust and fault-tolerant distributed systems. This is a vital criterion for anye-commerce application being implemented in the Internetenvironment.

3.3 Shortcomings of Current Agent-Based Systems

Although contemporary agent-based systems provide aneffective framework for the dispatching, communication,and management of multiple mobile agents in the Internetenvironment, from the point of view of the AI, there is alack of support for intelligent functionality in these systems.For instance, IBM Aglets [2] provide comprehensive APIsfor mobile agent, which support creating, dispatching,cloning, retracting, and disposing of aglets using theAgletContext class. They also support aglet messaging andcollaboration using the Aglet Message classes and Mobilityadapters. However, intelligent capabilities—such as frame-based learning and data mining at a macroscopic level, orneural-network modeling, fuzzy, and genetic learning at adetailed level—have not been adopted. This makes thedevelopment of AI-based agent applications more difficult.

4 IJADE ARCHITECTURE

4.1 iJADE Framework

The iJADE system framework is shown in Fig. 2. Unlikecontemporary agent systems and APIs such as IBM Aglets[2] and ObjectSpace Voyager [38], which focus on themultiagent communication and autonomous operations, theaim of iJADE is to provide comprehensive APIs for“intelligent” agents and applications for future e-commerceand Web mining applications.

Fig. 2 depicts the two levels of abstraction that are usedin the iJADE system: a) the iJADE system level—AITSmo-del, and b) the iJADE data level—DNA model. The AITSmodel consists of

1. the Application Layer,2. the Intelligent Layer,3. the Technology Layer, and the4. Supporting Layer.

The DNA model is composed of 1) the Data Layer, 2) theNeural-Network Layer, and 3) the Application Layer.

Compared with contemporary agent systems, whichprovide minimal and elementary data managementschemes, the iJADE DNA model provides a comprehensivedata manipulation framework that is based on neural-network technology. The “Data Layer” corresponds to theraw data and input “stimuli” (such as the facial imagescaptured from a Web camera and the product informationin a cyber store) from the environment. The “Neural-Network Layer” provides the clustering of different types ofneural networks for the purpose of organization, inter-pretation, analysis, and forecasting operations that are

LEE AND LIU: IJADE WEB-MINER: AN INTELLIGENT AGENT FRAMEWORK FOR INTERNET SHOPPING 463

based on the inputs from the “Data Layer.” The neural

networks are used by the iJADE applications in the

“Application Layer.”

4.2 Application Layer including the iJADE WebMiner

TheApplication Layer is the uppermost layer,which consists

of different intelligent agent-based applications. These

iJADE applications are developed through the integration

of intelligent-agent components from the “Intelligent Layer”

and the data “knowledge fields” from the DNA model.Current applications (iJADE v1.6) implemented in this

layer include the following:

. iJADE Stock Advisor, which is an intelligent agent-based stock prediction system using the time seriesneuro-oscillatory prediction technique [20],

. iJADE Web miner, the intelligent Web mining agentsystem that is discussed in this paper, and

. iJADE WeatherMAN, which is an intelligent weath-er forecasting agent that is the extension of previousresearch on multistation weather forecasting usingfuzzy neural networks [28].

4.3 Intelligent Layer

This layer provides the AI capabilities of the iJADE system

using the agent components provided by the “Technology

Layer.” The “Intelligent Layer” consists of the following

three main functional areas:

1. “Sensory Area”—for the recognition and interpreta-tion of incoming stimuli. It includes a) the visualsensory agents using the EGDLM (Elastic GraphDynamic Link Model) for invariant visual objectrecognition [24], [25], [26], and b) the auditory agents

based on a wavelet-based feature extraction andinterpretation technique [15].

2. “Logical Reasoning Area”—an area providing dif-ferent AI tools for logical “thinking” and rule-basedreasoning, such as fuzzy and GA (Genetic Algo-rithms) rule-based systems [23].

3. “Analytical Area”—consists of various AI tools foranalytical calculations, such as recurrent neural-network-based analysis for real-time prediction anddata mining [21].

4.4 Technology Layer Using IBM Aglets and JavaServlets

This layer provides all of the APIs that are necessary for

mobile agent implementation and for the development of

intelligent agent components in the “Intelligent Layer.”In the current version (v1.6) of the iJADE model, IBM

Aglets [2] are used as the agent “backbone.” The main

function of the AgletProxy class is to provide a handle that is

used to access the aglet. It also provides location transpar-

ency by forwarding requests to remote hosts and returning

results to the local host. Actually, all communication with

an aglet occurs through its aglet proxy. The AgletContext

class provides the runtime execution environment for aglets

within an agent server. Thus, when an aglet is dispatched to

a remote site, it is detached from the current AgletContext

object, serialized into a message bytestream, sent across the

network, and reconstructed in a new AgletContext, which in

turn provides the execution environment at the remote site.

The other critical component of the Aglet environment is

security. Aglets provide a security model in the form of an

AgletSecurityManager, which is a subclass of the “standard”

Java SecurityManager.


Fig. 2. System architecture of the iJADE (v 1.6) model (http://www.iJADK.com).

4.5 Supporting Layer

This layer provides all of the necessary system support forthe “Technology Layer.” This includes:

1. programming language support based on Java,2. support for network protocols such as HTTP

(Hypertext Transfer Protocol), HTTPS (secureHTTP), ATP (Asynchronous Transfer Protocol),etc., and

3. support for markup languages such as HTML(Hypertext Markup Language), XML (ExtensibleMarkup Language), and WML (Wireless MarkupLanguage).

5 IMPLEMENTATION

The iJADE Web miner consists of two main modules: amodule for visual data mining for intelligent user authenti-cation based on the FAgent (Face-recognition Agent) and amodule for Web product mining using a neuro-fuzzyshopping agent based on the FShopper (Fuzzy Shopper).

5.1 Intelligent Visual Data Mining for UserAuthentication Using the FAgent

5.1.1 FAgent: System Overview

In this section, we present FAgent (Face-recognition Agent),which is an automatic authentication scheme using a mobileagent-based, human-face-recognition system. Technically,the proposed system involves the integration of twocontemporary technologies: First, mobile agent technologyto facilitate mobile, heterogeneous, and fault-tolerantfunctionality in an e-commerce environment. Second, theElastic Graph Dynamic Link Model to perform neural-network-based face recognition using an elastic graphmatching technique, which has proven results in variousproblem domains ranging from human-face recognition[24] and scene analysis [26] to sophisticated tropical cyclonerecognition [25].

5.1.2 Invariant Face Recognition Using the Elastic

Graph Dynamic Link Model

Object recognition using the EGDLM is based on theframework of the Dynamic Link Architecture (DLA) [31],which describes the visual recognition problem as an elasticgraph matching mechanism between the attribute graphs ofthe image vectors (input layer) and a set of “memory”graphs from an object gallery (memory layer). The neuralinteractions are governed by the onset and offset of thedynamic links between the memory and input layers, whichsimulate the functionality of dynamic memory associationin short-term memory. Active research in this area includeshandwritten character recognition [29], [30] and roboticgesture recognition [37].

Unlike the traditional DLA model, the EGDLM inte-grates with the Active Snake Model [17] for the extraction ofthe facial contours. Sophisticated facial matching becomes asimplified elastic graph-matching problem. To enhance therecognition rate in terms of speed and accuracy, featurevectors are extracted from the “facial landmarks” of thefacial image.

One of the most striking features of the EGDLM is its“invariant” property. In the network model, only thetopological relations between the feature vectors areencoded into the network. The pattern matching processresembles “elastic graph matching” [31], which is invariantunder various transformations, such as translation, rotation,reflection, dilation, and occlusion. These transformationscommonly occur in pattern recognition problems such ashuman face recognition and scene analysis.

5.1.3 FAgent System—System Framework

The system architecture of the FAgent mainly consists oftwo subsystems, one at the client (i.e., customer) site andthe other at the server (e.g., a virtual shopping mall) site.A schematic diagram of the whole FAgent system isshown in Fig. 3.

In summary, the three kinds of intelligent agents operat-ing within the FAgent system are as follows:

1. FAgent Feature Extractor—a stationary agent at aclient machine that is used to extract facial featuresfrom a facial image of the client captured using theclient’s digital camera.

2. FAgent Messenger—a mobile agent that acts as a“messenger” and carries the facial image to theserver-side agent and reports back the latest status tothe client machine.

3. FAgent Recognizer—a stationary agent located atthe server. Its main duty is to perform invariant-facial pattern matching against the server-side facialdatabase.

The client-side subsystem consists of the following threeoperations:

1. Facial image capture using the client’s desktop videocamera.

2. Facial contour extraction using the Active ContourModel (ACM) [17].

3. Automatic facial landmark extraction using a Gaborfeature extractor [24].

5.1.4 Facial Contour Extraction—the Active Contour

Model (ACM)

The Active Contour Model involves the use of a “snake”[17] to locate the facial contours. The “snake” is acontinuous curve that forms an initial state (facial template)and then tries to deform itself dynamically on the image.External forces attract the snake toward image features, andinternal forces maintain the smoothness of the snake’sshape (Fig. 4). The sum of the membrane energy, denotingthe snake stretching, and the thin-plate energy, denoting thesnake bending, gives the following snake energy [17]:

EintðuðsÞÞ ¼ �ðsÞ usj ðsÞj2 þ �ðsÞ ussj ðsÞj2; ð1Þ

where uðsÞ ¼ ðxðsÞ; yðsÞÞ is the snake curve and s is the arc-length of the curve and ussðsÞ denotes the second orderderivatives of the snake curve [17]. The parameters ofelasticity � and � control the smoothness of the snake’scurve. This energy function is composed of the first-orderterm controlled by �ðsÞ and the second-order termcontrolled by �ðsÞ. The first-order term makes the snakeact like a membrane and the second-order term makes it act


like a thin plate. Adjusting the weights of these two

components controls the relative importance of the mem-

brane and thin-plate terms.The deformation of the “snake” is governed by the

external forces. These forces are associated with a potential

P ðx; yÞ, which is generally defined in terms of the gradient

module of the image convoluted by a Gaussian function:

P ðx; yÞ ¼ � rðGðx; yÞ � Iðx; yÞÞjj ð2Þ

or as a distance map of the edge points:

P ðs; yÞ ¼ dðx; yÞ; P ðx; yÞ ¼ �e�dðx;yÞ2 ; ð3Þ

where dðx; yÞ denotes the distance between a pixel ðx; yÞ andits closest edge point. The snake is moved by the potential

forces and tries to fall into a valley as if it were under the

effect of gravity.The total snake energy is given by:

Esnake ¼Z10

Eint þEext ds

¼Z10

�ðsÞ usj ðsÞj2 þ �ðsÞ ussj ðsÞj2 þ P ðuðsÞÞds:

ð4Þ

The minimum of the snake energy satisfies an Euler-

Lagrange equation:

� d

dsð�usðsÞÞ þ d2

ds2ð�ussðsÞÞ þ rP ðuðsÞÞ ¼ 0 ð5Þ

and boundary conditions.

5.1.5 Automatic Facial Landmarks Extraction Scheme

A total of 120 feature vectors of different attributes were

extracted automatically from 50 facial landmark positions

(e.g., noses, eyes, eyebrows, mouth, facial contours, etc.)

(Fig. 5). Gabor filters of 15 different frequency bands ð�Þand eight different orientations ð�Þ were used. The filter

function is given as follows [31]:

g�;�ðx; yÞ ¼1

�ffiffiffi�

p e �x2þy2

2�2

� �e2�i �ðx cos �þy sin �Þ: ð6Þ


Fig. 3. Schematic diagram of the FAgent system.

Fig. 4. Facial contour extraction using the ACM.

5.1.6 Server-Side Subsystem

The server-side subsystem, basically, consists of the

following modules:

1. The Dynamic Links Initialization scheme.2. The elastic attribute graph matching scheme (be-

tween the query image and the images from thefacial database).

In the Dynamic Link Initialization process, dynamic links

ðzij;klÞ are initialized between “memory” facial attribute

graphs and figure objects from the image gallery according

to the following rules:

zij;kl ¼ "JijJkl for Jij 2 A; Jkl 2 B; ð7Þ

where the Jij and Jkl are the feature vectors extracted from

the facial landmarks, zij;kl are the dynamic link values, " is a

parameter value between 0 and 1, and A and B denote the

figure graph and memory graph, respectively.In the elastic attribute graph matching module, the

attribute graph of the figure is “dynamically” matched with

each “memory” object attribute graph by minimizing the

following energy function:

HðzÞ ¼ �X

i;j2B;k;l2Azijzjl zik zkl þ �

Xi2B

Xk2A

zik � 1

!2

þ �Xk2A

Xi2B

zik � 1

!2

ð8Þ

within tolerance level �.HðzÞ is minimized using gradient descent

zijðtþ 1Þ ¼ zijðtÞ � @HðzðtÞÞ@zijðtÞ

� �w; ð9Þ

where ½. . .�w denotes the value of zij confined to the interval

½0; w�. At equilibrium (within a chosen tolerance level �),

HðzÞ will be minimized and the connection pattern in the

memory layer represents the pattern recalled by the figure

pattern.

5.2 Neural-Fuzzy Agent-Based Shopping Using theFShopper

The system framework of the FShopper consists of the

following six main modules:

1. Customer requirements definition (CRD).

2. Requirement-fuzzification scheme (RFS).3. Fuzzy agents-negotiation scheme (FANS).4. Fuzzy product-selection scheme (FPSS).5. Product-defuzzification scheme (PDS).6. Product-evaluation scheme (PES).7. A schematic diagram of the fuzzy shopper, FShop-

per, is shown in Fig. 6.

5.2.1 Customer Requirements Definition (CRD)

As shown in Fig. 6, at the client machine, there are twotypes of iJADE FShopper agents:

. FShopping Broker—a stationary agent that acts as abroker for a customer. This is an autonomoussoftware agent that contains all of the necessaryinformation (e.g., e-form) and analytical techniquesto act as a broker, such as the requirement forfuzzification and defuzzication and product evalua-tion techniques.

. Fuzzy Buyer—a mobile agent that acts as a virtualbuyer in the virtual marketplace. It performs allagent communication, interaction, and negotiationoperations.

In the CRD module, the customer (via a browser) isprovided (by the FShopping Broker) with an electronic form(e-form) to specify his or her requirements. The FShoppingBroker also provides a set of “cues” to assist in productdefinition.

5.2.2 Requirement-Fuzzification Scheme (RFS)

Once a customer has input his or her product requirements(e.g., color, size, style, fit), the FShopping Broker convertsthese (fuzzy) requirements into fuzzy variables by using“embedded” knowledge (i.e., the membership functions)that is stored in its knowledge base. The FShopping Broker isalso responsible for validating the data in the e-form. Samplefuzzymembership functions for selected attributes for shoes,including color and degree of fit, are shown in Fig. 7.

5.2.3 Fuzzy Agents-Negotiation Scheme (FANS)

After the (fuzzy) requirements of the customer are collected,a Fuzzy Buyer starts its buying activity in the cyber world.To speed up the buying process, a Fuzzy Buyer makes useof the agent cloning capability to create duplicates so thatparallel buying activities can be executed.

In each relevant Virtual Shopping Mall (VSM), a FuzzyBuyer communicates and negotiates with a Fuzzy Seller,which is a stationary (selling) agent that acts as a virtualsalesman in relation to the shopping activities.

5.2.4 Fuzzy Product-Selection Scheme (FPSS)

Once a Fuzzy Seller has collected all of the customer (fuzzy)requirements, it performs the product selection activitiesbased on a fuzzy neural network. The fuzzy neural networkis an integration of fuzzy technology and a Feed-ForwardBack-Propagation (FFBP) neural network, as shown in Fig. 8.

Fig. 8 illustrates the FPSS using a fuzzy neural networkfor product selection (e.g., selection of a pair of shoes). Thefuzzy neural network consists of two parts: the fuzzymodule and the Feed-Forward Back-Propagation (FFBP)neural network module. The fuzzy module provides the


Fig. 5. Facial features extraction scheme.

network with a bundle of fuzzy variables as the inputnodes. In the example, the fuzzy variables consist of colorcomponents (i.e., red, yellow, and blue), size, length, degreeof fit, and price.

The FFBP neural network is responsible for the productselection, which is a kind of pattern classification schemeusing amultilayer neural network. The output layer is the listof items that are available in the store (under the appropriateproduct category).

To proceed with product classification, the fuzzy FFBP

needs to be trained beforehand. Implementation details on

network training are discussedwith the experimental results.

5.2.5 Product-Defuzzification Scheme (PDS) and

Product-Evaluation Schemes (PES)

Depending on a customer’s preferences, a Fuzzy Buyer willreturn information about a number of recommendedproducts to a client machine. Before these products aredisplayed in a client browser, two operations have to beperformed: the Product-Defuzzification Scheme (PDS) andthe Product-Evaluation Scheme (PES).

In the PDS, each recommended product undergoes adefuzzification process to return the fuzzy descriptions forall of the related product attributes (for ease of under-standing for a user). In addition, according to a user’s


Fig. 7. Sample membership functions for color and degree of fit.

Fig. 6. Schematic diagram of the FShopper.

predefined ranking of the “importance” of every product

attribute, an objective evaluation scheme can be used to

rank all of the recommended products, which can then be

displayed in a customer’s Web browser so that they can

make the final purchasing decision.

6 EXPERIMENTAL RESULTS

6.1 FAgent Test

In our experiment, images of 100 human subjects were

used to train the system. We tested the system using a set

of 1,020 patterns resulting from different facial expres-

sions, viewing perspectives, and various sizes of the stored

templates. A series of tested facial patterns were obtained

using a CCD camera, which provided a standard video

signal and digitized images at 512� 384 pixels with 8 bits

of resolution.

6.1.1 FAgent Test I: Training Image Retrieval Test

By using the 100 facial images for the ACM contour

extraction, followed by the EGDLM network coding

mechanism, the model was tested against these training

images to determine its image retrieval capability. With the

number of iterations for the ACM modeling ranging from 0to 1,000, the retrieval accuracy is shown in Fig. 9.

As shown in Fig. 9, the pattern retrieval accuracy issaturated when the number of iterations used in the ActiveContour Model goes beyond 300. The overall retrievalaccuracy is promising, with an average correct recognitionrate of over 96 percent for the 100 training patterns.

6.1.2 FAgent Test II: Facial Pattern Illumination Test

In the illumination test, we used 100 test patterns of variousdegree of brightness for facial recognition, with the degreeof brightness varying from +30 percent to -30 percent of the“optimum” brightness level. The experimental results areshown in Table 1.

The overall correct recognition rates range from 79 to95 percent under the varying conditions.

6.1.3 FAgent Test III: Viewing Perspective Test

In this test,weused aviewingperspective ranging from -30 toþ30 degrees (with reference to the horizontal and verticalaxis), with 100 test patterns for each viewing perspective. Therecognition results are presented in Table 2.

According to the “rotation invariant” property of theelastic graph implemented in the EGDLM model [24], [26],


Fig. 8. Fuzzy neural network for product selection.

Fig. 9. Stored pattern retrieval test.

TABLE 1Results for the Illumination Test

the FAgent possesses the same characteristic in the “contourmaps elastic graph matching” process. The overall correctrecognition rates range between 82 percent and 92 percent.

6.1.4 FAgent IV: Facial Pattern Dilation / Contraction

Test

In this test, we used 300 test patterns, with size ratio rangingfrom -30 percent (pattern contraction) to +30 percent(pattern dilation). We also tested “partial” dilation andcontraction. The recognition results are shown in Table 3.

Because of the “elastic graph matching” characteristic ofthe Elastic Graph Dynamic Link Model, the system alsopossesses the “dilation and contraction invariance.” Thisproperty is similar to the “dilation invariance” of the Chinesecharacters investigated in [29] and [30]. The overall correctrecognition rates range between 79 percent and 92 percent.

6.1.5 FAgent Test V: The Facial Pattern Occlusion and

Distortion Test

In this test, we divided the 120 test patterns into threecategories (Fig. 10):

. Wearing spectacles or other accessories,

. Partial occlusion of the face by obstacles such as cupsand books (in the reading and drinking processes),and

. Various facial expressions (such as laughing, angry,and gimmicky faces).

The pattern recognition results are shown in Table 4.Among the three different categories of facial occlusion,

“wearing spectacles” had the least negative affect on facialrecognition. This is due to the fact that all the main facialcontours are preserved in this situation. In the second typeof occlusion, the influence on the recognition rate dependson the proportion and portion of the face that is obscured.Nevertheless, the lowest correct recognition rate (of thethree categories) is 72 percent.

For the different categories of occlusion, facial expres-sions and gimmicky faces create the most difficult recogni-tion tasks. However, in our test, both these recognition tasksproduce promising results. Due to the “elastic graph”characteristic of our model, the recognition engine “inher-its” the “distortion invariance” property. The overall correctrecognition rates are between 72 percent and 87 percentunder the three different types of occlusion.


TABLE 2Results for the Viewing Perspective Test

TABLE 3Results of the Facial Pattern Dilation and Contration Test

Fig. 10. Occluded facial patterns.

6.2 The FShopper Test

In the product database, we used more than 200 items undereight categories to construct an electronic catalog (e-catalog).These categories were: T-shirts, shirts, shoes, trousers, skirts,sweaters, tablecloths, and napkins. We deliberately chosesoft-good items instead of hard-goods, such as books or CDs(as commonly found in most e-shopping agent systems), toallow more room for the definition of fuzzy user require-ments and product selection. For neural network training, allof the e-catalog items were “pretrained” in the sense that theattribute descriptions were predefined for all of the itemsthat were “fed” into a fuzzy neural network (for eachcategory of product) for product training. Thus, in total, eightdifferent neural networks were constructed, one for eachcategory of products.

From an experimental point of view, two sets of testswere conducted: the Round Trip Time (RTT) test and theProduct Selection (PS) test. The RTT test was aimed at theevaluation of the “efficiency” of the FShopper in the sensethat it involved calculating the complete round-trip time ofthe agents instead of calculating the difference between thearrival and departure times at the server. The RTT testcalculated the entire time based on the time associated witheach “component” of the system. This started at thecollection of user requirements, through the fuzzificationprocess to the product selection and evaluation steps, sothat a total picture of the efficiency of the system’sperformance could be determined. In the PS test, sincethere was no definite answer as to whether a product would“fit” the taste of a customer, a sample group of 40 candidates

were used to judge the “effectiveness” of the FShopper. Thedetails of these tests are illustrated in the following sections.

6.2.1 FShopper Test I: The Round Trip Time (RTT) Test

In this test, two iJADE servers were used: T1server andT2server. T1server was situated within the same LAN as theclient machine, whereas the T2server was in a remote site(but still within the campus). The results of the mean RTTafter 100 trials for each server are shown in Table 5.

As shown in Table 5, the total RTT is dominated by theFuzzy Product Selection Scheme, but the time spent is stillwithin an acceptable timeframe of five to seven seconds.The difference in RTT between the server situated withinthe same LAN as the client and the server at the remote siteis not significant except for the Fuzzy Agents-negotiationScheme (FANS). The Fuzzy Buyer needs to take a slightlylonger “trip” than for a server at the remote site. Of course,in reality, this depends mainly on the network traffic.

6.2.2 FShopper Test II: The Product Selection (PS) Test

Unlike the RTT test, in which objective figures can beobtained easily, the PS test results rely completely on userpreference. To get a more objective result, a sample group of40 candidates were invited to evaluate the system. In thistest, each candidate “bought” one product from eachcategory according to his or her own requirements. In theevaluation, the candidates browsed through the e-catalogand chose a list of their “best five choices” (L), which bestmatched their tastes. This list was compared with the topfive products that were recommended by the fuzzyshopper. To evaluate the effectiveness of the fuzzyshopper’s recommendation, the “Fitness Value” (FV) iscalculated as follows:

FV ¼

P5n¼1

n� i

15where i ¼ 1 if i 2 L

0 otherwise:

�ð10Þ

In the above calculation, scores from 5 to 1 (from the firstto the last ranked choice) are given to each of the correctmatches between a candidate’s list and the recommendationsfrom the FShopper. For example, if the same products thatare ranked 1, 2, 3, and 5 appear in the fuzzy shopper


TABLE 4Recognition Results of the Occlusion and Distortion Test

TABLE 5Mean RTT Summary after 100 Trials

recommendation, then the fitness value is 73 percent (i.e., the

sum of 1, 2, 3, and 5 divided by 15). The results under theeight different categories of products are shown in Table 6.

It is obvious that the performance of the FShopper is

highly dependent on the “variability” (or “fuzziness”) that

is associated with a description of the merchandise. The

higher the fuzziness (which means more variety), the lower

the fitness value (score). As shown in Table 6, the two

product categories of skirts and shoes are typical examples

in which skirts score 65 percent and shoes score 89 percent.

Nevertheless, the average score for the system is still over

81 percent. Note that these figures are only for illustration

since human justification and product variety in real

scenarios will vary case by case.

7 CONCLUSION

This paper proposes an innovative, intelligent agent-basedWeb mining application: the iJADE Web miner. This isbased on the integration of neuro-fuzzy-based Web mining

technology and intelligent visual data mining technology toautomate user authentication. We hope that it will provide anew era of Web-based data mining and knowledge

discovery using intelligent agent-based systems.The major contributions of this paper can be summarized

in two areas: research and application. From a researchpoint of view, this paper demonstrates a feasible andefficient solution (iJADE Web miner) for major aspects of

Web mining: 1) automatic human-face identification andrecognition and 2) interactive and mobile agent-basedproduct search from a large database. With respect to

human-face recognition, this paper proposes an automaticelastic graph matching model for human-face “contour”extraction, identification, and face recognition. Through theintegration of the Active Contour Model (ACM) with the

Elastic Graph Dynamic Link Model (EGDLM), this paperillustrates the efficiency and effectiveness of iJADE Webminer for automatic online human-face recognition. Withrespect to the interactive and mobile agent-based product

search, this paper illustrates how intelligent agent technol-ogy can be successfully integrated with AI technology foronline product identification and search. Through the

implementation of a fuzzy neuro-based product classifica-tion and search function in our proposed iJADE framework,

this paper illustrates the flexibility and efficiency of iJADE

Web miner for interactive online shopping.From an application point of view, this paper demon-

strates how various AI technologies (including fuzzy logicand neural networks) can be successfully integrated withmobile-agent technology to provide a truly intelligent,mobile, and interactive Web mining solution. We hope thatthe implementation of the iJADE Web miner can provide anew era for future e-commerce: intelligent e-commerce.

ACKNOWLEDGMENTS

The authors are grateful for the partial support provided by

the Central Research Grants, B-Q569 and G-T375, and the

Departmental (Department of Computing) Grant for the

iJADE project (Z042) from the Hong Kong Polytechnic

University.

REFERENCES

[1] ABE, http://www.research.ibm.com/iagents/home/html, 2003.[2] Aglets, http://www.trl.ibm.co.jp/aglets/, 2003.[3] P. Atzeni, A. Masci, G. Mecca, P. Merialdo, and E. Tabet,

“ULIXES: Building Relational Views over the Web,” Proc. 13thInt’l Conf. Data Eng., p. 576, 1997.

[4] AuctionBot, http://auction.eecs.umich.edu, 1999.[5] BargainFinder, http://bf.cstar.ac.com/bf, 2001[6] J. Bettman, An Information Processing Theory to Consumer Choice.

Addison-Wesley, 1979.[7] D. Caughey, I. Ingham, and P. Watson, “Metabroker: A Generic

Broker for Electronic Commerce,” Computer Networks and ISDNSystems, vol. 30, no. 1, pp. 619-620, 1998.

[8] D. Che, Y. Chen, K. Aberer, and H. Eisner, “The Advanced WebQuery System of GPCRDB,” Proc. 11th Int’l Conf. Scientific andStatistical Database Management, pp. 879-885, 1999.

[9] R. Cooley, B. Mobasherand, and J. Srivastava, “Web Mining:Information and Pattern Discovery on the World Wide Web,”Proc. Ninth Int’l Conf. Tools with A. I. ‘97, pp. 558-567, 1997.

[10] P. Dasgupta, N. Narasimhan, L.E. Moser, and P.M. Smith,“MAGNET: Mobile Agents for Networked Electronic Trading,”IEEE Trans. Knowledge and Data Eng., vol. 11, no. 4, pp. 509-525,July/Aug. 1999.

[11] J. Engeland and R. Blackwell, Consumer Behavior. CBS CollegePublishing, 1982.

[12] U.M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy,Advances in Knowledge Discovery and Data Mining. AAAI Press/The MIT Press, 1996.

[13] FTP software agents, http://www.ftp.com, 2000.[14] K. Hammond, R. Burke, C. Martin, and S. Lytinen, “FAQ-Finder:

A Case-Based Approach to Knowledge Navigation,” WorkingNotes of the AAAI Spring Symposium: Information Gathering fromHeterogeneous, Distributed Environments, pp. 1136-1142, AAAIPress, 1995.

[15] I. Hossain, J. Liu, and R. Lee, “A Study of Multilingual SpeechFeature: Perspective Scalogram Based on Wavelet Analysis,” Proc.IEEE Int’l Conf. Systems, Man, and Cybernetics (SMC ’99), vol. II,pp. 178-183, 1999.

[16] J. Howard and J. Sheth, The Theory of Buyer Behavior. John Wileyand Sons, 1969.

[17] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active ContourModels,” Int’l J. Computer Vision, pp. 321-331, 1987.

[18] M. Klusch, Intelligent Information Agents: Agent-Based InformationDiscovery and Management on the Internet. Berlin: Springer-Verlag,1999.

[19] R.S.T. Lee and J.N.K. Liu, “iJADE IWShopper—A New Age ofIntelligent Mobile Web Shopping System Based on Fuzzy-NeuroAgent Technology,” Web Intelligence: Research and Development(LNAI 2198), pp. 403-412, 2001.

[20] R.S.T. Lee and J.N.K. Liu, “iJADE Stock Advisor—A New Era ofE-Finance Using an Intelligent Agent-Based Time Series StockPrediction Model,” Proc. Int’l Conf. Artificial Intelligence (ICAI2001), pp. 43-46, 2001.


TABLE 6Fitness Values for the Eight Different Product Categories

[21] R.S.T. Lee and J.N.K. Liu, “Tropical Cyclone Identification andTracking System Using Integrated Neural Oscillatory ElasticGraph Matching and Hybrid RBF Network Track MiningTechniques,” IEEE Trans. Neural Network, vol. 11, no. 3, pp. 680-689, 2000.

[22] R.S.T. Lee and J.N.K. Liu, “FAgent—An Innovative E-ShoppingAuthentication Scheme Using Invariant Intelligent Face Recogni-tion Agent,” Proc. Int’l Conf. Electronic Commerce (ICEC 2000),pp. 47-53, 2000.

[23] R.S.T. Lee and J.N.K. Liu, “Teaching and Learning the A.I.Modeling,” Innovative Teaching Tools: Knowledge-Based Paradigms(Studies in Fuzziness and Soft Computing 36), pp. 31-86, Physica-Verlag, Springer, 2000.

[24] R.S.T. Lee and J.N.K. Liu, “An Integrated Elastic Contour Fittingand Attribute Graph Matching Model for Automatic Face Codingand Recognition,” Proc. Third Int’l Conf. Knowledge-Based IntelligentInformation Eng. Systems (KES ’99), pp. 292-295, 1999.

[25] R.S.T. Lee and J.N.K. Liu, “An Automatic Satellite Interpretationof Tropical Cyclone Patterns Using Elastic Graph Dynamic LinkModel,” Int’l J. Pattern Recognition and Artificial Intelligence, vol. 13,no. 8, pp. 1251-1270, 1999.

[26] R.S.T. Lee and J.N.K. Liu, “An Oscillatory Elastic Graph MatchingModel for Scene Analysis,” Proc. Int’l Conf. Imaging Science,Systems, and Technology (CISST ’99), pp. 42-45, 1999.

[27] H.V. Leighton and J. Srivastava, “Precision among WWWSearch Services (Search Engines): Alta Vista, Excite, Hotbot,Infoseek, Lycos,” http://www.winona.musus.edu/is-f/library-f/webind2/webind2.htm, 1997.

[28] J.N.K. Liu and R.S.T. Lee, “Rainfall Forecasting from MultiplePoint Source Using Neural Networks,” Proc. IEEE Int’l Conf.Systems, Man, and Cybernetics (SMC ’99), vol. II, pp. 429-434, 1999.

[29] J.N.K. Liu and R.S.T. Lee, “Invariant Character Recognition inDynamic Link Architecture,” Proc. Int’l Conf. Knowledge and DataExchange (KDEX ’97), pp. 188-195, 1997.

[30] J.N.K. Liu and R.S.T. Lee, “Invariant Handwritten ChineseCharacter Recognition,” Proc. Int’l Conf. Neural Information Proces-sing (ICONIP ’98), pp. 275-278, 1998.

[31] C. Malsburg, “Pattern Recognition by Label Graph Matching,”Neural Networks, vol. 1, no. 1, pp. 141-148, 1988.

[32] Odyssey, http://www.isysdev.com/index.shtml, 2003.[33] J. Pitkow, “In Search of Reliable Usage Data on the WWW,” Proc.

Sixth Int’l WWW Conf., pp. 451-463, 1997.[34] W. Rankl and W. Effing, Smart Card Handbook. Wiley, 1997.[35] U. Shardanand and P. Maes, “Social Information Filtering:

Algorithms for Automating ‘Word of Mouth’,” Proc. Int’l Conf.Human Factors in Computing Systems ‘95, pp. 210-217, 1995.

[36] E. Spertus, “ParaSite: Mining Structural Information on the Web,”Proc. Sixth Int’l WWW Conf., pp. 485-492, 1997.

[37] J. Triesch and C. Malsburg, “Robotic Gesture Recognition,” Proc.Second Conf. Automatic Face and Gesture Recognition ‘96, pp. 170-175,1996.

[38] Voyager, http://www.recursionsw.com/osi.asp, 2003.[39] J. Wang, Y. Huang, G. Wu, and F. Zhang, “Web Mining:

Knowledge Discovery on the Web,” Proc. Int’l Conf. Systems,Man and Cybernetics (SMC ’99), vol. 2, pp. 137-141, 1999.

Raymond S.T. Lee received the BSc degreefrom the Hong Kong University in 1989, the MScdegree in information technology, and the PhDdegree from the Hong Kong Polytechnic Uni-versity in 1997 and 2000, respectively. Aftergraduation from the Hong Kong University, hejoined the Hong Kong Government in the HongKong Observatory as a meteorological scientiston weather forecasting and developing meteor-ological telecommunication information systems

from 1989 to 1993. Prior to joining Hong Kong Polytechnic University in1998, he worked as an MIS manager and system consultant in HongKong. He is now an assistant professor in the Department of Computingat the Hong Kong Polytechnic University, working in the areas of neuralnetwork and pattern recognition. His current research interests include:artificial intelligence, intelligent e-commerce systems, intelligent agents,weather simulation and forecasting, and chaotic neural networks. He is amember of the IEEE and the ACM.

James N.K. Liu received the BSc (Hons) andMPhil degrees in mathematics and computa-tional modeling from Murdoch University, Aus-tralia, in 1982 and 1987, respectively. Hereceived the PhD degree in artificial intelligencefrom La Trobe University, Australia, in 1992.While working on his degree, he worked as acomputer scientist at the Defence Signal Direc-torate in Australia from 1988 until 1990. Hejoined the Aeronautical Research Laboratory

(ARL) at the Defence Science and Technology Organization in Australiaas a research scientist in 1990. Dr Liu joined the Department ofComputing at the Hong Kong Polytechnic University in 1994 and is nowan associate professor. His current interests include intelligent businesscomputing, multilingual system development, weather simulation andforecasting, data mining and Web-based information systems, andagent modeling. He is a member of the IEEE and AAAI.

. For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


Documents

iJADE web-miner: an intelligent agent framework …gkmc.utah.edu/7910F/papers/IEEE TKDE iJade web miner for...iJADE Web-Miner: An Intelligent Agent Framework for Internet Shopping