Cloud based unsupervised learning architecture based on Mirroring Neural Networks

K. ESWARAN and C.CHAITANYA Computer Science and Engineering

Sri Nidhi Institute of Science and Technology Yamnampet,Ghatkesar,Hyderabad

INDIA kumar.e@gmail.com,chaitanya@ozonetel.com http://www.altechimaging.com

Abstract: - In this paper we build upon the Mirroring theorem introduced in [15] as a new method of unsupervised hierarchical pattern classification. The Mirroring theorem affirms that "given a collection of samples with enough information in it such that it can be classified into classes and sub-classes then

1. There exists a mapping which classifies and sub-classifies these samples 2. There exists a hierarchical classifier which can be constructed by using Mirroring Neural Networks (MNNs) in

combination with a clustering algorithm that can approximate this mapping." This paper visualizes a cloud based scalable self learning engine, Pioneer, on top of the mirroring neural network architecture. Specifically we discuss about:

1. The modularity and scalability of MNNs to lend themselves to a cloud based architecture. 2. Validation methodology adopted to validate the parallelizing of Mirroring theorem 3. Exposing Pioneer through web service APIs to allow people to build their own unsupervised systems and

allow the crowd sourcing of intelligence. Key-Words: - Cloud computing, Unsupervised learning, Neural Networks, Mirroring Theorem 1 Introduction Unsupervised learning in computers has for long been considered as the holy grail of computer problems. The human brain is a complex learning system which has evolved to process inputs from various sources (eyes, ears, tongue, skin and nose), remove noise from the information it receives from the input sources, extract patterns from these inputs and store the core values of these patterns for future reference. The brain processes these patterns using a common cortical algorithm (evidence for the existence of the common cortical algorithm is from the studies of the neuroscientists in [1-4]). It is assumed that the common cortical algorithm has the general set of assumptions [5] for processing the pattern learning and recognition, from the fact that the same algorithm is implemented on various categories of sensory data. The fundamental similarities (identical properties) of various neocortical areas of neocortex were discussed in [6]. The organization of the cerebral cortex is hierarchical [7, 8] and the information flow is from one cortical area to another (discussed in [9]). Furthermore, these cortical regions are inter-connected to recognize the patterns level-by-level i.e., one or more neocortical regions of different levels of the hierarchy (on different pattern detail abstractions) perform distinct tasks of

recognition based on a common cortical (learning) algorithm. In the process of recognition, the pattern features are passed from one level to another level. Seemingly, many such hierarchical tree-like structures exist in the human brain representing the recognition structure of the patterns and they may belong to any one of the sensory categories like image patterns, voice data, odors etc. And the brain can also associate different kinds of patterns (an example case is association of image data with voice data) by mapping these hierarchical structures of the recognition system. [10] Discusses unsupervised learning and introduces the concept of Mirroring Neural Networks. MNNs imitate the basic hierarchical architecture of human neo cortex for self-learning, automatic feature extraction (performed along with data reduction), unsupervised hierarchical pattern classification and memory association of the patterns. The “Mirroring Theorem” proves that a hierarchical pattern recognizer exists and can be approximated by a treelike architecture consisting of mirroring neural networks. It is our belief that with current explosion of cloud services, we can harness the computing resources and provide an architectural solution which utilizes the MNNs as the basic unit to solve the problem of unsupervised learning.

Recent Researches in Artificial Intelligence, Knowledge Engineering and Data Bases

ISBN: 978-960-474-273-8 350

The mirroring neural networks proposed in [10] have the following attributes:

1. Input independence (which can be applied to the real world recognition problems)

2. Mechanism of accelerated learning 3. A learning in unsupervised mode which is easy

to program 4. The attribute of imitating the natural neuronal

learning 5. The inherent flexibility to be used as a module

in the hierarchical structure MNNs have been proven to work in an unsupervised learning environment. Let us now look at possible solutions and challenges in trying to scale the MNN based learning engine. 2 Using MNNs in a cloud based architecture The main problems with the system proposed in [15] are scalability and volume of input data. Though [15] presents a working example of the MNN based learning system that system cannot scale for classifying objects belonging to more than (say) 10 classes. If we want to build a truly unsupervised system, then we have to think beyond a single instance and an architecture which should scale simply and automatically. To solve the problem of input data volume, our system should provide a transparent mechanism through which users can feed the data to the system. We suggest that the first problem can be solved through cloud based architecture [18] and the second problem can be solved by exposing a REST API [17] for data feeding. The learning engine proposed in [15] has the following characteristics: (i) hierarchical (ii) modular (iii) unsupervised (iv) runs on a single common algorithm(MNN

associated with clustering). The advantage of developing a recognition system with these 4 characteristics is that the learning method does not depend on the problem size and the learning network can be simply extended as the recognition task becomes more complex. It has been surmised by investigators that the architecture of human neo-cortex does, loosely speaking, possess the above 4 characteristics (except that instead of (iv) there is some kind of analog classification process procedure) performed by sets of neurons, which seemingly behave in a similar manner). Since this architecture imitates the neural architecture (though admittedly in a crude manner), it is reasonable to expect

that it would work with greater successes if the system is adapted to the cloud and computing resources can be provisioned on request.. 2.1 Feasibility Since the learning engine specified in [15] is hierarchical and modular in nature, it lends itself perfectly to a cloud computing architecture. Since Amazon launched their web service framework, AWS [14], cloud computing has proliferated. We are now able to provision computing resources on demand at a fraction of the cost. This perfectly fits with our vision of an unsupervised learning system which can request computing resources on demand and relinquish them when not needed. At the highest level, each MNN can be deployed as an instance in the cloud and these can be arranged in a hierarchical fashion through a messaging framework. A basic view of such a system is given below.

Fig.1 Basic diagram depicting each MNN deployed as a separate instance in a cloud framework In the above figure each set of MNN (in one of the blocks shown) can be self-trained to recognize a few classes, and another set of MNN –self-trained to recognize some other class of images. Though an instance per object is overkill, it is good enough to show the feasibility of the concept. Later on, cloud instances can be replaced by dedicated devices. By deploying the learning system on the cloud, it will be always on continuously taking visual inputs and classifying objects in an unsupervised way. The system will scale on demand and can classify numerous objects limited only by the number of instances in the cloud. Pioneer will automatically classify the objects, but a human has to tag those objects and give a name for them. This is similar to a toddler being able to

Recent Researches in Artificial Intelligence, Knowledge Engineering and Data Bases

ISBN: 978-960-474-273-8 351

differentiate between different objects, but his parents have to name the objects for him 2.2 Validation As validation of this architecture, a basic learning system, Pioneer, is being built which will run continuously grabbing video frames and automatically classifies objects. Pioneer will receive visual data continuously. For this we will connect a camera to this system which will continuously feed image frames to the system. The frames will first pass through an object detection framework, which will scan the frame on incremental window sizes for objects. When objects are detected, they are then passed to the MNN framework which classifies them into either a known bucket or a new bucket. To achieve this Pioneer will broadcast a message to the MNN instances and the MNN instances will respond in a predetermined way. Based on the responses Pioneer will categorize this as a new object and create a new MNN instance or assign it to the correct MNN instance. The broadcasting functionality can be similar to ARP [16]. A new object bucket can be created automatically using cloud provisioning APIs like Amazon web service APIs etc. The system then randomly chooses a 6 letter name for this object bucket. We can then tag the buckets with our naming scheme or allow the system to run as it is which will basically mean that the system will come out with its own language to identify objects. The flow chart for this system is given below.

Fig 2. Flow chart for Pioneer system. A tree like architecture consisting of one MNN at the first level and 3 MNN’s at the second level automatically classifying a collage of images belonging to three classes is shown below.

Ig#.

Fig. 3 A tree like architecture of two levels of MNNs 2.3 Learning system APIs: The effectiveness of any cloud based system is only as good as the input it receives. To enable our MNNs access to a varied data set, Pioneer, will have well defined web service interface using which users from everywhere can feed it visual data. Pioneer will then try to classify the object into existing buckets. In case it is a new object, Pioneer will then query the user for more information about the object and create a new bucket for it. As an example, Pioneer will expose a REST based API for interaction with users. Users can feed data to Pioneer using a simple http request like http://www.pioneer.com/newfeed?data=................, here data is the binary image More information on creating domain specific APIs can be found in [19]. Pioneer will then use this as a new input into its system and try to learn about the input. More APIs can be exposed to name the objects, edit object information etc. This way, Pioneer will depend on crowd sourced data to learn about its world. Through this crowd sourcing experiment we hope to build the best unsupervised object classification system. 4 Conclusion In this paper we have provided a basic analysis of how MNN based learning systems can be scaled using cloud based architecture. Using this as a base future researchers can try multiple architectural permutations and combinations to solve the unsupervised learning

Recent Researches in Artificial Intelligence, Knowledge Engineering and Data Bases

ISBN: 978-960-474-273-8 352

problem. The basic system can be put to use in real world problem scenarios like content based image retrieval and video tagging. With the amount of online video being generated, these can be fed to Pioneer as input and Pioneer will continually process the data frames and continue to learn from the frames. References: [1] Mountcastle, Vernon B., An organizing principle for

cerebral function: The unit model and the distributed system, in The Mindful Brain, Eds. G.M. Edelman and V.B. Mountcastle, Cambridge, Mass.MIT Press, 1978.

[2] L von Melchner, S L Pallas, and M Sur. Visual behaviour mediated by retinal projections directed to the auditory pathway. Nature, 404(6780):871-876, 2000.

[3] Paul Bach-y Rita and Stephen W Kercel. Sensory substitution and the human-machine interface. Trends Cogn Sci, 7(12):541-546, 2003.

[4] Paul Bach-y Rita. Tactile sensory substitution studies. Ann N Y Acad Sci, 1013:83-91, 2004.

[5] Dileep George, How the brain might work: A hierarchical and temporal model for learning and recognition, Ph. D. Thesis, Stanford University, June 2008.

[6] Creutzfeldt, Otto D., Generality of the functional structure of the Neocortex, Naturwissenschaften, 64, 507-517,1977.

[7] D. J. Felleman and D. C. Van Essen, Distributed hierarchical processing in the primate cerebral cortex, Cerebral Cortex, 1(1), 1-47, January/February 1991.

[8] Rao, R. P., and D.H. Ballard, Predictive coding in the visual cortex: A functional interpretation of some extra-classical-receptive-field effects, Nature Neuroscience, 2(1), 79-87, 1999.

[9] Sherman, S.M., and R.W. Guillery, The role of the thalamus in the flow of information to the cortex, Philosophical Transactions of the Royal Society of London, 357(1428), 1695-708, 2002.

[10] Dasika Ratna Deepthi, G. R. Aditya Krishna & K. Eswaran, Automatic pattern classification by unsupervised learning using dimensionality reduction of data with mirroring neural networks, IEEE International Conference on Advances in Computer Vision and Information Technology (IEEE, ACVIT-07), pp. 354-360, 2007.

[11] Juan Carlos Niebles, Hongcheng Wang and Li Fei-Fei, Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words,

[12] Yuanhao Chen, Long (Leo) Zhu, Alan Yuille, Hongjiang Zhang, Unsupervised Learning of Probabilistic Object Models (POMs) for Object

Classification, Segmentation, and Recognition Using Knowledge Propagation, PAMI,Oct. 2009

[13] Face.com, A cloud based API for face detection. [14] aws.amazon.com: Amazon web services [15] Dasika Ratna Deepthi & K. Eswaran, A Mirroring

Theorem and its Application to a New Method of Unsupervised Hierarchical Pattern Classification, International Journal of Computer Science and Information Security, p16-25, no 1, vol 6, 2009

[16] ARP, http://www.faqs.org/rfcs/rfc826.html [17]REST,http://en.wikipedia.org/wiki/Representational

_State_Transfer [18] Michael Miller, Cloud Computing: Web-Based

Applications That Change the Way You Work and Collaborate Online, Que Publishing Company, 2008

[19] E. Michael Maximilien, Hernan Wilkinson, Nirmit Desai and Stefan Tai, A Domain-Specific Language for Web APIs and Services Mashups, ICSOC 2007, Volume 4749/2007, 13-26

Recent Researches in Artificial Intelligence, Knowledge Engineering and Data Bases

ISBN: 978-960-474-273-8 353