13: Practical Artificial Intelligence & Machine Learning (Arturo Servin)

  • Published on
    31-Oct-2014

  • View
    10

  • Download
    3

Embed Size (px)

DESCRIPTION

 

Transcript

<ul><li> 1. Practical Artificial Intelligence and Machine Learning <ul><li>arturo.servin_at_gmail.com </li></ul></li></ul> <ul><li>http://arturo.servin.googlepages.com/ </li></ul> <p> 2. About this presentation </p> <ul><li>Some theory on AI and ML </li></ul> <ul><li>Some practical ideas and simple how to </li></ul> <ul><li>What's out there using AI </li></ul> <ul><li>Resources, Kits and Data </li></ul> <p> 3. Artificial Intelligence </p> <ul><li>Machine Learning </li></ul> <ul><li>Natural Language Processing </li></ul> <ul><li>Knowledge representation </li></ul> <ul><li>Plannning</li></ul> <ul><li>Multi-Agent Systems </li></ul> <ul><li>and some other stuff depending of the author of the book </li></ul> <p> 4. Machine Learning </p> <ul><li>A program islearningwhen it executes a taskTand acquires experienceEand the measured performancePofTimproves with experienceE(T. Mitchell, Machine Learning, 1997) </li></ul> <p> 5. Machine Learning Flavours </p> <ul><li>Supervised Learning </li></ul> <ul><li><ul><li>Programs learn a concept/hypothesis by means of labeled examples </li></ul></li></ul> <ul><li><ul><li>Examples: Artificial Neural Networks, Bayesian Methods, Decision Trees </li></ul></li></ul> <ul><li>Unsupervised Learning </li></ul> <ul><li><ul><li>Programs learn to categorise unlabelled examples </li></ul></li></ul> <ul><li><ul><li>Examples: Non-negative matrix factorization and self-organising maps </li></ul></li></ul> <p> 6. More flavours </p> <ul><li>Reinforcement Learning </li></ul> <ul><li><ul><li>Programs learn interacting with the environment, the execution of actions and observing the feedback in the form of + or rewards </li></ul></li></ul> <ul><li><ul><li>Examples: SARSA, Q-Learning </li></ul></li></ul> <p> 7. Training Examples </p> <ul><li>Continuous </li></ul> <ul><li>Discrete </li></ul> <ul><li>Inputs know asVectorsorFeatures </li></ul> <ul><li>Example in Wine Classification: Alcohol level, Malic acid, Ash, Alcalinity of ash, etc.</li></ul> <p> 8. </p> <ul><li>Linear and Non-linear feature relations </li></ul> <ul><li>source: Oracle Data Mining Concepts </li></ul> <p> 9. More complex feature relations 10. Decision Trees </p> <ul><li>Easy to understand and to interpret </li></ul> <ul><li>Hierarchical structure </li></ul> <ul><li>They useEntropyandGini impurityto create groups </li></ul> <ul><li>Disadvantage: It's an off-line method </li></ul> <ul><li>Examples: ID3, C4.5 </li></ul> <p>Source: http://www2.cs.uregina.ca/~dbd/cs831/notes/ml/dtrees/c4.5 11. Decision Trees, example </p> <ul><li>Create .names and .data files with training data </li></ul> <ul><li>Generate tree and rules (c4.5 -f and c4.5rules -f ) </li></ul> <ul><li>Outlook Temperature Humidity Windy Play or Don't Play</li></ul> <ul><li>Sunny80 90 true Don't Play</li></ul> <ul><li>Overcast83 78 false Play</li></ul> <ul><li>Rain 70 96false Play</li></ul> <ul><li>Categorize new data (consult,consultr). Use GPS/Geocoding, Google Maps and Yahoo Weather APIs to enhance </li></ul> <ul><li>aservin@turin:~/Projects/C45: consult -f golf </li></ul> <ul><li>C4.5 [release 8] decision tree interpreter Sat Jan 17 00:05:16 2009 </li></ul> <ul><li>------------------------------------------ </li></ul> <ul><li>outlook: sunny </li></ul> <ul><li>humidity: 80 </li></ul> <ul><li>Decision: </li></ul> <ul><li>Don't PlayCF = 1.00[ 0.63 - 1.00 ] </li></ul> <p> 12. Bayesian Classifiers </p> <ul><li>Bayes Theorem:P(h|D)=P(D|h) P(h) P(D) </li></ul> <ul><li>P(h)= prior probability of hypothesish </li></ul> <ul><li>P(D)= prior probability of training dataD </li></ul> <ul><li>P(h|D) = probability ofhgivenD </li></ul> <ul><li>P(D|h) = probability ofDgivenh </li></ul> <ul><li>Naive Bayes Classifier, Fisher Classifier </li></ul> <ul><li>Commonly used in SPAM filters </li></ul> <p> 13. Classifying your RSS feeds </p> <ul><li>Use the unofficial Google Reader APIhttp://blog.gpowered.net/2007/08/google-reader-api-functions.html </li></ul> <ul><li>Some Python Code(Programming Collective Intelligence, Chapter 6) </li></ul> <ul><li>Tag interesting and non-interesting items </li></ul> <ul><li>Train using Naive-bayes or Fisher classsifier </li></ul> <ul><li>&gt;&gt; cl.train('Google changes favicon','bad') </li></ul> <ul><li>&gt;&gt; cl.train('SearchWiki: make search your own','good') </li></ul> <ul><li>New items are tagged as interesting or not </li></ul> <ul><li>&gt;&gt; cl.classify('Ignite Leeds Today') </li></ul> <ul><li>Good </li></ul> <ul><li>You can re-train online </li></ul> <ul><li>Add more features, try with e-mail </li></ul> <p> 14. Finding Similarity </p> <ul><li>Euclidean Distance, Pearson Correlation Score, Manhattan </li></ul> <ul><li>Document Clustering </li></ul> <ul><li>Price Prediction </li></ul> <ul><li>Item similarity </li></ul> <ul><li>k-Nearest Neighbors, k-means, Hierarchical Clustering, Support-Vector Machines, Kernel Methods </li></ul> <p> 15. Similar items Source: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html 16. Artificial Neural Networks </p> <ul><li>Mathematical/Computational model based on biological neural networks </li></ul> <ul><li>Many types. The most common useBackpropagationalgorithm for training andFeedforwardalgorithm to get results/training </li></ul> <p> 17. Artificial Neural Networks </p> <ul><li>Input is high-dimensional, discrete or real-valued (e.g. raw sensor input) </li></ul> <ul><li>Output is discrete or real valued </li></ul> <ul><li>Output is a vector of values </li></ul> <ul><li>Perceptron, linear </li></ul> <ul><li>Sigmoid, non-linear and multi-layer </li></ul> <p> 18. Example, finding the best price </p> <ul><li>Create training data using Amazon/Ebay API </li></ul> <ul><li>Laptop prices. Use price, screen size as features </li></ul> <ul><li>Use a ANN, i.e. Fast Artificial Neural Network (FANN) </li></ul> <ul><li>struct fann *ann = fann_create_standard(num_layers, num_input, num_neurons_hidden, num_output); #C++ </li></ul> <ul><li>ann = fann.create(connection_rate, (num_input, num_neurons_hidden, num_output))#Python </li></ul> <ul><li>$ann = fann_create(array(2, 4, 1),1.0,0.7); // PHP </li></ul> <ul><li>You can also tryk-Nearest Neighbours </li></ul> <ul><li>Try it! </li></ul> <p> 19. Resources 1 </p> <ul><li>Books </li></ul> <ul><li><ul><li>Practical Artificial Intelligence Programming in Java, Mark Watsonhttp://www.markwatson.com/opencontent/(There is a Ruby one as well) </li></ul></li></ul> <ul><li><ul><li>Programming Collective Intelligence, Toby Segaran; O'Reilly </li></ul></li></ul> <ul><li><ul><li>Artificial Intelligence: A Modern Approach, S. Russell, P. Norvig, J. Canny; Prentice Hall, </li></ul></li></ul> <ul><li><ul><li>Machine Learning, Tom Mitchell; MIT Press </li></ul></li></ul> <ul><li>Online Stuff </li></ul> <ul><li><ul><li>ML course in Stanford http://www.stanford.edu/class/cs229/materials.html </li></ul></li></ul> <ul><li><ul><li>Statistical MLhttp://bengio .abracadoudou.com/lectures/ </li></ul></li></ul> <p> 20. Resources 2 </p> <ul><li>Code </li></ul> <ul><li><ul><li>FANNhttp://leenissen.dk/fann/index.php </li></ul></li></ul> <ul><li><ul><li>NLPhttp://opennlp.sourceforge.net/ </li></ul></li></ul> <ul><li><ul><li>C4.5 http://www2.cs.uregina.ca/~dbd/cs831/notes/ml/dtrees/c4.5/tutorial.html </li></ul></li></ul> <ul><li><ul><li>ML and Java http://www.developer.com/java/other/article.php/10936_1559871_1 </li></ul></li></ul> <ul><li>Data </li></ul> <ul><li><ul><li>UC Irvine Machine Learning Repositoryhttp://archive.ics.uci.edu/ml/ </li></ul></li></ul> <ul><li><ul><li>Amazon Public Datasetshttp://aws.amazon.com/publicdatasets/ </li></ul></li></ul> <p> 21. More info </p> <ul><li>For questions, projects and job offers: </li></ul> <ul><li><ul><li>arturo.servin (at) gmail.com </li></ul></li></ul> <ul><li><ul><li>http://twitter.com/the_real_r2d2 </li></ul></li></ul> <ul><li><ul><li>http://arturo.servin.googlepages.com/ </li></ul></li></ul>