Click here to load reader

Tom M. Mitchell E. Fredkin Professor and Department Head March 2007

  • View

  • Download

Embed Size (px)


The Discipline and Future of Machine Learning. Tom M. Mitchell E. Fredkin Professor and Department Head March 2007. The Discipline of Machine Learning. The defining question: - PowerPoint PPT Presentation

Text of Tom M. Mitchell E. Fredkin Professor and Department Head March 2007

  • Tom M. MitchellE. Fredkin Professor and Department HeadMarch 2007 The Discipline and Futureof Machine Learning

  • The Discipline of Machine Learning

    The defining question: How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

    A process learns with respect to if it Improves its performance Pat task Tthrough experience E

  • Machine Learning - PracticeSpeech RecognitionControl learning Reinforcement learning Supervised learning Bayesian networks Hidden Markov models Unsupervised clustering Explanation-based learning ....Extracting facts from text

  • Machine Learning - TheoryPAC Learning Theory# examples (m)representational complexity (H)error rate (e)failure probability (d)Other theories for Reinforcement skill learning Semi-supervised learning Active student querying also relating: # of mistakes during learning convergence rate asymptotic performance bias, variance VC dimension(for supervised concept learning)

  • The Discipline of Machine LearningMachine Learning: How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

    Computer Science:How can we build machines that solve problems, and which problems are inherently tractable/intractable?

    Statistics:What can be learned from data with a set of modeling assumptions, while taking into account the data-collection process?

  • Animal learning (Cognitive science, Psychology, Neuroscience)Machine learningStatistics Computer scienceAdaptive Control TheoryandRoboticsEvolutionEconomics

  • ML and CSMachine learning already the preferred approach toSpeech recognition, Natural language processingComputer visionMedical outcomes analysisMany robot control problems

    The ML niche will growWhy?All softwareML software

  • ML and Empirical SciencesEmpirical science is a learning process, subject to automation and to studyimprove performance P (accuracy)at task T (predict which gene knockouts will impact the aromatic AA pathway, and how)with experience E (active experimentation)

    Functional genomic hypothesis generation and experimentation by a robot scientist, King et al., Nature, 427(6971), 247-252Which protein ORFs influence which enzymes in the AAA pathway

  • Our current state:The problem of tabula-rasa function approximation is solved (in an 80-20 sense): Given: Class of hypotheses H = {h: X Y}Labeled examples {}Determine:The h from H that best approximates f

    Its time to move onEnrich the function approx problem definitionUse function approx as building blockWork on new problems

  • Some Current Research QuestionsWhen/how can unlabeled data be useful in function approximation?

    How can assumed sparsity of relevant features be exploited in high dimensional nonparametric learning?

    How can information learned from one task be transferred to simplify learning another?

    What algorithms can learn control strategies from delayed rewards and other inputs?

    What are the best active learning strategies for different learning problems?

    To what degree can one preserve data privacy while obtaining the benefits of data mining?

  • The Future of Machine Learning

  • A Quick Look Back19601970198019902000Samuels checker learnerPerceptronsWinstons symbolic concept learnerRule learningDecision tree learningNeural networksExplanation-based learningDimensionality reductionBayes netsPAC learning theoryArchitectures for learning and problem solvingReinforcement learningSemi-supervised learningNon-parametric methodsStatistical perspective on learningHMMsSVMsTheories of grammar inductionLarge scale dataminingSpeech applicationsRobot controlPrivacy preserving data miningTransfer learningVersion SpacesTheories of perceptron capacity and learnabilityEvolutionary and revolutionary changesWhat might lead to the next revolution?

  • Use Machine Learning to help understand Human Learning(and vice versa)

  • Models of Learning Processes# of examplesError rateReinforcement learningExplanations

    Learning from examplesComplexity of learners representationProbability of successPrior probabilitiesLoss functions

    # of examplesError rateReinforcement learningExplanations

    Human supervisionLecturesQuestions, HomeworksAttention, motivationSkills vs. PrinciplesImplicit vs. Explicit learningMemory, retention, forgettingHebbian learning, consolidation Machine Learning:Human Learning:

  • Reinforcement Learning[Sutton and Barto 1981; Samuel 1957]Observed immediate rewardLearned sum of future rewards

  • Reinforcement Learning in MLr =100V=1000V=72V=81V=90 = .9S0S2S1S3

  • Reinforcement Learning in MLVariants of RL have been used for a variety of practical control learning problems Temporal Difference learningQ learning Learning MDPs, POMDPs

    Theoretical results tooAssured convergence to optimal V(s) under certain conditionsAssured convergence for Q(s,a) under certain conditions

  • Dopamine As Reward Signal[Schultz et al., Science, 1997]t

  • Dopamine As Reward Signal[Schultz et al., Science, 1997]t

  • Dopamine As Reward Signal[Schultz et al., Science, 1997]t

  • RL Models for Human Learning[Seymore et al., Nature 2004]

  • [Seymore et al., Nature 2004]

  • Human and Machine LearningAdditional overlaps:

    Learning of perceptual representationsDimensionality reduction methods, low level perceptsLewicky et al.: optimal sparse codes of natural scenes yield gabor filters found in primate visual cortex. Similar result for auditory cortex.

    Learning with redundant sensory inputCoTraining methods, Sensory redundancy hypothesis in developmentDe Sa & Ballard; Coen: co-clustering voice/video yields phonemesMitchell & Perfetti: co-training in second language learning

    Learning and explanationsExplanation-based learning, teaching concepts & skills, chunkingVanLehn et al: explanation-based learning accounts for some human learning behaviors.Chi: students learn best when forced to explainNewell; Anderson: chunking/knowledge-compilation models

  • 2. Never-ending learning

  • Never-Ending LearningCurrent machine learning systems: Learn one functionAre shut down after they learn itStart from scratch when programmed to learn the next function

    Lets study and construct learning processes that:Learn many different thingsFormulate their own next learning taskUse what they have already learned to help learn the next thing

  • Example: Never-ending learning robotImagine a robot with three goals: (1) avoid collisions, (2) recharge when battery low, and (3) find and collect trash

    What is stopping us from giving it some trash examples, then letting it learn for a year?

    What must it start with to formulate and solve relevant learning subtasks?Learn to recognize trash in sceneLearn where to search for trash, and whenLearn how close to get to find out whether trash is thereLearn to manipulate trashTransfer what it learned about paper trash to help with bottle trashDiscover relevant subcategories of trash (e.g., plastic versus glass bottles), and of other objects in the environment

  • Core Questions for Never-Ending Learning AgentWhat function or fact to learn next?Self-reflection on performance, credit assignment

    What representation for this target function or fact?Choice of input-output representation for target functionE.g., classify whether its trash

    How to obtain (which type of) training experience?Primarily self-supervised, but occasional teacher inputE.g., classify whether its trash

    Guided by what prior knowledge?Transfer learning, but transfer between what?XPaperTrash help learn XPlasticTrash ?State(t) x Action(t) State(t+1) help learn XPlasticTrash ?

  • Example: Never-ending language learnerRead the Web project: Create 24x7 web agent that each day:Extracts more facts from the web into structured databaseLearns to extract facts better than yesterday

    Starting point:Ontology of hundreds of categories and relationsand 6-10 training examples of each Never-ending learning architectureState of art language processing primitivesLearning mechanismsTop level task:Populate a database of these categories and relations by reading the web, and improve continually[Carlson, Cohen, Fahlman, Hong, Nyberg, Wang, ...]

  • Q: how can it obtain useful training experience (i.e., self-supervise)?A: redundancy

  • Bootstrapping: Learning to extract named entitiesI arrived in Pittsburgh on Saturday.location?x1: I arrived in _________ on Saturday.x2: Pittsburgh

  • Bootstrap learning to extract named entities[Riloff and Jones, 1999], [Collins and Singer, 1999], ...

  • Co-TrainingAnswer1Classifier1Answer2 Classifier2I flew to New York today.New YorkI flew to ____ todayIdea: Train Classifier1 and Classifier2 to:1. Correctly classify labeled examples2. Agree on classification of unlabeled

  • Co-Training Theory [Blum&Mitchell 98; Dasgupta 04, ...]Final Accuracy# unlabeled examplesConditional dependence among inputs# labeled examplesNumber of redundant inputs want inputs less dependent, increased number of redundant inputs, disagreement over unlabeled examples can bound true error

  • Example Bootstrap learning algorithms:

    Classifying web pages [Blum&Mitchell 98; Slattery 99]Classifying email [Kiritchenko&Matwin 01; Chan et al. 04]Named entity extraction [Collins&Singer 99; Jones 05]Wrapper induction [Muslea et al., 01; Mohapatra et al. 04]Word sense disambiguation [Yarowsky 96]Discovering new word senses [Pant

Search related