Upload
peter-morgan
View
174
Download
0
Embed Size (px)
Citation preview
Outline
• Concepts• AIMarket• Data• Software• Hardware• Conferences• Applications
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 2©IvyDataScience
DeepLearning/AIFrameworks- TheBigPicture
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 6
AIFrameworksCognitive
Architectures
MLFrameworksSupervised,
Unsupervised&Reinforcement
DeepLearningFrameworks
NeuralNetworks
©IvyDataScience
WhatisIntelligence?• IntelligenceisanAgent’s abilitytoadapttoandtoachievegoalswithinitsEnvironment• Humanvsmachineintelligence– ultimatelythesame• “Thetermartificialintelligenceissomewhatnonsensical.Somethingiseitherintelligentoritisn’t.Justassomethingeitherfliesoritdoesn’t.Wedon’ttalkaboutartificialflying”- Zoubin Ghahramani,CambridgeUniversity• Informationprocessing, computation,physics,hardware• Explorationvsexploitation• Biological(anyspecies)versusmachine(anytype)•Doessubstratematter- carbonvssilicon?
7© Ivy Data Science
WhatisLearning?
• Learningalgorithms– systemgetsbetterwithmoredatauntilnofurtherimprovement• Trainthesystem– justlikeanimalslearn• Supervised,unsupervisedandreinforcementlearning• Physically,itisthestrengtheningofconnections(synapses)betweennodes(neurons)• Memory(shortandlongterm)isinvolved• Deeplearningisasteptowardsthegoalofartificialgeneralintelligence(AGI)• Ensembleoftechniques
8© Ivy Data Science
DeepLearning=NeuralNetworks
• Referstosystemsthatlearnfromdata• Thesesystemsarebasedonartificialneuralnetworks(ANNs),whichinturnarebasedonbiologicalneuralnetworks(BNN),suchasthehumanbrain
• Inpracticesuchlearningsystemsconsistofdata,multiplelayers,nodes,weightsandoptimisationalgorithms
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 10©IvyDataScience
DeepLearningIsEatingtheWorld• Whataboutthe“AIwinters”?1974–80and1987–93,whereAIcompaniesover-promisedandunder-deliveredhttps://en.wikipedia.org/wiki/AI_winter
• Duetomorelabeled data,morecomputepower,betteroptimizationalgorithms,andbetterneuralnetmodelsandarchitectures,deeplearninghasstartedtosupersedehumanswhenitcomestoimagerecognitionandclassification
• Workisbeingdonetoobtainsimilarlevelsofperformanceinnaturallanguageprocessing andunderstanding
• AccordingtoJeffDeaninarecentinterview,GooglehaveimplementedDLinoveronehundredoftheirproductsandservicesincludingsearchandphotos
• AIisenjoyingarenaissancenow,notsimplybecauseofthepromiseitholdsforthefuturebutbecauseoftheimpactitishavingonbusinessestoday
• Timingiseverything,andthetimeisnow!
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 12©IvyDataScience
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 15
• Hardware(compute)– Nvidia GPU,Intel(Nervana),AMDRadeon• Dataavailable - structuredandunstructured• Researchactivity• Conferenceattendance(e.g.,NIPS)• Meetupgroups• PhDenrolmentsinCSandmachinelearning• Performancemeasures- chess,Jeopardy,Go,computervision,languageprocessing,...• NumberofpapersbeingpublishedinAI/ML
AITrends1
©IvyDataScience
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 16
• Availabilityofopensourcedeeplearningframeworks,includingTensorFlow,Mxnet,etc.- Numberofpackages- GitHubcommits- Contributorsetc.
• Forexample,TF ismostdownloadedrepofromGitHubinunderayear• FactthatmajorcorporatesopensourcedtheirAIframeworks,startingwithGoogle(TF,etc.)• NumberofAIrelatedjobsonjobboards• SalariesforAIexperts
AITrends2
©IvyDataScience
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 17
• BackingandrealignmentbycorporatestorebrandasAIcompanies-Microsoft,IBM,Amazon(GoogleandFacebookwerealreadythere)• Forexample:- IBMWatsonHQinNYC- Microsoftannouncing5000strongAIdivision- AppleannouncingatNIPSthatitwouldbeopensourcingitsAIresearch- Siri, Cortana,AlexaareallNLPappsusingneuralnets
• NumberofAIproductsandapps• NumberofAI/deeplearningstartups
AITrends3
©IvyDataScience
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 18
• VenturecapitalinvestmentinAIstartups• Numberofpress/newsarticles• AnnouncementsfromAIexpertswith30yearsexperienceconfirmingthatthistimeisforreal- therewillbenomoreAIwinters• NumberofprofessorsbeinghiredwayfromacademiatojoinAIcompanies,e.g.UberandCMU,GoogleandOxford,Facebook,Apple,etc.• GovernmentlevelpanelsonthedevelopmentandimpactofAIonjobs,societyandpolicy,e.g.,WhitehouseandU.K.parliament• AISafetyconsortiumannouncedlastmonthbetweenGoogle,Microsoft,IBMandAmazontotrackdevelopmentsinAI• RecentbookspublishedbyprofessorsandengineersonAIdevelopment
AITrends4
©IvyDataScience
AIRelatedBooks• Barrat,James,OurFinalInvention,St.Martin'sGriffin,2014• Bengio,Yoshua etal,DeepLearning,MITPress,2016• Brynjolfsson,ErikandAndrewMcAfee,TheSecondMachineAge,W.W.Norton&Co.,2014• Domingos,Pedro,TheMasterAlgorithm,BasicBooks,2015• Ford,Martin,RiseoftheRobots:TechnologyandtheThreatofaJoblessFuture,BasicBooks,2015• Kaku,Michio,TheFutureoftheMind, Doubleday,2014• Kurzweil,Ray,TheSingularityisNear, PenguinBooks,2006• Kurzweil,Ray,HowtoCreateaMind, PenguinBooks,2013• RussellandNorvig,ArtificialIntelligence,AModernApproach,Pearson,2009• Yampolskiy,Roman- ArtificialSuperintelligence,AFuturisticApproach,CRC,2015
©IvyDataScience 19
Imageclassification
Progressinmachineclassificationofimages- errorratebyyear.Redlineistheerrorrateofatrainedhuman.Under3%asofNov2016.
21IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016©IvyDataScience
GPUFasterthanMoore’sLaw
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 24© Ivy Data Science 24
Wheredoesthedatacomefrom?• Science – particle,astrophysics• Industry – oil,finance,telecom(allverticals)• Social – Facebook,LinkedIn,Twitter• Medicine – genome,neuroscience• Government – census,education,police• Sports– statistics• Environment – weather,sensors
31©IvyDataScience
DataSets• Rawdatainputintotheneuralnetworkcanoriginatefromanyenvironmentalsource• Itcanberecordedandstoredinadatabase(e.g.,text,images,audio,video),orlive(incidentdirectlyfromtheenvironment)streamingdata
• ExamplesofrecordeddatasetsincludeMNIST,Labeled FacesintheWild(LFW),ImageNet,CIFARandYouTube-8M
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 32
MNIST LFW
©IvyDataScience
ConvolutionalNeuralNetworks• Firstdevelopedin1970’s• Widelyusedforimagerecognitionandclassification• Inspiredbybiologicalprocesses,CNN’sareatypeoffeed-forwardANN• Theindividualneuronsaretiledinsuchawaythattheyrespondtooverlappingregionsinthevisualfield
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 35©IvyDataScience
RecurrentNeuralNetworks• Firstdevelopedin1970’s• RNN’sareneuralnetworksthatareusedtopredictthenextelementinasequenceortimeseries
• Thiscouldbe,forexample,wordsinasentenceorlettersinaword• Applicationsincludepredictingorgeneratingmusic,stories,news,code,financialinstrumentpricing,text,speech,infactthenextelementinanyeventstream
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 36©IvyDataScience
LSTMandNTM
• LongShortTermMemory(LSTM)• LSTM(Schmidhuber,1997)isanRNNarchitecturethatcontainsblocksthatcanrememberavalueforanarbitrarylengthoftime• Itsolvesthevanishingorexplodinggradientproblemwhencalculatingbackpropagation• AnLSTMnetworkisuniversal inthesensethatgivenenoughnetworkunitsitcancomputeanythingaconventionalcomputercancompute,providedithastheproperweightmatrix• LSTMoutperforms alternativeRNNsandHiddenMarkovModelsandothersequencelearningmethodsinnumerousapplications,e.g.,inhandwritingrecognition,speechrecognitionandmusiccomposition• NeuralTuringMachines(NTM)• NTMsareamethodofextendingthecapabilitiesofrecurrentneuralnetworksbycouplingthemtoexternalmemoryresources
37© Ivy Data Science
TensorFlow• TensorFlowisthenewly(Nov2015)opensourceddeeplearninglibraryfromGoogle
• Itistheirsecondgenerationsystemfortheimplementationanddeploymentoflarge-scalemachinelearningmodels
• WritteninC++withapythoninterface,itisbornefromresearchanddeployingmachinelearningprojectsthroughoutawiderangeofGoogleproductsandservices
• InitiallyTFranonlyonasinglenode(yourlaptop,say),butGooglehavenowreleasedaversionthatrunsonadistributedcluster
• AvailableinthecloudonGCP• https://www.tensorflow.org/
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 39©IvyDataScience
CognitiveToolkit• Microsoft opensourcedeeplearningframework(Jan25,2016)• Version2.0releasedOct25,majorupgrade• RenamedCNTKtoMicrosoftCognitiveToolkit• AnnouncedpartnershipwithNvidia andOpenAI(ElonMuskbackedAIstartup),Nov16• LanguagesarePython,C++orBrainScript• CanrunonAzureGPU’s• https://www.microsoft.com/en-us/research/product/cognitive-toolkit/
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 40©IvyDataScience
Torch• Firstreleasedin2000,withover50,000downloads,companyusersincludeGoogle,Facebook,Twitter• ThegoalofTorchistohavemaximumflexibilityandspeedinbuildingscientificalgorithmswhilemakingtheprocessextremelysimple• TorchisaneuralnetworklibrarywritteninLuawithaC/CUDAinterfaceoriginallydevelopedbyateamfromtheSwissinstituteEPFL• AttheheartofTorcharepopularneuralnetworkandoptimizationlibrarieswhicharesimpletouse,whilebeingflexibleinimplementingdifferentcomplexneuralnetworktopologies• http://torch.ch/
41© Ivy Data Science
TypesofHardware
• Sensors,processors,storage,memory,network• Processors- CPU,GPU,FPGA,ASIC• GPU - GraphicsProcessingUnitswerefirstbroughttomarketbyNvidia in2007tomeetthedemandsofthegamingmarket• Massivelyparallelprocessing(MPP)• 100xspeedupcomparedwithCPU’s• Widespreadapplication– science,industry,government• Nvidia www.nvidia.com• Intel XeonPhihttp://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html• AMD Radeonwww.amd.com
43IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016
DeepLearningFullstack(Example)
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 46©IvyDataScience
DLaaS - CloudServices
• DL/MLasaServiceisofferedbythemajorcloudproviders• AWS(Amazon) http://aws.amazon.com/• Azure(Microsoft) http://azure.microsoft.com/• GCP(Google) https://cloud.google.com/• Bluemix (IBM) https://www.ibm.com/cloud-computing/bluemix/
© Ivy Data Science 47
AIConferences– BusinessFocussed
• O’ReillyAIhttp://conferences.oreilly.com/artificial-intelligence/ai-ny• AIFrontiershttp://www.aifrontiers.com/• AIWorldhttp://aiworldexpo.com/program/• AIEuropehttp://ai-europe.com/• AISummithttps://theaisummit.com/london/• Re:Work https://www.re-work.co/events/• WorldofWatsonhttps://www-01.ibm.com/software/events/wow/
© Ivy Data Science 48
AIConferences– ResearchFocussed• NIPS=NeuralInformationProcessingSystemshttps://nips.cc/• IJCNN=InternationalJointConferenceonNeuralNetworkshttp://www.ijcnn.org/• IJCAI=InternationalJointConferenceonArtificialIntelligencehttp://ijcai.org/• ICANN=InternationalConferenceonArtificialNeuralNetworkshttp://www.icann2017.org/• IWANN=InternationalWork-ConferenceonArtificialNeuralNetworkshttp://iwann.uma.es/• ICONIP=InternationalConferenceonNeuralInformationProcessinghttp://www.iconip2017.org/papers.html• ICAART=InternationalConferenceonAgentsandArtificialIntelligencehttp://www.icaart.org/• ISIS=InternationalSymposiumonAdvancedIntelligentSystemshttp://isis2017.org/• AAAI=AssociationofAdvancementofArtificialIntelligencehttp://www.aaai.org/Conferences/conferences.php• ACM=AssociationofComputingMachineryhttps://www.acm.org/conferences• AGI=ArtificialGeneralIntelligenceConferencehttp://agi-conf.org/• TensorCon =TensorFlowConferencehttps://ti.to/TensorCon/
© Ivy Data Science 49
AIinHealthcare- Market
• $600min2014,expectedtoreach$6bnby2021- Frost&Sullivan• Movedfrompilotsandproofofconcepts,tocommercialisation,adoption,andutilisation
• Drivenbyrisingmedicalcostsandincreasingvolumesofdata• Helpingphysiciansmakemoreinformedandaccuratedecisionsaboutpatientcare
• Evaluatesandinterpretsmedicaldata,offersrecommendations,andmakespredictions
• Unlockhiddeninsightswithinthedatawithdeepsemanticunderstandingofthecontent
• Fightdiseaseslikecancer,heartstroke,autismandParkinsons
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 52©IvyDataScience
WatsonTacklesCancer
• Watsonwastestedon1,000cancerdiagnosesmadebyhumanexperts.In99%ofthem,Watsonrecommendedthesametreatmentasthe oncologists• In30%ofthecases,Watsonalsofoundatreatmentoptionthehumandoctorsmissed• Sometreatmentswerebasedonresearchpapersthatthedoctorshadnotread— morethan160,000cancerresearchpapersarepublishedayear• Othertreatmentoptionssurfacedinnewclinicaltrialstheoncologistshadnotyetseenannouncedontheweb
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 53©IvyDataScience
Scan&X-rayDiagnostics
• X-raystodetectfractures• Greaterspeedandaccuracy• Takesworkloadoffdoctors
• Tumor detection• fMRIscans&EEG’s• Anomalydetection
• Eyes,kidneys,lungs,etc.
• Imageclassification• Usingconvolutionalneuralnetworks(CNNs)
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 54©IvyDataScience
The team was able to:• Process 10,900 payments per second• Cut false alarm rates by 50%• Free up resources to combat true fraud
PayPal is improving cybersecurity with deep learning
©IvyDataScience
Fintech
Chatbots (NLP)
• Chatbots,intheformofassistantsandautomatedcustomerservicereps,arebecomingincreasinglycommonacrosstheindustry
• AIisalsoprimedtomakethemassivelycomplicated(anddata-rich)worldoflogisticsmucheasierforretailers
• AIthatcanintuitwhatashopper’sstyleisandadaptitsrecommendationsassheorheshops
• AIthatcanevolveawebsitetospecificconsumerneeds• AIthatcanunderstanduserconcernsandanswercomplicatedquestions
• Makeshoppingbotheasierandmorepersonal
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 56©IvyDataScience
Whereareweheaded?
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 57
World Economic Forum (WEF) Report, 2016:Today, we are at the beginning of a Fourth Industrial Revolution. Developments in genetics, artificial intelligence, robotics, nanotechnology, 3D printing and biotechnology, to name just a few, are all building on and amplifying one another. This will lay the foundation for a revolution more comprehensive and all-encompassing than anything we have ever seen
Deepmind Mission: Solve intelligence then solve everything else
©IvyDataScience
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 58
WhereIvyDataScienceFitsInTheFirstArtificialIntelligenceBootcamp
CertificationfromExperfyHarvardi-lab|PaidConsultingProjects|JobPlacement
© Ivy Data Science
TrainingwithJobPlacement
• LedbyExpertsfromHarvard,Columbia,UMass,UCL• Realtimepaidprojectsinthelastfourweeksofbootcamp• Practicalhands-ontrainingthatwillprepareyouforacareer asaDataScientist,MachineLearningorDeepLearningResearcher/Practitioner
IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 60© Ivy Data Science
TuitionFees
• Online Instructorledcoursesstartingat$1,999- 50hrs• EveningBootcamp startingat$4,999- 50hrs• WeekendBootcamp startingat$6,999- 100hrs• FullImmersive(12week)Bootcamp startingat$11,999*- 320hrs*earlybird• Financingavailable
IvyDataScienceAIwithApplicationsv0.10PeterMorganDec2016 61© Ivy Data Science