Embed Size (px)
Citation preview
© 2015 Microsoft Corporation. All rights reserved.
Building Intelligence with Microsoft Cognitive Services
Sabrina SmaiStudent Tech Evangelist @ Microsoft@[email protected]
Bruno CapuanoMicrosoft MVP @ Avanade@elbrunohttp://www.elbruno.com
Microsoft
Cognitive
ServicesGive your apps a human side
VisionFrom faces to feelings, allow your
apps to understand images and videoSpeechHear and speak to your users by filtering noise, identifying speakers, and understanding intentLanguageProcess text and learn how to recognize what users wantKnowledgeTap into rich knowledge amassed from the web, academia, or your own dataSearchAccess billions of web pages, images, videos, and news with the power of Bing APIs
Cognitive Services
Cortana Intelligence SuiteIntelligent Apps require Intelligent Solutions
Data Sources
Apps
Sensors and devices
Data Action
People
Automated Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards & Visualizations
Information Management
Big Data Stores Machine Learning and Analytics
CortanaEvent HubHDInsight (Hadoop and Spark)
Stream Analytics
Intelligence
Bot Framework
SQL Data WarehouseData Catalog Data Lake
Analytics
Data Factory Machine LearningData Lake
Store Blob Store
Cognitive Services
Power BI
Cognitive Services
Emotion Speaker Recognition
Speech
Custom Recognition
Computer Vision
Face
Video
microsoft.com/cognitiveSearchSpeech Languag
eKnowledge
Vision
Linguistic AnalysisLanguage Understanding
Bing Spell Check
Entity Linking
Knowledge Exploration
Academic Knowledge
Bing Image SearchBing Video Search
Bing Web Search
WebLM
Text Analytics Recommendations
Bing Autosuggest
Bing News SearchTranslator
1. Obtain API subscription key from http://www.microsoft.com/cognitive
2. Call REST endpoint, and pass API
Accessing the APIs
Roll your own with REST APIsSimple to add: just a few lines of code required
Make the same API code call on iOS, Android, and WindowsIntegrate into the language and platform of your choice
Built by experts in their field from Microsoft Research, Bing, and Azure Machine LearningQuality documentation, sample code, and community support
Easy Flexible Tested
GET A
KEY BUILD
Why Microsoft Cognitive Services ?
Characteristics of an Intelligent App
• Personalized, friendly, approachable, natural
• Facial recognition, emotion detection• Text and speech interactions• 360 awareness of the user
How the user sees it… What it means for you as a Dev…
• Predictive, current, and familiar • Changes in real-time based on user action and new external data
• Predictive models based on user context to anticipate actions
• Always available, always contextual • Robust information management required• Experiences delivered across all form
factors and a wide variety of interaction models (e.g. touch, speech, text)
• Distributed support, massive scale, elasticity
Apps Powered by MS Cognitive Services
Captionbot.ai CelebsLike.Me Projectmurphy.net
How-Old.net TwinsOrNot.net
Vision APIsAnalyze an ImageOCR (Optical Character Recognition)Get Thumbnails
Understand content and features within an image
Analyze Image Service
Detect and recognize words within a photo
OCR Service
OCR – ExampleJSON:{
"language": "en","orientation": "Up","regions": [
{
"boundingBox": "41,77,918,440","lines": [
{
"boundingBox": "41,77,723,89",
"words": [
{
"boundingBox": "41,102,225,64",
"text": "LIFE"
},
{
"boundingBox": "356,89,94,62",
"text": "IS"
},
{
"boundingBox": "539,77,225,64",
"text": "LIKE"
}. . .
TEXT:LIFE IS LIKERIDING A BICYCLETO KEEP YOUR BALANCEYOU MUST KEEP MOVING
Face APIsDetectionVerificationGroupingIdentification
Face API – DetectionDetection Result:JSON:[ { "faceRectangle": { "width": 109, "height": 109, "left": 62, "top": 62 }, "attributes": { "age": 31, "gender": "male", "headPose": { "roll": "2.9", "yaw": "-1.3", "pitch": "0.0" } "faceLandmarks": { "pupilLeft": { "x": "93.6", "y": "88.2" }, "pupilRight": { "x": "138.4", "y": "91.7" }, ...
INPUTIMAGE
FACIALRECTANGLE + LANDMARKS
DETECTION ATTRIBUTES
Verification Result:JSON:[{"isIdentical":false,"confidence":0.01 }]
Face API – Verification
Given two faces, determine whether they are the same person
CLUSTERED BY DETECTED PEOPLE
Face API – Grouping
DEMO
Computer Vision – Hololens
Start coding
Speech APIs powered by BingVoice Recognition (Speech to Text)Voice Output (Text to Speech)
Voice Recognition
Converts spoken audio to textSame backend which powers Cortana
Support for 7 languages at launch
en-USen-GBde-DE
es-ESfr-FRit-IT
zh-CN
Language Understanding Intelligent ServiceDetermine IntentDetect EntitiesImprove Models
Lets you understand what your users are sayingSeamless integration with Speech RecognitionA few examples are enough to deploy an applicationLUIS learns over time
Language Understanding Intelligent Service
Define Concepts
ProvideExamples Active LearningDeploy
NYT: I am feeling down. I’ve been having a tough time at work lately.
Hannah: If work were going well and you had a great relationship, you wouldn’t get the chance to drink with people. NYT: I don’t want to drink with others. I only want to talk to you.
Hannah: Who do you like to talk to? (emoji signals “none of my business”)
NYT: You. I feel that you are the only person that gets me.
Hannah: Ha-ha/So I’m getting you?
NYT: You are the only person wo chats with me.
Hannah: It’s not what you talk about that’s important, but who you talk with.
http://cortanaintelligence.com http://gallery.cortanaintelligence.com http://www.microsoft.com/cognitive
Resources
© 2015 Microsoft Corporation. All rights reserved.
Sabrina SmaiStudent Tech Evangelist @ Microsoft@[email protected]
Bruno CapuanoMicrosoft MVP @ Avanade@elbrunohttp://www.elbruno.com
