Smarter & Cuter Bots

Smarter & Cuter Bots

About Me!Rachel White

@ohhoeTechnical Evangelist @ Microsoft

I love catsI love cute thingsI love twitter bots!

Smarter & Cuter Bots -@ohhoe

About Me!RoboKitty – Automated Cat Feeder

https://github.com/rachelnicole/robokitty


About Me!Glitch Art!


About Me!Artist Residency @ Pioneer Works


About Me!Video Games


About Me!Cute Twitter Bots


Notable Bot Types

Machine LearningMarkov Chain

Corpus FedGenerative Image Bots

Just Plain ~ Weird ~


Machine LearningWhat is it?


“Machine learning is the subfield of computer science that gives computers the ability to learn without being explicitly programmed .”

Arthur Samuel, 1959

Machine LearningWhat is it?


Machine Learning


Within the field of data analytics, machine learning is a method used to devise complex models and algorithms that lend themselves to prediction; in commercial use, this is known as predictive analytics.

Source: https://en.wikipedia.org/wiki/Machine_learning

Machine Learning


These analytical models allow researchers, data scientists, engineers, and analysts to "produce reliable, repeatable decisions and results" and uncover "hidden insights" through learning from historical relationships and trends in the data.

Source: https://en.wikipedia.org/wiki/Machine_learning

Source: http://freakonometrics.hypotheses.org/20274

Machine Learning


Computer Vision, Predictive Text, Emotional Analysis, Facial Detection, Deep Learning, Neural Networks, etc etc etc etc and so on

Machine Learning


There’s a lot.But they’re still trained models, they still require some sort of input from a human being.

Natural Language Processing


Modern NLP algorithms are based on machine learning, especially statistical machine learning. The paradigm of machine learning is different from that of most prior attempts at language processing. Prior implementations of language-processing tasks typically involved the direct hand coding of large sets of rules.

Source: https://en.wikipedia.org/wiki/Natural_language_processing#Using_machine_learning



The machine-learning paradigm calls instead for using general learning algorithms — often, although not always, grounded in statistical inference — to automatically learn such rules through the analysis of large corpora of typical real-world examples. A corpus (plural, "corpora") is a set of documents (or sometimes, individual sentences) that have been hand-annotated with the correct values to be learned.

Source: https://en.wikipedia.org/wiki/Natural_language_processing#Using_machine_learning



Types of NLP:Linguistic AnalysisLanguage UnderstandingText AnalyticsAnalyzing Tone

Markov ChainWhat is it?


“a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.”

Markov ChainOh.


Source: http://setosa.io/ev/markov-chains/

With two states (A and B) in our state space, there are 4 possible transitions (not 2, because a state can transition back into itself). If we're at 'A' we could transition to 'B' or stay at 'A'. If we're at 'B' we could transition to 'A' or stay at 'B'.

Markov Chain


Source: http://setosa.io/ev/markov-chains/

Corpus FedWhat is it?


“In linguistics, a corpus (plural corpora) or text corpus is a large and structured set of texts (nowadays usually electronically stored and processed). They are used to do statistical analysis and hypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.”

Source: https://en.wikipedia.org/wiki/Text_corpus

Corpus FedWhat is it?


Often used in Natural Language Processing, but more often used with generic twitter bots that combine or utilize text snippets in a more generalized way.

Corpus Fed


Source: https://github.com/dariusk/corpora

Corpus Fed


Source: https://github.com/rachelnicole/isReallyCute-bot

Generative ImagesWhat is it?


SVG generative through traceryCompiled through Image Magick / Graphics MagickAltering raw image binaries

Generative Images


Source: @amandaglosson made @tinyneighbor

Generative Images


Source: @v21 made @softlandscapes


Source: http://cheapbotsdonequick.com/source/softlandscapes

WeirdWhat is it?


Idk. Just weird stuff

Seriously though, a lot of corpus fed text, random images, and general nonsensical markov chains for the purpose of being weird and fun.


Source: @deathmtn made @cleromancer

Weird


Source: @ckolderup made @OminousZoom

So What Are We Making?


Twitter Bot! ( obviously )Utilizing Twitter Streaming APIsGraphicsMagickMicrosoft’s Cognitive Services – specifically the Face API

Twit


Make sure you have node & npm installedhttps://github.com/ttezel/twitSign up for twitter app for auth tokens

Graphics Magick


GraphicsMagick is a tool to create, edit, and compose / composite images. It’s super versatile, you can access it from the cli or C, C++, Lua, Perl, PHP, Python, Tcl, Ruby, Windows .NET, or Windows COM programming interfaces.There’s a node port of it you can use via npm install gm.ButIt’s kind of a pain in the ass

Source: http://www.graphicsmagick.org/

Graphics Magick


Down / Up Sampling of images are kind of poor qualityIt requires some hacky workaroundsIn node, you’ve got to use a ton of callbacks to do multiple compositing.To host it on a server you have to be able to install some binaries L

Source: http://www.graphicsmagick.org/

Microsoft Cognitive Services


It’s rad.Whole suite of Machine Learning REST APIsComputer Vision, Content Moderator, Emotion, Face, Video, Bing Speech, Custom Recognition, Speaker Recognition, Bing Spell Check, Language Understanding, Linguistic Analysis, Text Analytics, Translator, WebLM, Academic, Entity Linking, Knowledge Exploration, Recommendations, Lots of Bing things.

Source: https://www.microsoft.com/cognitive-services/

Microsoft Cognitive Services


It’s super easy to useIf you know how to hit an endpoint, you can use everything.We’re going to use the Face API

Source: https://www.microsoft.com/cognitive-services/

Microsoft Face API


Detect one or more human faces in an image and get back face rectangles for where in the image the faces are, along with face attributes which contain machine learning-based predictions of facial features.The face attribute features available are: Age, Gender, Pose, Smile, and Facial Hair along with 27 landmarks for each face in the image.

Source: https://www.microsoft.com/cognitive-services/en-us/face-api

Microsoft Face API


Let’s try it out.

Source: https://www.microsoft.com/cognitive-services/en-us/face-api

Okay, but Rachel….


What exactly are we making?

Magical and Cute Image Bot!!!



~ Code Demo Time ~


Magical and Cute Image Bot


I Launched it in November!


People immediately tried to break it!

Sometimes it glitched out

I’m Launching it Right NOW!!!!!!!!


Thank You!


Follow me on twitter: @ohhoehttp://rachelisaweso.mehttp://imcool.onlineGithub Source: https://github.com/rachelnicole/magicalncutehttp://aka.ms/smarterandcuter

Internet

Smarter & Cuter Bots