Upload
chris-schalk
View
114
Download
4
Tags:
Embed Size (px)
DESCRIPTION
Presentation given by Google Developer Advocate Chris Schalk on building integrated applications with Google's Cloud Technologies.
Citation preview
Building Integrated Applications on Google's Cloud TechnologiesDevFest 2011Jakarta
Christian SchalkGoogle Developer Advocateprofiles.google.com/cschalk@cschalk
Agenda
● Introduction
● App Engine Recap
● Google's new Cloud Technologies○ Google Cloud Storage○ Prediction API○ BigQuery○ Google Cloud SQL
● Summary Q&A
Introduction
Google App Engine
Introduction
Google BigQuery Google Prediction API
Google Cloud Storage
Google App Engine
Google Cloud SQL
New Google Cloud Technologies
● Google Cloud Storage○ Store your data in Google's cloud
● Prediction API○ Google's machine learning tech in an API
● BigQuery○ Hi-speed data analysis on a massive scale
● Cloud SQL○ A relational database in the cloud
Google Cloud StorageStore your data in Google's cloud
What Is Google Cloud Storage?
● Store your data in Google's cloud○ any format, any amount, any time
● You control access to your data○ private, shared, or public
● Access via Google APIs or 3rd party tools/libraries
Sample Use Cases
Static content hostinge.g. static html, images, music, video Backup and recoverye.g. personal data, business records Sharinge.g. share data with your customers Data storage for applicationse.g. used as storage backend for Android, AppEngine, Cloud based apps Storage for Computatione.g. BigQuery, Prediction API
Google Cloud Storage Benefits
High Performance and Scalability Backed by Google infrastructure
Strong Security and Privacy Control access to your data
Easy to UseGet started fast with Google & 3rd party tools
Google Cloud Storage Technical Details● RESTful API
○ Verbs: GET, PUT, POST, HEAD, DELETE ○ Resources: identified by URI○ Compatible with S3
● Buckets ○ Flat containers
● Objects ○ Any type○ Size: 100 GB / object
● Access Control for Google Accounts ○ For individuals and groups
● Two Ways to Authenticate Requests ○ Sign request using access keys ○ Web browser login
Early Google Cloud Storage Adopters
and more...
Simple Google Cloud Storage Demos
GSutil
GS (Web) Manager
Google Prediction APIGoogle's prediction engine in the cloud
Google Prediction API as a simple example
Predicts outcomes based on 'learned' patterns
Potentially endless number of applications...
CustomerSentiment
TransactionRisk
SpeciesIdentification
MessageRouting
Legal DocketClassification
SuspiciousActivity
Work RosterAssignment
RecommendProducts
PoliticalBias
UpliftMarketing
EmailFiltering
Diagnostics
InappropriateContent
CareerCounselling
ChurnPrediction
... and more ...
Prediction API Capabilities
Data● Input Features: numeric or unstructured text● Output: up to hundreds of discrete categories
Training● Many machine learning techniques● Automatically selected ● Performed asynchronously
Access from many platforms:● Web app from Google App Engine● Apps Script (e.g. from Google Spreadsheet)● Desktop app
How does it work?
"english" The quick brown fox jumped over the lazy dog.
"english" To err is human, but to really foul things up you need a computer.
"spanish" No hay mal que por bien no venga.
"spanish" La tercera es la vencida.
? To be or not to be, that is the question.
? La fe mueve montañas.
The Prediction APIfinds relevantfeatures in the sample data during training.
The Prediction APIlater searches forthose featuresduring prediction.
Using the Prediction API
1. Upload
2. Train
Upload your training data toGoogle Storage
Build a model from your data
Make new predictions3. Predict
A simple three step process...(REST Calls)
Prediction API - key features
● Multi-category prediction○ Tag entry with multiple labels
● Multiple Prediction Output○ Finer grained prediction rankings based on multiple labels
● Mixed Inputs○ Both numeric and text inputs are now supported
Can combine continuous output with mixed inputs
Prediction Demos
● Command line Demos ○ Training a model○ Checking training status○ Making predictions
● A complete Web application using the JavaScript API for Prediction
Google BigQueryInteractive analysis of large datasets in Google's cloud
Introducing Google BigQuery
● Google's large data adhoc analysis technology○ Analyze massive amounts of data in seconds
● Simple SQL-like query language
● Flexible access○ REST APIs, JSON-RPC, Google Apps Script
Many Use Cases ...
Spam TrendsDetection
Web Dashboards
Network Optimization
Interactive Tools
Key Capabilities of BigQuery
● Scalable: Billions of rows
● Fast: Response in seconds
● Simple: Queries in SQL
● Web Service○ REST○ JSON-RPC○ Google App Scripts
Using BigQuery
1. Upload
2. Import
Upload your raw data toGoogle Storage
Import raw data into BigQuery table
Perform SQL queries on table
3. Query
Another simple three step process...
(REST Calls)
Writing Queries
Compact subset of SQL○ SELECT ... FROM ...
WHERE ... GROUP BY ... ORDER BY ...LIMIT ...;
Common functions○ Math, String, Time, ...
Statistical approximations○ TOP○ COUNT DISTINCT
BigQuery via REST
GET /bigquery/v1/tables/{table name}
GET /bigquery/v1/query?q={query}
Sample JSON Reply:{ "results": { "fields": { [ {"id":"COUNT(*)","type":"uint64"}, ... ] }, "rows": [ {"f":[{"v":"2949"}, ...]}, {"f":[{"v":"5387"}, ...]}, ... ] }}Also supports JSON-RPC
BigQuery Security and Privacy
Standard Google Authentication● Client Login● AuthSub● OAuth
HTTPS support● protects your credentials● protects your data
Relies on Google Storage to manage access
What's New in BigQuery V2?
● A new REST API● A new web user interface● Support for JOIN statements● Export table or query result to a CSV file in Google
Cloud Storage● Support for ACLs on collections of tables● A new object architecture describing tables, groups of
tables, and queries. This new architecture is described under Main Concepts below.
New!
BigQuery Demo
BigQuery Web Browser
Google Cloud SQLA relational database in the cloud
Google Cloud SQL
● Developer console○ Easy to use
● Fully managed
● High availability○ Synchronous replication to multiple data centers
● Integrated with Google App Engine○ Java: JDBC, Python: DB-API○ Use with High Replication Datastore
● MySQL Compatible○ Import / export
Cloud SQL Demos!
● Cloud SQL Console● SQL via JDBC Access● SQL from PHP!
Recap
● Google App Engine○ General purpose application development platform for
the cloud
● Google Cloud Storage○ High speed cloud data storage on Google's
infrastructure
● Prediction API○ Google's machine learning technology
● BigQuery○ Interactive analysis of very large data sets
● Google SQL○ A relational SQL database in the cloud
Further info available at:● Google App Engine
○ http://code.google.com/appengine
● Google Cloud Storage○ http://code.google.com/apis/storage
● Prediction API○ http://code.google.com/apis/predict
● BigQuery○ http://code.google.com/apis/bigquery
● Cloud SQL○ http://code.google.com/apis/sql
Thank You!
Christian SchalkGoogle Developer Advocateprofiles.google.com/cschalk@cschalk
(Follow me on G+ to get the slides!)