Embed Size (px)
DESCRIPTION
Citation preview
BREAKING DOWN “DATA SILOS” WITH �
THE OPEN DATA PROTOCOL (ODATA)
Chris Woodruff
WHO AM I?
CHRIS WOODRUFFMVP, Data Platform
Development
[email protected]://chriswoodruff.comhttp://deepfriedbytes.comhttp://nplus1.orgtwitter @cwoodruff
Today, the tools for capturing data both at the mega-scale and at the milli-scale are just dreadful. After you have captured the data, you need to curate it before you can start doing any kind of data analysis, and we lack good tools for both data curation and data analysis.
Jim Gray
QUOTE
Click icon to add picture
WHAT IS DATA NOW?
WHAT COULD DATA BE TOMORROW?
An architectural style for building loosely coupled systems defined by a set of very general constraints (principles) the Web (URI/HTTP/HTML/XML) is an instance of this style
The Web used correctly (i.e., not using the Web as transport) HTTP is built according to RESTful principles services are built on top of Web standards without misusing
them most importantly, HTTP is an application protocol (not a
transport protocol)
Anything that uses HTTP and XML (XML without SOAP) XML-RPC was the first approach for this violates REST because there is no uniform interface
REST: 3 DEFINITIONS
The name Atom applies to a pair of related standards. The Atom Syndication Format is an XML language used for web feeds, while the Atom Publishing Protocol (AtomPub) is a simple HTTP-based protocol for creating and updating web resources.
ATOMPUB
http://services.odata.org/OData/OData.svc\_______________________________________/ | service root URI
http://services.odata.org/OData/OData.svc/Category(1)/Products?$top=2&$orderby=name\_______________________________________/ \__________________/ \_________________/ | | | service root URI resource path query options
WHAT IS A URI?
Creation of standard Web API’s
Allow internal Information Workers to be more productive
Give IT staff more time to work on meaningful projects
Monetize/Open Data for others to use and benefit from
WHAT ARE THE BENEFITS OF
ODATA?
THE BEAUTY OF METADATA
THE BASICS
• Feeds, which are Collections of typed Entries
• OData services can expose Service Operations
• OData services expose all these constructs via URIs
• OData service may also expose a Service Metadata Document
Type Example DescriptionBinary <d:foo
m:type="Edm.Binary">FRwvAAI...</d:foo>Fixed or variable length binary data in Base64 encoded format.
Boolean <d:foo m:type="Edm.Boolean">true</d:foo> Represent the mathematical concept of binary valued logic.
Byte <d:foo m:type="Edm.Byte">1</d:foo> An unsigned 8-bit integer value
DateTime <d:foo m:type="Edm.DateTime">2010-02-26T17:08:53.0900752-08:00</d:foo>
A date and time with value ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D.
DateTimeOffset <d:foo m:type="Edm.DateTimeOffset">2002-10-10T17:00:00Z</d:foo>
Date and time as an Offset in minutes from GMT, with values ranging from 12:00:00 midnight, January 1, 1753 A.D. through 11:59:59 P.M, December 9999 A.D.
Time <d:foo m:type="Edm.Time">P120D</d:foo> Time duration.
Decimal <d:foo m:type="Edm.Decimal">3.3</d:foo> A numeric value with fixed precision and scale from -10^255 + 1 to 10^255 - 1.
Double <d:foo m:type="Edm.Double">4.4</d:foo> A floating point number with 15 digits precision that can represent values with approximate range of ± 2.23e -308 through ± 1.79e +308.
Single <d:foo m:type="Edm.Single">4.4</d:foo> A floating point number with 7 digits precision that can represent values with approximate range of ± 1.18e -38 through ± 3.40e +38.
Guid <d:foo m:type="Edm.Guid">223b00d9-e617-4a80-bf18-31bfd6e8e312</d:foo>
A 16-byte (128-bit) unique identifier value
Int16 <d:foo m:type="Edm.Int16">1</d:foo> Signed 16-bit integer value.
Int32 <d:foo m:type="Edm.Int32">2</d:foo> Signed 32-bit integer value.
Int64 <d:foo m:type="Edm.Int64">3</d:foo> Signed 64-bit integer value.
String <d:foo>Beverages</d:foo><d:foo m:type="Edm.String">Beverages</d:foo>
Fixed or variable length character data. String is the default type.
DATATYPES
FULL SQL LIKE QUERY “LANGUAGE”
HTTP Command (Verb)
SQL Command
GET SELECT
PUT UPDATE
POST INSERT
DELETE DELETE
Click icon to add picture
How we get around OData
QUERY NAVIGATION
Addressing entities and sets
Presentation options
Entity-set /Bookmarks
Single entity /Bookmarks(123)
Member access /Bookmarks(123)/Name
Link traversal /Bookmarks(123)/Tags
Deep access /Bookmarks(123)/Tags('abc')/Name
Raw value access /Bookmarks(123)/Names/$value
Sorting /Bookmarks?$orderby=Name
Filtering /Bookmarks?$filter=Created gt '2007-05-07'
Paging /Bookmarks?$top=10&$skip=30
Inline expansion /Bookmarks?$expand=Tags
Query Option
Description
$expand Expand related data inline with the results, e.g. Categories/$expand=Products would expand Product data inline with each Category entry.
$filter A Boolean expression for whether a particular entry should be included in the feed, e.g. Categories?$filter=CategoryName eq 'Produce'. The Query Expression section describes OData expressions.
$format One of “atom” (the default), “json” or “xml” depending on how you’d like your data returned to you.
$inlinecount
Includes the number of entries without the filter applied as a count element on the feed itself, e.g. Categories?$top=4&$inlinecount=allpages will return 8 with the default Northwind database installed.
$orderby One or more comma-separated expressions with an optional “asc” (the default) or “desc” depending on the order you’d like the values sorted, e.g. Categories?$orderby=CategoryName desc.
$select Limit the properties on each entry to just those requested, e.g. Categories?$select=CategoryName,Description.
$skip How many entries you’d like to skip, e.g. Categories?$skip=4.$skiptoken
Skips entries up to one past the one that matches the unique key. Used for paged access into a feed. Best used with $orderby, e.g. Categories?$orderby=CategoryID&$skiptoken=5 will skip everything ‘til it reaches the CategoryID of 5, then return entries past that.
$top Return entries from the top of the feed, e.g. Categories?$top=4
QUERY OPTIONS
Operation Example DescriptionGrouping (x add 4) eq 3 Used to make clear or override default operator
precedence (which I can never remember anyway).
Member access Categories(3)/Description
Access of entity by ID or property by name.
Method call startswith(Description, “Bread”)
Call a built in method.
Negate -x Change numeric sign.Not not x Logical not.And x and y Conditional and.Or x or y Conditional or.Multiply x mul y Multiplication.Divide x div y Division.Modulo x mod y Remainder.Add x add y Addition.Subtract x sub y Subtraction.Less than x lt y Comparison.Greater than x gt y Comparison.Less than or equal
x le y Comparison.
Greater than or equal
x ge y Comparison.
Equals x eq y Comparison.Not equals x ne y Comparison.
FILTER EXPRESSIONS
Method Example Descriptionendswith endswith(x, 'foo') Whether the end of the first parameter value matches the
second parameter value.indexof indexof(x, 'foo') Index of the first occurrence of the second parameter value in
the first parameter value or -1 otherwise.replace replace(x, 'foo',
'bar')String value with all occurrences of the second parameter value replaced by the third parameter value in the first parameter value.
startswith startswith(x, 'foo') Whether the beginning of the first parameter values matches the second parameter value.
tolower tolower(x) String value with the contents of the parameter value converted to lower case.
toupper toupper(x) String value with the contents of the parameter value converted to upper case.
trim trim(x) String value with the contents of the parameter value with all leading and trailing white-space characters removed.
substring substring(x)substring(x, y)
String value starting at the character index specified by the second parameter value in the first parameter string value.
substringof substringof(x)substringof(x, y)
Whether the second parameter string value occurs in the first parameter string value.
concat concat(x)concat(y)
String value which is the first and second parameter values merged together with the first parameter value coming first in the result.
length length(x) The number of characters in the specified parameter value.
FILTER METHODS
Examples
http://.../OData.svc/GetProductsByRating?rating=5
Identifies the "GetProductsByRating" Service Operation and specifies a value of 5 for the "rating" input parameter.
SERVICE OPERATIONS(FUNCTIONS)
WHERE CAN WE PRODUCE ODATA?
WCF Data Services
WCF RIA SERVICES
Data Access LayerEntity
Framework
Custom LINQ
provider
Relational
databaseOther
sources
Service Runtime
Hosting/HTTP listener
HTTP
IQueryable (+ IUpdatable)
Create services directly from Visual Studio
Various data sourcesEntity FrameworkLINQ providers
Model-drivenStructural description in Entity Data Model
Metadata shapes service
Visual Studio 2010SQL Server 2008 SP1ASP.NET 4.0
WCF Data Services Framework uses ASP.NET for Authentication and Authorization
WHAT DO YOU NEED FOR WCF DATA SERVICES?
WHERE CAN WE CONSUME ODATA?
LINQ
CONSUMING FROM .NET APPLICATIONS
Demo
OPEN YOUR DATA FOR OTHERS
MORE INFORMATION
WCF Data Services Framework Home -- http://tinyurl.com/wcfdataservices
WCF Data Services Framework Team Blog -- http://blogs.msdn.com/astoriateam
ODataPrimer.com – http://odataprimer.com
NOTE ABOUT INTERNET EXPLORER
Note, Internet Explorer by default renders ATOM in a friendly format that makes OData feeds unreadable. To fix this, turn off IE feed viewing by turning off the option
Tools -> Internet Options -> Content -> Feeds (settings) -> Turn on feed viewing
Click icon to add picture
Required Reading
WRAP UP
The Fourth Paradigm: Data-Intensive Scientific Discovery
MORE REQUIRED READING
