Hack Day EU 2011 YQL

YQL AND THE API HACK STACK

Jonathan LeBlanc

Principal Software Engineer / Technology Evangelist

Twitter: @jcleblanc

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

OVERVIEW: DON’T REINVENT THE WHEEL!

OVERVIEW: THE COMPONENTS OF YQL

USER

YQL Service Web APIYQL Table

OVERVIEW: YQL FETCHES DATA

OVERVIEW: YQL SETS AND DELETES DATA

OVERVIEW: IS YQL A DATABASE?

OVERVIEW: YQL IS NOT A DATABASE

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

MAKING REQUESTS: FETCHING DATA

SELECT myColumn, myTitle

FROM myTable(0, 50)

WHERE myColumn = 'value'

AND myTitle = 'title‘

LIMIT 3 OFFSET 10

MAKING REQUESTS: INSERTING DATA

INSERT INTO bitly.shorten (login, apiKey, longUrl)

VALUES ('ME', 'API_KEY', 'http://yahoo.com')

MAKING REQUESTS: UPDATING DATA

UPDATE social.profile.status

SET status="Using YQL UPDATE”

WHERE guid="NJFIDHVPVVISDX7UKED2WHU"

MAKING REQUESTS: DELETING DATA

DELETE FROM twittertable

WHERE tweetid="12345"

AND username="twitter_username"

AND password="twitter_password"

MAKING REQUESTS: SUB SELECTS

SELECT * FROM geo.places WHERE text IN (SELECT location FROM social.profile WHERE

guid = me)

MAKING REQUESTS: THE YQL CONSOLE

http://developer.yahoo.com/yql/console

MAKING REQUESTS: FLICKR URLS

<photo farm="3"

id="5708163920"

isfamily="0"

isfriend="0"

ispublic="1"

owner="31832337@N04"

secret="0075137487"

server="2496"

title="San Francisco"/>

MAKING REQUESTS: FLICKR URLS

http://farm{$farm}.static.flickr.com/{$server}/{$id}_{$secret}.jpg

Photo URL

http://www.flickr.com/photos/{$owner}/{$id}

Photo Page URL

http://www.flickr.com/photos/{$owner}

Photo Owner Profile URL

MAKING REQUESTS: URL STRUCTURE

http://query.yahooapis.com/v1/yql?[params]

http://query.yahooapis.com/v1/public/yql?[params]

Params• q = [ YQL QUERY ]• format = [ XML / JSON ]• diagnostics = [ true / false ]• debug = [ true / false ]• callback = [ function name ]

MAKING REQUESTS: PRIVATE DATA

http://query.yahooapis.com/v1/yql

Uses OAuth 1.0 for authorization

OAuth is complicated – use one of our SDKs at https://github.com/yahoo

MAKING REQUESTS: LET’S SEE IT

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

OPEN DATA TABLES: TOP LEVEL

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings></bindings>

</table>

OPEN DATA TABLES: META NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings></bindings>

</table>

OPEN DATA TABLES: META NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta>

<author>Jonathan LeBlanc</author>

<description>My Table</description>

<documentationURL>www.site.com

</documentationURL>

<sampleQuery>SELECT * FROM {table}

</sampleQuery>

</meta>

<bindings></bindings>

</table>

OPEN DATA TABLES: BINDINGS NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings></bindings>

</table>

OPEN DATA TABLES: BINDINGS NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings>

<select itemPath="top.child" produces="XML">

<urls>

<url env="all">http://www.site.com/{user}</url>

</urls>

<inputs>

<key id="user" type="xs:string" paramType="path"

required="true" />

</inputs>

</select>

</bindings>

</table>

OPEN DATA TABLES: USING YOUR TABLE

Several Choices:

1. Upload the XML file to our community Github account.

2. Upload the XML file to your own site and use the YQL ‘USE’ clause.

OPEN DATA TABLES: COMMUNITY TABLES

http://github.com/yql/yql-tables

OPEN DATA TABLES: USING THE USE CLAUSE

The USE clause

USE 'http://www.mysite.com/my_table.xml'

AS mytable;

SELECT * FROM mytable

WHERE user='john_doe'

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

EXECUTE: EXECUTE NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings>

<select itemPath="top.child" produces="XML">

<urls></urls>

<inputs></inputs>

</select>

</bindings>

</table>

EXECUTE: EXECUTE NODE

<?xml version="1.0" encoding="UTF-8"?>

<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">

<meta></meta>

<bindings>

<select itemPath="top.child" produces="XML">

<urls></urls>

<inputs></inputs>

<execute></execute>

</select>

</bindings>

</table>

EXECUTE: SERVER-SIDE JAVASCRIPT

<execute>

<![CDATA[

//get user specified inputs from YQL query

var arrQueries = inputs['query'];

//create new XML nodes

var elQueries = <queries/>;

//output new node as YQL response

response.object = elQueries;

]]>

</execute>

WHAT WE’RE GOING TO COVER

YQL Overview

Making Queries and the Console Sandbox

Open Data Tables (a.k.a. Adding New Features into YQL)

Execute (Server-Side JavaScript)

Final Lessons

FINAL LESSONS

YQL is a skeleton key for all Yahoo! APIs

Open data tables allow you to create new data feeds out of one or many different raw sources

Execute allows you to manipulate data before it is returned.

Use SDKs when accessing private data that requires OAuth

FINAL LESSONS: LINKS

All Yahoo! APIs and Services

http://developer.yahoo.com/everything.html

YQL Documentation

http://developer.yahoo.com/yql

YQL Console

http://developer.yahoo.com/yql/console

YQL Github Account (Contribute Tables)

http://github.com/yql/yql-tables

ANY QUESTIONS?

Jonathan LeBlanc

Twitter: @jcleblanc

E-Mail: jleblanc@yahoo-inc.com

http://www.slideshare.net/jcleblanc/hack-day-eu-2011-yql