Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs
Defer and Stream Directives in GraphQLImprove Latency with Incremental Delivery
About
• Online marketplace for luxury items across multiple verticals
• Front-End stack: Node, GraphQL, React, and Relay
• Offices in New York, Vilnius, Lithuania, and Bangalore, India
What are @defer and @stream?
• @defer and @stream are proposed directives to the GraphQL Specification to support incremental delivery for state-less queries
• Championing since January 2020
• In collaboration with GraphQL Working Group
• In this talk, we will discuss:
• Motivation
• Specification proposal overview
• Code Examples
• Reference implementation in GraphQL.js
• Open-source contribution
• Best practices
query TalksQuery { talks(first: 6) @stream ( label: "talkStream", initialCount: 3 ) { name ...TalkComments @defer(label: "talkCommentsDefer") } }
fragment TalkComments on Talk { comments { body } }
Why @defer and @stream?
• Large datasets may suffer from latency
• All requested data may not be of equal importance
• Current options for applications to prioritize data, such as query splitting and pre-fetching, come with undesirable trade-offs
• @defer and @stream would allow GraphQL clients to communicate priority of requested data to the server without undesirable trade-offs
Query Splitting
• Fetch expensive/non-essential fields in a separate query after initial query
• Trade-offs:
• Increased latency for lower priority fields
• Client resource contention
• Increased server cost
/** Original Query */query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture }}
fragment SpeakerPicture on Speaker { picture { height width url } }
/** Split Queries */query SpeakerInitialQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name }}
query SpeakerFollowUpQuery($speakerId: String!) { speaker(speakerId: $speakerId) { ...SpeakerPicture }}
Pre-fetching
• Optimistically fetching data based on a prediction that a user will execute an action
• Trade-offs:
• Increased server cost due to incorrect predictions
https://www.apollographql.com/blog/introducing-defer-in-apollo-server-f6797c4e9d6e/
What about Subscriptions?
• Intention is for real-time and long connections
• @defer and @stream, intention is to lower latency for short-lived connections
Specification Proposal for @defer and @stream
@defer
• The @defer directive may be specified on a fragment spread.
• if: Boolean
• When true fragment may be deferred, if omitted defaults to true.
• label: String
• A unique label across all @defer and @stream directives in an operation.
directive @defer(if: Boolean, label: String) on FRAGMENT_SPREAD
@deferExample
query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture @defer(label: “speakerPictureDefer”) }}
fragment SpeakerPicture on Speaker { picture { height width url } }
// Response Payloads
// Payload 1
{ "data": { "speaker": { "name": "Jesse Rosenberger" } }, "hasNext": true}
// Payload 2{
"label": "speakerPictureDefer", "path": ["speaker"], "data": { "picture": { "height": 200, "width": 200, "url": "jesse-headshot.jpg" } }, "hasNext": false}
@stream
• The @stream directive may be provided for a field of List
• if: Boolean
• When true fragment may be deferred, if omitted defaults to true.
• label: String
• A unique label across all @defer and @stream directives in an operation.
• initialCount: Int
• The number of list items the server should return as part of the initial response.
directive @stream(if: Boolean, label: String, initialCount: Int) on FIELD
@streamExample
query SpeakersQuery { speakers(first: 3) @stream(label: “speakerStream", initialCount: 1) { name }}
// Response Payloads
// Payload 1{ "data": { "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true}
// Payload 2{ "label": "speakerStream", “path": ["speakers", 1], "data": { "name": "Liliana Matos" }, "hasNext": true}
// Payload 3{ "label": "speakerStream", "path": ["speakers", 2], "data": { "name": "Rob Richard" }, "hasNext": false}
Response FormatOverview
• When an operation contains @defer or @stream directives, the GraphQL execution will return multiple payloads
• The first payload is the same shape as a standard GraphQL response
• Any fields that were only requested on a fragment that is deferred will not be present in this payload
• Any list fields that are streamed will only contain the initial list items
Response FormatDetails
• label — The string that was passed to the label argument of the @defer or @stream directive that corresponds to this results
• path — A list of keys from the root of the response to the insertion point
• hasNext — A boolean that is present and true when there are more payloads that will be sent for this operation.
• data — The data that is being delivered incrementally.
• errors — An array of errors that occurred while executing deferred or streamed selection set
• extensions — For implementors to extend the protocol
Response FormatExample
query SpeakersQuery { speakers(first: 2) @stream(label: "speakerStream", initialCount: 1) { name ...SpeakerPicture @defer(label: "speakerPictureDefer") }}
fragment SpeakerPicture on Speaker { picture { url } }
// Response Payloads
// Payload 1{ "data": { "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true}
// Payload 2{ "label": "SpeakerPictureDefer", "path": ["speakers", 0, "picture"], "data": { "url": "jesse-headshot.jpg" }, "hasNext": true}
// Payload 3{ "label": "SpeakerStream", "path": ["speakers", 1], "data": { "name": "Liliana Matos" }, "hasNext": true}
// Payload 4{ "label": "SpeakerPictureDefer", "path": ["speakers", 1, "picture"], "data": { "url": "liliana-headshot.jpg" }, "hasNext": false}
How to use @defer and @stream in your GraphQL Server
• @defer - existing resolvers will work effectively
• @stream - need to consider how resolvers return data
Execution
fragment SpeakerPicture on Speaker { picture { height width url } }
{ "data": { "speaker": { "name": "Jesse Rosenberger" } }, "hasNext": true}
query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture @defer(label: “speakerPictureDefer”) }}
{
"label": "speakerPictureDefer", "path": ["speaker"], "data": { "picture": { "height": 200, "width": 200, "url": "jesse-headshot.jpg" } }, "hasNext": false}
Fork execution to dispatcher
Initial Payload
Subsequent Payload
How to use @stream in your GraphQL Server
• Any List field can use the @stream directive
• What you return from your resolver matters
List return types in GraphQL-JS• GraphQL-JS supports returning several different data types in List resolvers.
• Array<T>, any Iterable, Promise<Iterable<T>>
• GraphQL engine will get all results at once
• Initial payload will be held up by this resolver
• Subsequent payloads will be sent immediately after
const resolvers = { Query: { items: async function (_, { filters }): Promise<Array<Item>> { const items = await api.getFilteredItems({ filters }); return items; }, },};
List return types in GraphQL-JS
• Array<Promise<T>>
• GraphQL engine will start waiting for all results
• Initial payload will be sent as soon as the "initialCount" values are ready
• Subsequent payloads will be sent as each promise resolves
• Requires knowing how many results there will be before the resolver returns
Returning Array<Promise<T>>
const resolvers = { Query: { items: async function (_, { filters }): Array<Promise<<Item>> { const itemIds = await api.filterItems({ filters }); return itemIds.map(async itemId => await api.getItemById(itemId)); }, },};
List return types in GraphQL-JS
• AsyncIterable, Async Generator function
• GraphQL engine will yield each result from the iterable
• Initial payload will be sent as soon as the "initialCount" values are ready
• Subsequent payloads will be sent as each new value is yielded
• Can determine asynchronously if the list is completed
Async Generator Function Resolverconst resolvers = { Query: { users: async function* (): AsyncIterable<User> { const db = new Database(); while (true) { // select one document const result = await db.getNext();
// end iteration if there are no documents returned if (!result) { break; }
yield result; }
return; }, },};
Server - Client Communication
• No websockets or any other stateful connection mechanism
• Works with common infrastructure and old browsers
• Spec is transport-agnostic, so you could use websockets
"Chunked transfer encoding (CTE) is a mechanism in which the encoder sends data to the player in a series of chunks. The player doesn’t have to wait until the complete segment is available"
transfer-encoding: chunked
Multipart HTTP
• Standard for encoding multiple payloads in a single HTTP request response
• Used for File Uploads to attach binary file data to form requests
• Used for Emails to add attachments to email body
• GraphQL Response can be encoded as multipart data of multiple JSON payloads
Multipart HTTP---Content-Type: application/jsonContent-Length: 590{ "data": { "talks": [{ "title": "Opening Keynote", "time": "10:30-11:00", "speaker": { "name": "Jesse Rosenberger" }, }, ...] }}---Content-Type: application/jsonContent-Length: 140{ "path": ["talks", 0, "comments"], "data": [{ "body": "Loved this!" }]}
query ConferenceQuery { talks { title time speaker { name } ...commentsFragment @defer }}
fragment commentsFragment on Talk { body}
GraphQL over HTTP
• GraphQL over HTTP is a proposed specification to define how GraphQL should be served over HTTP
• We have an RFC to add incremental delivery to this spec using chunked encoding and multipart responses
• https://github.com/graphql/graphql-over-http/pull/124
Server Codefunction sendPartialResponse( response: $Response, result: AsyncExecutionResult,): void { const json = JSON.stringify(result, null, 2); const chunk = Buffer.from(json, 'utf8'); const data = [ '', '---', 'Content-Type: application/json; charset=utf-8', 'Content-Length: ' + String(chunk.length), '', chunk, '', ].join('\r\n'); response.write(data);}// Close connectionresponse.end();
Client Code - Fetch
const response = await fetch(url, { method, headers, body })
// Don't call response.json()! // That waits for the whole response to finish loading.// const json = await response.json()
const reader = response.body.getReader();while (true) { const { value, done } = await reader.read(); if (done) return; handleChunk(value);}
Client Code - XMLHttpRequest
// This works in Internet Explorer 7!const xhr = new XMLHttpRequest();let index = 0;xhr.open(method, url);xhr.addEventListener("readystatechange", function onReadyStateChange() { const chunk = xhr.response.substr(index); handleChunk(chunk); index = xhr.responseText.length;});xhr.send(body);
Open Source Implementationfetch-multipart-graphql
• https://github.com/relay-tools/fetch-multipart-graphql
References
• Spec RFC: https://github.com/graphql/graphql-spec/blob/master/rfcs/DeferStream.md
• Spec Proposal: https://github.com/graphql/graphql-spec/pull/742
• GraphQL-JS: https://github.com/graphql/graphql-js/pull/2319
• express-graphql: https://github.com/graphql/express-graphql/pull/583
• fetch-multipart-graphql: https://github.com/relay-tools/fetch-multipart-graphql
• GraphQL over HTTP RFC: https://github.com/graphql/graphql-over-http/pull/124