HTTP::Parser::XSwriting a fast & secure XS module

Cybozu Labs, Inc.Kazuho Oku

Pros & Cons of XS

ProsFaster (well, not always, but…)

ConsDifficult and time-consumingHigher security risk

Mistakes in C programming leads to remote code injection

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 2

The Best Practice, in General

Don’t use XSUntil you hit performance bottleneck

Rewrite the bottleneck (only) using XS

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 3

HTTP::Parser::XS

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 4

Summary

An HTTP request parserDesigned and implemented to be

simple, fast & secureStatelessPSGI-compatible

Required or recommended by many Plack serversStarman, Starlet, Twiggy, etc.

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 5

Two-layered Approach

PicohttpparserHTTP request / response parser written in Chas its own test suitedoes not parse request / response content

request line (response line) and headers onlycopy-less

faster, lesser probability of security holes and memory leaks

HTTP::Parser::XSglue code to access picohttpparser from

perlMay 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 6

Speed and Complexity

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 7

reqs/sec. lines of code

HTTP::HeaderParser::XS

116,000 1,166

HTTP::Parser::XS 140,000 487

Plack::HTTPParser::PP 10,100 104

HTTP::Parser::XS is simple, and fastmost of the time is not spent in

picohttpparser, but in the glue code constructing hashrefpicohttpparser/trunk can handle >1Mreqs/sec.

Why is it a Stateless?

Faster and simpler than a stateful parserlower security risks

most HTTP requests / responses arrive in a single packet, anyway

if written optimally in C, the cost of re-parsing is smaller than storing headers into a perl hashref

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 8

Why is it Stateless? (cont’d)

Easy to determine the end of an multi-packet HTTP requestby looking for “\r\n\r\n” within the last

packet (and preceding three bytes)mainly as a countermeasure for Slowloris

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 9

Consistent Design

Functions take same arguments buf – points to current char buf_end – points to end of buffer *ret – error value returns pointer to the next char

or null on error (the reason will be stored in *ret)

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 10

const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret){ EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P'); EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret);}

Macros

Consistent design is essential to heavy use of macrosGood abstraction ⇒

safe code

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 11

#define CHECK_EOF() \ if (buf == buf_end) { \ *ret = -2; \ return NULL; \ }#define EXPECT_CHAR(ch) \ CHECK_EOF(); \ if (*buf++ != ch) { \ *ret = -1; \ return NULL; \ }

const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret){ EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P'); EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret);}

Micro-optimization (only in picohttpparser)

Reduce # of conditional branches, optimize for pipeline (27% faster)

Unroll loops (36% faster)

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 12

#define unlikely(x) __builtin_expect(!!(x), 0)

for (; ; ++buf) { CHECK_EOF(); if (unlikely((unsigned char)*buf <= '\r') && (*buf == '\r' || *buf == '\n')) goto EOL_FOUND;}

Release History of HTTP::Parser::XS

Current version: 0.07No security hole found since initial

releaseTwo memory leaks were found and

fixed

Please let me know if you find any security holes (especially the ones that lead to arbitrary code execution)

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 13

Conclusion – KISS

Keep it simple, stupidfor fast developmentsimple design leads to more secure codeuse perl whenever possible

simple operations (like tokenization) is worth converting to XS

complex operations (from handling of strings to database queries) are not so slow in perl

May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 14