JeffKnupp@[email protected]
Authorof“WritingIdiomaticPython”Full-timePythondeveloper@AppNexusBloggeratjeffknupp.comCreatorofthe“sandman”Pythonlibrary
We'regoingtousePythonto generateaRESTAPI.
Andwe'regoingtodoitwithoutwritingasinglelineofcode.
We'llgooverwhataRESTAPIis,howitworks,andwhyit'susefulWe'llreviewtheHTTPprotocolandhowthewebworksWe'llseealotofPythoncode
Sevenletters.Twoacronyms.Buywhatdoesitmean?
Programmaticwayofinteractingwithathird-partysystem.
WaytointeractwithAPIsoverHTTP(thecommunicationprotocoltheInternetisbuilton).
"REST"wascoinedbyRoyFieldinginhis2000doctoraldissertation.Includessetofdesignprinciplesandbestpracticesfordesigningsystemsmeanttobe"RESTful".
InRESTfulsystems,applicationstateismanipulatedbytheclientinteractingwithhyperlinks.Arootlink(e.g.
)describeswhatactionscanbetakenbylistingresourcesandstateashyperlinks.
http://example.com/api/
HTTPisjustamessagingprotocol.HappenstobetheonetheInternetisbasedon.
RESTfulsystemsusethisprotocoltotheiradvantagee.g.cachingresources
GETPOSTPUTPATCHDELETE
TounderstandhowRESTAPIswork,wehavetounderstandhowthewebworks.
EverythingyouseeonthewebistransferredtoyourcomputerusingHTTP.
Whathappenswhenwetypehttp://www.jeffknupp.comintoourbrowser?
Let'stracethelifecycleofabrowser'srequest.
AprotocolcalledtheDomainNameService(DNS)isusedtofindthe"real"(IP)addressofjeffknupp.com.
GET
ThebrowsersendsaGETrequestto192.168.1.1forthepageataddress/(thehomeor"root"page).
The (aprogramusedtoserviceHTTPrequeststoawebsite)receivestherequest,findstheassociatedHTMLfile,andsendsitasanHTTPResponse.
Ifthereareanyimages,videos,orscriptsthattheHTMLmakesreferenceto,separateHTTPGETrequestsaremade
forthoseaswell.
Programs,likecurl,canalsoissueHTTPrequests
CURL
curltalkstothewebserver,usingapublicAPI(viaHTTP)
ARESTAPIexposesyourinternalsystemtotheoutsideworld
It'salsoafantasticwaytomakeasystemavailabletoother,internalsystemswithinanorganization.
ExamplesofpopularRESTAPIs:TwitterGitHubGoogle(foralmostallservices)
Ifyou'reaSaaSprovider,youareexpectedtohaveaRESTAPIforpeopletowriteprogramstointeractwithyour
service.
FourcoreconceptsarefundamentaltoallRESTservices(courtesyWikipedia)
WhenusingHTTP,thisisdoneusingaURI.Importantly,aresourceand arecompletelyorthogonal.Theserverdoesn'treturndatabaseresultsbutratherthe
JSONorXMLorHTMLrepresentationoftheresource.
Whentheservertransmitstherepresentationoftheresourcetotheclient,itincludesenoughinformationforthe
clienttoknowhowtomodifyordeletetheresource.
Eachrepresentationreturnedbytheserverincludesinformationonhowtoprocessthemessage(e.g.usingMIME
types
Clientsare .Theyknownothingabouthowtheserviceislaidouttobeginwith.Theydiscoverwhatactionstheycan
takefromtherootlink.Followingalinkgivesfurtherlinks,definingexactlywhatmaybedonefromthatresource.
Clientsaren'tassumedtoknow exceptwhatthemessagecontainsandwhattheserveralreadytoldthem.
ARESTAPIallows tosend tomanipulate .
...SoweneedtowriteaservercapableofacceptingHTTPrequests,actingonthem,andreturningHTTPresponses.
Yep.ARESTfulAPIServiceisjustawebapplicationand,assuch,isbuiltusingthesamesetoftools.We'llbuildours
usingPython,Flask,andSQLAlchemy
EarlierwesaidaRESTAPIallowsclientstomanipulateviaHTTP.
Prettymuch.Ifyou'resystemisbuiltusingORMmodels,yourresourcesarealmostcertainlygoingtobeyourmodels.
Webframeworksreducetheboilerplaterequiredtocreateawebapplicationbyproviding:
ofHTTPrequeststohandlerfunctionsorclassesExample:/foo=>defprocess_foo()
ofHTTPresponsestoinjectdynamicdatainpre-definedstructure
Example:<h1>Hello{{user_name}}</h1>
ThemoretimeyouspendbuildingRESTAPIswithwebframeworks,themoreyou'llnoticethesubtle(andattimes,
glaring)impedancemismatch.
URLsas toprocessingfunctions;RESTAPIstreatURLsastheaddressofaresourceorcollectionHTMLtemplating,whileRESTAPIsrarely.JSON-relatedfunctionalityfeelsbolted-on.
Imaginewe'reTwitterweeksafterlaunch.AshtonKutcherseemstobeabletouseourservice,butwhatabout
?
That'sright,we'llneedtocreateanAPI.Beinganinternetcompany,we'llbuildaRESTAPIservice.Fornow,we'llfocus
ontworesources:usertweet
Allresourcesmustbeidentifiedbyauniqueaddressatwhichtheycanbereached,theirURI.Thisrequireseachresource
containauniqueID,usuallyamonotonicallyincreasingintegerorUUID(likeaprimarykeyinadatabasetable).
OurpatternforbuildingURLswillbe/resource_name[/resource_id[/resource_attribute]]
Herewedefineourresourcesisafilecalledmodels.py:
classUser(db.Model,SerializableModel):__tablename__='user'
id=db.Column(db.Integer,primary_key=True)username=db.Column(db.String)
classTweet(db.Model,SerializableModel):__tablename__='tweet'
id=db.Column(db.Integer,primary_key=True)content=db.Column(db.String)posted_at=db.Column(db.DateTime)user_id=db.Column(db.Integer,db.ForeignKey('user.id'))user=db.relationship(User)
classSerializableModel(object):"""ASQLAlchemymodelmixinclassthatcanserializeitselfasJSON."""
defto_dict(self):"""Returndictrepresentationofclassbyiteratingoverdatabasecolumns."""value={}forcolumninself.__table__.columns:attribute=getattr(self,column.name)ifisinstance(attribute,datetime.datetime):attribute=str(attribute)value[column.name]=attributereturnvalue
Here'sthecodethathandlesretrievingasingletweetandreturningitasJSON:
frommodelsimportTweet,User
@app.route('/tweets/<int:tweet_id>',methods=['GET'])defget_tweet(tweet_id):tweet=Tweet.query.get(tweet_id)iftweetisNone:response=jsonify({'result':'error'})response.status_code=404returnresponseelse:returnjsonify({'tweet':tweet.to_dict()})
Let'scurlournewAPI(preloadedwithasingletweetanduser):
$curllocalhost:5000/tweets/1{"tweet":{"content":"Thisisawesome","id":1,"posted_at":"2014-07-0512:00:00","user_id":1}}
@app.route('/tweets/',methods=['POST'])defcreate_tweet():"""CreateanewtweetobjectbasedontheJSONdatasentintherequest."""ifnotall(('content','posted_at','user_id'inrequest.json)):response=jsonify({'result':'ERROR'})response.status_code=400#HTTP400:BADREQUESTreturnresponseelse:tweet=Tweet(content=request.json['content'],posted_at=datetime.datetime.strptime(request.json['posted_at'],'%Y-%m-%d%H:%M:%S'),user_id=request.json['user_id'])db.session.add(tweet)db.session.commit()returnjsonify(tweet.to_dict())
InRESTAPIs,agroupofresourcesiscalleda .RESTAPIsareheavilybuiltonthenotionofresourcesand
collections.Inourcase,the oftweetsisalistofalltweetsinthesystem.
ThetweetcollectionisaccessedbythefollowingURL(accordingtoourrules,describedearlier):/tweets.
@app.route('/tweets',methods=['GET'])defget_tweet_collection():"""ReturnalltweetsasJSON."""all_tweets=[]fortweetinTweet.query.all():all_tweets.append({'content':tweet.content,'posted_at':tweet.posted_at,'posted_by':tweet.user.username})
Allthecodethusfarhasbeenprettymuchboilerplate.EveryRESTAPIyouwriteinFlask(modulobusinesslogic)willlook
identical.Howcanweusethattoouradvantage?
Wehaveself-drivingcarsanddeliverydrones,whycan'twebuildRESTAPIsautomatically?
Thisallowsonetoworkatahigherlevelofabstraction.Solvetheproblemonceinageneralwayandletcodegeneration
solveeachindividualinstanceoftheproblem.
Partof
SANDBOY
ThirdpartyFlaskextensionwrittenbythedashingJeffKnupp.Defineyourmodels.Hitabutton.BAM!RESTfulAPI
servicethat .
(Thenamewillmakemoresenseinafewminutes)
GeneralizesRESTresourcehandlingintonotionofa(e.g.the"TweetService"handlesalltweet-relatedactions).classService(MethodView):"""Baseclassforallresources."""
__model__=None__db__=None
defget(self,resource_id=None):"""ReturnresponsetoHTTPGETrequest."""ifresource_idisNone:returnself._all_resources()else:resource=self._resource(resource_id)ifnotresource:raiseNotFoundExceptionreturnjsonify(resource.to_dict())
def_all_resources(self):"""ReturnallresourcesofthistypeasaJSONlist."""ifnot'page'inrequest.args:resources=self.__db__.session.query(self.__model__).all()else:resources=self.__model__.query.paginate(int(request.args['page'])).itemsreturnjsonify({'resources':[resource.to_dict()forresourceinresources]})
Here'showPOSTworks.Noticetheverify_fieldsdecoratoranduseof**request.jsonmagic...
@verify_fieldsdefpost(self):"""ReturnresponsetoHTTPPOSTrequest."""resource=self.__model__.query.filter_by(
**request.json).first()ifresource:returnself._no_content_response()instance=self.__model__(**request.json)self.__db__.session.add(instance)self.__db__.session.commit()returnself._created_response(instance.to_dict())
Wehaveourmodelsdefined.HowdowetakeadvantageofthegenericServiceclassandcreateservicesfromour
models?defregister(self,cls_list):"""RegisteraclasstobegivenaRESTAPI."""forclsincls_list:serializable_model=type(cls.__name__+'Serializable',(cls,SerializableModel),{})new_endpoint=type(cls.__name__+'Endpoint',(Service,),{'__model__':serializable_model,'__db__':self.db})view_func=new_endpoint.as_view(new_endpoint.__model__.__tablename__)self.blueprint.add_url_rule('/'+new_endpoint.__model__.__tablename__,view_func=view_func)self.blueprint.add_url_rule('/{resource}/<resource_id>'.format(resource=new_endpoint.__model__.__tablename__),view_func=view_func,methods=[
'GET','PUT','DELETE','PATCH','OPTIONS'])
TYPE
InPython,typewithoneargumentreturnsavariable'stype.Withthreearguments,
.
TYPE
serializable_model=type(cls.__name__+'Serializable',(cls,SerializableModel),{})
new_endpoint=type(cls.__name__+'Endpoint',(Service,),{'__model__':serializable_model,'__db__':self.db})
Let'splaypretendagain.Nowwe'reaIaaScompanythatletsusersbuildprivateclouds.We'llfocusontworesources:
cloudandmachine
classCloud(db.Model):__tablename__='cloud'
id=db.Column(db.Integer,primary_key=True)name=db.Column(db.String,nullable=False)description=db.Column(db.String,nullable=False)
classMachine(db.Model):__tablename__='machine'
id=db.Column(db.Integer,primary_key=True)hostname=db.Column(db.String)operating_system=db.Column(db.String)description=db.Column(db.String)cloud_id=db.Column(db.Integer,db.ForeignKey('cloud.id'))cloud=db.relationship('Cloud')is_running=db.Column(db.Boolean,default=False)
fromflaskimportFlaskfromflask.ext.sandboyimportSandboyfrommodelsimportMachine,Cloud,db
app=Flask(__name__)app.config['SQLALCHEMY_DATABASE_URI']='sqlite:///db.sqlite3'db.init_app(app)withapp.app_context():db.create_all()sandboy=Sandboy(app,db,[Machine,Cloud])app.run(debug=True)
Incaseswherewe'rebuildingaRESTAPIfromscratch,thisisprettyeasy.Butwhatif:
WehaveanexistingdatabaseWewanttocreateaRESTfulAPIforitIthas200tables
OnlydownsideofFlask-Sandboyisyouhavetodefineyourmodelclassesexplicitly.Ifyouhavealotofmodels,this
wouldbetedious.
...Idon'tdotedious
Wehaveprivatecompaniesbuildingrocketshipsandelectriccars.Whycan'twehaveatoolthatyoupointatanexistingdatabaseandhitabutton,then,BLAM!RESTfulAPIservice.
SANDMAN
,alibrarybyteenheartthrobJeffKnupp,createsaRESTfulAPIservicefor with
.
Here'showyourunsandmanagainstamysqldatabase:
$sandmanctlmysql+mysqlconnector://localhost/Chinook*Runningonhttp://0.0.0.0:8080/*Restartingwithreloader
$curl-vlocalhost:8080/artists?Name=AC/DCHTTP/1.0200OKContent-Type:application/jsonDate:Sun,06Jul201415:55:21GMTETag:"cea5dfbb05362bd56c14d0701cedb5a7"Link:</artists/1>;rel="self"
{"ArtistId":1,"Name":"AC/DC","links":[{"rel":"self","uri":"/artists/1"}],"self":"/artists/1"}
ETagsetcorrectly,allowingforcachingresponsesLinkHeadersettoletclientsdiscoverlinkstootherresourcesSearchenabledbysendinginanattributenameandvalue
Wildcardsearchingsupported
Wecancurl/andgetalistofallavailableservicesandtheirURLs.Wecanhit/<resource>/metatogetmeta-infoaboutthe
service.Example(the"artist"service):
$curl-vlocalhost:8080/artists/metaHTTP/1.0200OKContent-Length:80Content-Type:application/jsonDate:Sun,06Jul201416:04:25GMTETag:"872ea9f2c6635aa3775dc45aa6bc4975"Server:Werkzeug/0.9.6Python/2.7.6
{"Artist":{"ArtistId":"integer(11)","Name":"varchar(120)"}}
Andnowfora(probablybroken)live-demo!
"Real"RESTAPIsenableclientstousetheAPIusingonlytheinformationreturnedfromHTTPrequests.sandmantriestobeas"RESTful"aspossiblewithoutrequiringanycodefrom
theuser.
WouldbenicetobeabletovisualizeyourdatainadditiontointeractingwithitviaRESTAPI.
1. Codegeneration2. Databaseintrospection3. Lotsofmagic
sandmancamefirst.HasbeennumberonePythonprojectonGitHubmultipletimesandisdownloaded25,000timesa
month.Flask-Sandboyissandman'slittlebrother...
ThefactthattheendresultisaRESTAPIisnotespeciallyinterestingMoreimportantaretheconceptsunderpinningsandmanandFlask-Sandboy
WorkathigherlevelofabstractionSolveaproblemonceinagenericmannerReduceserrors,improvesperformance
Ingeneral:
Speakingofautomation,here'showmybookis"built"...
sandman=Flask+SQLAlchemy+LotsofGlueRequiresyouknowthecapabilitiesofyourtoolsPartoftheUNIXPhilosophy
ThebestprogrammingadviceIevergotwasto"belazy"
SandmanexistsbecauseIwastoolazytowriteboilerplateORMcodeforanexistingdatabaseFlask-SandboyexistsbecauseIwastoolazytowritethesameAPIservicesoverandoverBeinglazyforcesyoutolearnyourtoolsandmakeheavyuseofthem
Recommended
