-
HowtoReadaTechnicalPaperbyJasonEisner(2009)
Multipassreading
Skimthepaperfirst,skippingoveranythingthatwouldtakemuchmentaleffort.Justgetanideaofwherethepaperisgoing,whyitwaswritten,what'soldhatandwhat'snewtoyou.Toforceyourselftokeepmoving,giveyourselfalimitedtimebudgetperpageorusetheautoscrollfeatureofyourPDFreader.
Now,assumingthepaperstillseemsworthwhile,gobackandreadthewholethingmorecarefully.
Whynotpracticeonthiswebpage?Goahead,skimitfirst.
S.Keshavdescribesthreepassreadingindetail:Whatareyoutryingtodooneachpass?
WriteasyoureadWriteasyouread.Thiskeepsyourattentionfocusedandmakesyouengagewiththepaper.
Lowlevelnotes
Oftenitiseasiesttoscribblenotesontheprintedoutpaperitself,respondingincontexttotheformulas,figures,andtext.Inthatcase,fileorscanyourannotatedcopyforfuturereference.
(OrperhapsannotatethePDFfiledirectly,withoutprintingorscanning.AfreealternativetoAcrobatisPDFXChangeViewer,aWindowsprogramthatcanalsoberunonLinuxviawine.)
Youcanusenotesonthepaperto
restateunclearpointsinyourownwordsfillinmissingdetails(assumptions,algebraicsteps,proofs,pseudocode)annotatemathematicalobjectswiththeirtypescomeupwithexamplesthatillustratetheauthor'sideas,andexamplesthatwouldbeproblematicfortheauthordrawconnectionstoothermethodsandproblemsyouknowaboutaskquestionsaboutthingsthataren'tstatedorthatdon'tmakesensechallengethepaper'sclaimsormethodsdreamupfollowupworkthatyou(orsomeone)shoulddo
Highlevelnotes
Lowlevelnotesaren'tenough.Alsokeephighlevelnotesaboutpapers.It'squiteusefultodistillthepaperdown:summarizethethingsthatinterestedyou,contrastwithotherpapers,andrecordyourownquestionsandideasforfuturework.Writingthisdistillationgivesyouagoalwhilereadingthepaper,andthenoteswillbeusefultoyoulater.
MichaelMitzenmacherwrites:"Readcreatively.Readingapapercriticallyiseasy,inthatitisalwayseasiertotearsomethingdownthantobuilditup.Readingcreativelyinvolvesharder,morepositivethinking.Whatarethegoodideasinthispaper?Dotheseideashaveotherapplicationsorextensionsthat
-
theauthorsmightnothavethoughtof?Cantheybegeneralizedfurther?Aretherepossibleimprovementsthatmightmakeimportantpracticaldifferences?Ifyouweregoingtostartdoingresearchfromthispaper,whatwouldbethenextthingyouwoulddo?"
Isuggestsortingyourfileofnoteschronologically,bywhenyoureadthepaper,sincethatmayhelpyoufindvaguelyrememberedpapersorrememberwhatelseyouwerereadingatthetime.Sometimesyou'llwanttosearchbyauthor/title/etc.,sostartthenotesforeachpaperwitharoughcitation.(SeealsoHowtoOrganizeYourFiles.)
Ifyouhadtoputalotofeffortintoreallyunderstandingsomepoint,youcansharethateffortwithothers(andrecorditforyourownfuturereference)byimprovingthediscussionofthatpointontherelevantWikipediapage.
Whenandwheretoread
Startearly.Leaveenoughtimethatifyourattentionwanders,youcanputthepaperdownandpickitupagainwhenyou'reinabetterreadingmood.Thisisbetterthantryingtoforceyourselfthroughitonadeadline.
Somepeoplefinditeasiertoreadatparticulartimesofday,orwhileeatingorwalkingorridinganexercisebike.Doyouhabituallypickuptheclosestthingtoreadwhenyou'reatthebreakfasttableorinthebathroom?Thenleavepapersthereforyourself.
Tryreadingwithafriend!Sitnexttoeachother,lookingatthesamecopyofthepaper,andstaysynchronizedattheparagraphorsentencelevel.Readaloudattimes.You'llkeepeachothermovingandhelpeachotherthroughthehardparts.Discussasyougoalong.
Setasidetime
Whenyouarestartingoutinanewarea,itmaytakeyouhourstoreadaconferencepaperthoroughly.That'sokay.It'sworthspendingthatmuchtimetoreallyunderstandagoodorfoundationalpaper.Itwillpayoffinyourfuturereadingandresearch.
I'llneverfindthetime!Don'tworry.Notallpaperstakethatlong.Manyideasarereusedacrosspapers,soyouwillgetfasteratreading.Bynow,inanareaIknowwell,Icanoftenreadapaperin30minutesorless,becausethemotivationisfamiliarandIcanrecognizemuchofthesetupasstandardpractice.(Afterall,mostpapersfallintoanexistingtradition.Theyextendexistingworkwithoneortwogenuinenewideas,andsomesupportingdetailsthatmayormaynotbesignificant.)
ButI'malreadyathirdyearstudent.Whyisthispapertakingmesolong?Thereisnoshameinreadingslowly.ItstilltakesmeseveralhourstoabsorbapaperonsomethingthatIgenuinelydon'tknowwell.(Also,ittakesmehourstoreviewapapereveninmyownarea,becausetheburdenisonmetospotalltheproblemsoropportunitiesforimprovement.75%ofsubmittedconferencepapersarerejected,andmostoftheremaining25%alsoneedimprovementbeforepublication.)
WhichpartstofocusonSodoyoureallyhavetoreadthewholepapercarefullyonyoursecondpass?Sometimes,butnotalways.Itdependsonwhyyou'rereadingthepaper.
Idothinkthatwhenyouarelearninganewarea,youshouldreadatleastsomepapersextremelythoroughly.Thatmeansknowingwhateverysentenceandeverysuperscriptisdoing,sothatyoureallylearnallofthetechniquesusedinthepaper.Andunderstandingwhythingsweredoneasthey
-
were:askyourselfdumbquestionsandanswerthem.Practicetheabilitytodecodetheentirepaperasifyouwerereviewingitcriticallyandtryingtocatchanyerrors,sloppythinking,orincompleteness.Thiswillsharpenyourcriticalthinking.Youwillwanttoturnthispracticedcriticaleyeonyourselfasyouplan,execute,andwriteupyourownresearch.
However,therewillalsobeoccasionalpaperswhereitisnotworthreadingallthedetailsrightnow.Maybethedetailsareoflimitedinterest,oryousimplydon'tfeelequippedtounderstandthemyet.Considerthepartsofatypicalpaper:
Motivation.You'llwanttounderstandthisfairlywell,orthere'snopointinreadingthepaperatall.Butpartofthemotivationmaydependonthingsyoudon'tknow(mathematicalbackgroundorpastwork).Ifyoudon'twanttochasethosereferencesdownnow,youcouldjustraisetheirpriorityonyourreadinglist.
Mathematicsandalgorithms.Thesepartsarethetechnicalheartofthepaper.Sodon'tmakeahabitofskimmingthem.(Youcanlearnalotfromhowtheauthorssolvedtheirproblems.)Nonetheless,youmightskimatechnicalsectionif
Itseemslikeanexplanationofsomethingyoualreadyknow.Inthatcase,justcheckthatitreallysayswhatyouthink.
Whileyouprobablywouldbenefitrightawayfromknowingthemethodindetail,thispaperisjustnotagoodplacetolearnit,oritistooadvancedforyourightnow.Understandwhatyoureasonablycan,andthenputitonyourlistofthingstolearnforreal.Perhapsasksomeoneelsetoexplainittoyouortorecommendareading.
Itseemslikeanuglyadhocsolutionthatnoonewouldeverwanttouseanyway.Theonlyreasontounderstanditfullywouldbeifyouwantedtocriticizeitorimproveuponit.(Still,evenifyouskiptheuglydetails,understandwhattheauthors'intuitionswere.Thinkabouthowtocapturethoseintuitionsmoreelegantly.)
It'senoughtoknowfornowthatthemethodexists.Itseemsspecialized,soyoumightneverneedit.You'llcomebacktothepaperifyoudo.
Butyoushouldstillachieveclaritynowaboutwhatthemethodaccomplishes(itsinterface).Alsotrytogleanwhenitisapplicable,howharditwouldbetouse,andwhatdeterminesitsruntimeandaccuracy.Thenyou'llrememberthemethodwhenyouneedit.
Whatyoumightskipfornowarethehardparts:theinternalworkingsofthemethod(itsimplementation)andanyproofsofcorrectnessorefficiency.
Experiments.Manypaperstesttheirmethodsempirically.Whenyou'renewtoafield,youshouldexaminecarefullyhowthisistypicallydone(andwhetheryouapprove!).Itcanalsobehelpfultonoticewhatdatasetsandcodewereusedasyoumaywanttousethemyourselfinfuture.
Butonceyou'velearnedtheropes,youmaynotalwayscaresomuchaboutapaper'sexperiments.Afterall,sometimesyou'reonlyreadingthepapertostokeyourcreativitywithsomenewproblemsortechniques.IconfessthatIoftenpaylessattentiontotheexperimentaldetailsthoughexamplesorerroranalysisdocatchmyattentionbecausetheyoftenshedlight.
Ifyoudocareabouttheconclusionsofthepaper("didthemethodwork?""shouldIuseit?"),thenyoushouldgobackandcarefullyexaminetheexperimentaldesign,includingthechoiceofdata.Weretheexperimentsfair?Dotheysupporttheclaims?What'sreallygoingon?Aretheconclusionslikelytogeneralizebeyondthisexperimentalscenario?
-
Inshort,investyourtimewisely.Focusonwhatisvaluabletotakeaway.Ifyoucan'tfigureoutwhichpartsofthepaperaremost"interesting"or"important,"doasksomeonewhoshouldknow!Ifyoudon'tknowwhotoask,findotherpapersthatcitethisone(viaGoogleScholar)andseewhattheysayaboutthispaper.
DelipRaosuggests:"NeverreadtheoriginalpaperonXfirst.InsteadreadseverallaterpapersonwhattheysayaboutX,getanideaofXandthenreadtheoriginalpaper.Somehowtheresearchcommunityismuchbetterinexplainingideasclearlythantheoriginalauthorsthemselves."
Whattoread
docreativewebsearchexperimentwithseveralsearchesputyourselfinanauthor'sshoeswhatphrasesmighttheyhaveused?becomeapowersearcher!(readthehelppagesforyoursearchengine)specificallysearchattheACLAnthology,GoogleScholar,etc.
trackdownrelatedwork(onceyou'vegotarelevantpaper)backwardreferences:followthebibliographytoearlierpapersforwardreferences:seewhoelsehascitedthework(viaaninterfacesuchasGoogleScholar)
hassomeoneelsealreadylistedtherightpapersforyou?surveypapersinjournals(alsocalled"reviewarticles")coursesyllabireadinggroupwebpageschaptersintextbooksonlinetutorialsliteraturereviewchaptersfromdissertationsdirectrecommendationsfromfriendsorprofessors(perhapsatotherinstitutions)
breadthfirstexplorationreadalotofabstracts(andskimthepapersasneeded)beforedecidingwhichpapersarebesttoreadit'sokaytoreadmultiplerelatedpapersatonce,flippingbackandforthsothattheyclarifyoneanothertogetafeelfortheresearchlandscapeinanarea,flipthroughtheproceedingsofarelevantrecentworkshop,conference,orspecialthemejournalissue
whenthegoinggetstough,switchtobackgroundreadingtextbooksortutorialsreviewarticlesintroductionsandlitreviewchaptersfromdissertationsearlypapersthatareheavilycitedsometimesWikipedia
Thispageonline:http://cs.jhu.edu/~jason/advice/[email protected](suggestionswelcome)
LastMod$Date:2013/06/2416:54:40$
