1

AdventuresinScriptingLandSvenErikKnop,PerforceSoftwareLtd

PresentedatthePerforceuserconference2009attheFlamingoHotel,LasVegas

2

TableofContents1 Overview........................................................................................................................................ 4

1.1 Introduction ........................................................................................................................... 4

1.2 Whatarescriptinglanguages?............................................................................................... 4

1.3 WhatareP4Python,P4PerlandP4Ruby?.............................................................................. 5

2 P4Object ....................................................................................................................................... 6

2.1 Overview................................................................................................................................ 6

2.2 Definingtheenvironment...................................................................................................... 6

2.3 Attributes............................................................................................................................... 7

2.4 FirstExamples ........................................................................................................................ 8

2.5 Runmethod ........................................................................................................................... 9

2.5.1 Argumentsandreturnvalues ........................................................................................ 9

2.5.2 Errors,WarningsandExceptions ................................................................................. 10

2.5.3 Generatedandoverloadedrunmethods .................................................................... 11

2.5.4 Formhandling.............................................................................................................. 12

3 P4::Mapclass ............................................................................................................................... 14

4 Examplesfromthewild ............................................................................................................... 16

4.1 Overview.............................................................................................................................. 16

4.2 Scriptexample ..................................................................................................................... 16

4.3 Triggerexamples.................................................................................................................. 16

4.4 Applicationexample ............................................................................................................ 18

3

4

1 Overview

1.1 IntroductionScriptshavelongbeenusedaroundaPerforceservertocreatesmalltools,automatedprocessessuchasbuilds,triggersandfull‐blownapplications.

Perforcenowfullysupportsintegrationsintothreescriptinglanguages:Perl,PythonorRuby.Theseintegrations,dubbedP4Perl,P4PythonandP4Ruby,extentthescriptinglanguagebyaddingaP4

class.Withthisclass,communicationwiththeserverandprocessingoftheoutputbecomesmuchsimplerandallowsascriptwritertoconcentrateontheessentialfunctionsofthescriptinsteadofworryingaboutregularexpressionsandparsingofoutputs.

Thiswhitepaperisnotintendedasareplacementforthedocumentation,P4Script.pdf,availableon

thePerforcewebsite.Insteadthepapershouldbeseenasanintroductionintotheconceptsofthescriptingintegrationsandtoshowsomeworkingexamples.

1.2 Whatarescriptinglanguages?Manyscriptinglanguageshavecommoncharacteristics:

Scriptinglanguagesareinterpretedorcompiledforavirtualmachine.

Thismeansscriptinglanguagescanbeusedinteractivelyinashellforquicktestsandtry‐outs.Theyarealsoverysuitableforrapidprototyping.

Scriptinglanguagesusedynamictyping.

“Ifitwalks,looksandquackslikeaduck,itisprobablyaduck”.

Insteadofstaticallydeterminingthetypeofavariable,scriptinglanguagesvariablesaredynamicallytyped,thatis,avariablecanbeofanytypeandcanacceptanymethodcall,eitherrespondingorrejectingthemethod.Thismakesscriptinglanguagesusefulforrapidprototyping,but,astheexampleofSmalltalkshows,doesnotpreventthemfrombeingusedinlargerapplications.Scriptinglanguagesareoftenobject‐oriented.Theobject‐orientedprogrammingparadigmmakescreatinglarger,reusablecomponentseasierandencouragescleaninterfacesthroughencapsulation.Theyuseautomaticmemorymanagement,oftenviaagarbagecollector.Insteadofworryingaboutmemorymanagement,usersofscriptinglanguagecanfocusonthetaskathand.Theyhavealargecollectionofusefullibrary.Scriptinglanguagesareoftenthegluethatbindsapplicationstogether.Forthatpurposetheycomewithmanybuilt‐inlibrariesthatcandealwithaplethoraoftasksfromfilemanagement,networkprogramming,anddatabaseaccesstofull‐blownGUIs.

5

ScriptinglanguagescannormallybeextendedviaC/C++orsometimesJava(JPython,JRubyetc).Thisencouragesprogrammersandenthusiaststocreateadditionallibrariesforparticulartasks.P4Python,P4PerlandP4Rubyaresuchlibraries.

1.3 WhatareP4Python,P4PerlandP4Ruby?P4Python,P4PerlandP4Ruby(collectivelyalsoknownasP4Script)arescriptinglanguageextensions

writteninC++,whichwrapthestandardPerforceC++API(P4API).TheP4APIisverypowerfulbutnoteasytouseforrapidprototypingandrequiresaccesstoaC++compiler.

P4Scriptallowuserstocreatesimplescripts,butalsolargerapplicationsthatcanaccessthePerforceserverinasimpleandstandardizedway.

EachinterfacedefinesaP4classwith,languageandnamingconventionspermitting,identical

interface.TheonlymajordifferencebetweenP4PerlandtheotherinterfacesisthatPerlhasnorealconceptofexceptions.ThereforeerrorhandlinginP4Perlscriptsisusuallydifferentthanin

P4PythonandP4Ruby.

TheP4Scriptinterfacesareprovidedinsourcecode.Consultthereleasenotesonhowtobuildandinstalleachinterface,whichrequiresaccesstoaC++compiler,normallyprovidedonanyUNIXandUNIX‐likeplatform.Thereisready‐madecompiledversionavailableforWindows.Sourcecodeand

installerscanbedownloadedfromthePerforceftpserver.

ThecurrentversioninstalledofparticularP4Scriptcanbeidentifiedbyinvokingtheclassmethod“identify()”:

P4Script MethodP4Perl P4.Identify()P4Ruby P4.identifyP4Python P4.identify()

Theoutputofthemethodisastringin3linesandlookssimilartothis:

Perforce - The Fast Software Configuration Management System. Copyright 1995-2009 Perforce Software. All rights reserved. Rev. P4Ruby/NTX86/2008.2/184124 (2008.2 API) (2009/01/08).

6

2 P4Object

2.1 OverviewAP4instancerepresentsaconnectiontoaPerforceserverandcanbeusedtosendoneorseveralcommandstotheserver.

Connectionmustbeexplicitlyestablishedwiththe“connect()”command.

Comparethistocallingthecommandlineclientdirectlyorviaanexecutemethodfromthescript.

There,theconnectionisimmediatelyavailablebutmustbere‐establishedforeverysinglecommand,whichimpliescreatinganewsocketpairandspawninganewthreadorforkinganewprocessontheserverside.

FortheP4Scriptintegrations,theconnectionstaysopenuntilterminatedwiththe“disconnect()”

commandorwhenthescriptfinishes,inwhichcasetheconnectionisnormallycleanedupbythegarbagecollector.Nevertheless,itisconsideredgoodformtodisconnectfromPerforcebeforeexitingthescript.

Command P4Python P4Perl P4Ruby UsageConnect P4.connect() P4.Connect() P4#connect() Establishconnection

withtheserverDisconnect P4.disconnect() P4.Disconnect() P4#disconnect() Disconnectsfromthe

server.Connectioncanbeestablishedagainlater.

Connected P4.connected() P4.Connected() P4#connected?() Booleanmethodindicatingcurrentconnectionstate.

Thecentralmethodusedisthe“run()”method.Withthis,thescriptsendscommandstothePerforceserver.CommandsavailableandsyntaxareidenticaltothecommandlineclientP4.

2.2 DefiningtheenvironmentBeforeestablishingtheconnection,and,exceptforthePort,foreverycommand,theenvironmentcanbesetthroughattributesoftheP4object.

DefiningtheenvironmentthoughthesettingsofattributestakespriorityoverallotherwaystheP4

objectsetsupitsenvironment,whichare,inorderofprecedence:

‐ P4CONFIGfile‐ Shellenvironmentvariables

‐ Registry(onWindows)‐ Defaults

AusercancheckwhichP4CONFIGwasusedbyinterrogatingtheread‐onlystringattribute

“p4config_file”.

7

2.3 AttributesThefollowingattributesareavailable:

Name Type Descriptionport String P4PORTuser String P4USERclient String P4CLIENTcharset String P4CHARSEThost String P4HOSTcwd String Currentworkingdirectorypassword String P4PASSWDticket_file String P4TICKETSprog String Thenameoftheapplication(asshowninp4monitorandthelog)version String Theversionoftheapplication(asshowninp4monitorandthelog)api_level Integer Lockserveroutputformattospecificclientleveltagged Integer Setto1orTrue,providesoutput intaggedform,otherwise instring

formmaxresults Integer Overridesmaxresultsfromthegroupspecmaxscanrows Integer Overridesmaxscanrowsfromthegroupspecmaxlocktime Integer Overridesmaxlocktimefromthegroupspecexception_level Integer Defines under which circumstances exceptions are thrown (see

below)server_level Integer Serverlevel(readonly)debug Integer Setthedebugvalue,whichprovidesvariesadditionaloutput

Allwriteableparameterscanbechangedafteraconnectionhasbeenmade,withtheexceptionof

“port”.Changingtheporthasnoeffectafterconnectingandsettingthisattributemightraiseanexception.

Theattributes“prog”and“version”canbeusedtoidentifytherunningscriptinthemonitorandthelog.Withoutsettingtheseattributes,thenameshowninthemonitorwillbe“unnamedp4‐XXX

script”(withXXXreplacedbyperl,pythonorruby).

Thedefault“api_level”isdefinedbytheP4APIcompiledintotheintegration.Herearethecurrentclientlevels:

• client protocol 1: 97.1 • client protocol 2: 97.2 • client protocol 3: 97.3 • client protocol 4: 98.1 • client protocol 5: 98.2 • client protocol 6: 99.1 • client protocol 7: 99.1 • client protocol 8: 99.2 • client protocol 51: 2001.1 • client protocol 52: 2001.2 • client protocol 54: 2002.1 • client protocol 55: 2002.2

8

• client protocol 56: 2003.2 • client protocol 57: 2004.1 • client protocol 58: 2005.2 • client protocol 59: 2006.1 • client protocol 60: 2006.2 • client protocol 61: 2007.2 • client protocol 62: 2007.3 • client protocol 63: 2008.1 • client protocol 64: 2008.2

(Thesevaluesaretakenfromtheknowledgebasearticlehttp://kb.perforce.com/P4dServerReference/ProtocolLevels/PerforceClientLevels).

Changingtheclientlevelcausestheservertorespondtorequestintheformatthatexistedatthatlevel.Thisattributecanbeusedtoensurethatascriptreceivestheexpectedresponsefromaserver

eveniftheclientAPIisupgraded.Forexample,bysetting“P4.api_level=64”fora2008.2P4Scriptprogram,aprogrammercanensurethatwhenthescriptisrunwithalaterversionofP4Script,the

responseoftheserverisstillidenticaltothe2008.2level.

Taggedmodeandexceptionlevelwillbeexplainedbelow.

2.4 FirstExamplesItistimetoshowsomeexamplesonhowtouseP4Perl,P4PythonandP4Ruby.Hereisthesamescriptinthe3differentscriptinglanguagescurrentlysupported:

P4Perl:

P4Ruby:

P4Python:

use P4; my $p4 = new P4; $p4->SetPort( "1666" ); $p4->Connect() or die ("connect"); for my $user ($p4->Run("users")) { print "Hello $user->{ 'User' }\n"; } $p4->Disconnect();

require “P4” p4 = P4.new P4.port = “1666” p4.connect p4.run(“users”).each { |user| puts “Hello #{user[“User”]}” } p4.disconnect

9

Allthreeprogramsproduceexactlythesameoutputiftheconnectionsucceeds.Incaseofafailure,

thePythonandRubyscriptswillthrowexceptions.

Line#1importstheP4moduleintothescript.Line#2createsaP4object.ThereisnoconnectiontothePerforceserveryet.Line#3setsanattributeasanexampleofhowtheenvironmentcouldbedefined.

Line#4establishestheconnectiontothePerforceserver.Line#5runsthecommand“p4users”anditeratesthroughtheoutput,usingthevariable“user”.Line#6printsout“Hello<username>”foreachuser,accessingthefield“User”intheresultset.

Line#7closestheblockoftheiteration(notnecessaryinPython)Line#8(#7inthePythonscript)disconnectsfromtheserveragain.

2.5 Runmethod

2.5.1 Argumentsandreturnvalues

2.5.1.1 ArgumentsThestandardformofthe“run()”methodis

P4.run(command, args)

Here,“command”isastandardPerforcecommandlike“sync”or“edit”,and“args”arethecommandarguments.

Argumentshavetobeprovidedasalistorasseparatearguments,butnotasinglestring.Forexample,toruntheequivalenttothefollowingP4commandfromthecommandline,

p4 changes –m 1 –c bruno_ws

useoneofthefollowingsyntaxes

p4.run(“changes”, “-m”, “1”, “-c”, “bruno_ws”) p4.run(“changes”, [“-m”, “1”, “-c”, “bruno_ws”])

Donotuseasinglestringforallargumentsbecausetheservercannotparsetheseandwilldeliverthewrongresult.

2.5.1.2 ReturnvaluesandtaggedmodeThe“run”commandalwaysreturnsanarray,evenifthereisonlyasingleresult.Thecontentofthearraydependsonthesettingofthe“tagged”attribute.

Thestandardistouseataggedmodeof1(True).Thismeanstheserverreturnvaluesinkey‐valuepairs,whichwillbestoredinahashdictionary.ThisisequivalenttorunningtheP4commandline

clientwiththeoption“‐ztag”.

import P4 p4 = P4.P4() p4.port = “1666” p4.connect() for user in p4.run(“users”): print “Hello %s” % user[“User”] p4.disconnect()

10

Whensettingthetaggedmodeto0(False),theserverwillreturnstringsinstead.

Observethedifferencebetweentaggedanduntaggedmode(hereinPython):

Inmostcases,thetaggedmodeiseasiertouse,becauseretrievingresultsisasimplelookup.

Untaggedmodenormallyrequiresaparsingoftheoutput,whichcanbeerrorprone.

Theservernowadays(2008.2)providesthesameinformationandoftenevenmoreinformationintaggedformthanthestandardoutput.Therearefewexceptions,oneofwhichtodateis“p4diff2‐ds”,thesummaryformatforthediff2command,whichhasnotaggedmodeyetandrequires

untaggedoutput.

Anotherexampleis“p4.run_counter()”,whichintaggedmodereturnsanarraywithtwokey‐valuepairs,oneforthenameandtheotherforthevalue.Theuntaggedoutputsimplyreturnsanarraycontainingthecountervalueasastring.

2.5.2 Errors,WarningsandExceptionsIftheserverreturnsanerrororawarning,P4PythonandP4RubywillthrowanexceptionoftypeP4.P4ExceptionandP4::P4Exception,respectively.

Perldoesnothaveanystandardizedexceptionhandling,sousersofP4Perlwillneedtocheckthe

resultofthemethodsP4.ErrorCount()andP4.WarningCount()toseeifanyerrorsoccurred.ErrorsandwarningsarethenretrievedasanarraywiththecommandP4.Errors()andP4.Warnings().

Bydefault,P4PythonandP4Rubywillthrowanexceptionforeveryerrorandeverywarning.Awarningarises,forexample,if“p4sync”returnswiththemessage“file(s)up‐to‐date”.

YoucancontrolwhenP4PythonandP4Rubythrowexceptionsthroughthe“exception_level”

attribute.

Level Name Description0 RAISE_NONE Noexceptionsarethrown;needtocheckerrorsandwarnings

p4.tagged = 1 p4.run(“changes”, “-m1”) [{'status': 'submitted', 'client': 'bruno_ws',

'user': 'bruno', 'time': '1237901268', 'change': '824', 'desc': 'changed filetype\n'}]

p4.tagged = 0 p4.run(“changes”, “-m1”) ["Change 824 on 2009/03/24 by bruno@bruno_ws 'changed

filetype '"]

p4.tagged = 1 p4.run(“counter”, “change”) [{'counter': 'change', 'value': '1033'}] p4.tagged = 0 p4.run(“changes”, “-m1”) ['1033']

11

1 RAISE_ERRORS Onlyerrorsarethrown,warningsareignored2 RAISE_ALL Errorsandwarningsarethrown

• Thedefaultvaluefortheexception_levelis2(RAISE_ALL).

• ThesymbolicnamesaredefinedasclassconstantsintheP4classforP4PythonandP4Ruby.

P4Rubyalsohasamethodthatcanchangetheexception_levelforablockonlyandthenresetsitagaintoitsoriginalvalue:

Inthiscodesnippet,thecommand“sync”isrunwiththeexception_levelloweredtoavoidthrowinganexceptioniftheserversendsawarningthatallfilesarealreadyup‐to‐date.

FutureversionsofP4Pythonwillsupportthesamemethod,tobeusedwiththePython“with”

keyword.

IfyoudonotcatchtheP4Exception,yourscriptwillterminatewithanuncaughtexceptionerror,printingoutthemessagefromtheserveraswellasthestacktraceofwhereinthescripttheexceptionhappened.

2.5.3 GeneratedandoverloadedrunmethodsPython,PerlandRubyallallowuserstocallarbitrarymethodsonanobject.ThisenablesP4Python,

P4PerlandP4Rubytogenerateadditionalmethods.Thistechniqueisusedforthe“run”method.

Theideaistoreplacethesequence

p4.run(“command”)

withthegeneratedmethod

p4.run_command() # Python, Ruby $p4->RunCommand() # Perl

Thismakesthesyntaxalittlebiteasiertoread,butitalsopermitsustooverridesomeofthesenewmethodstoprovideadditionalfunctionality.

Theform“run_command()”and“run(‘command’)”arenormallyequivalent(i.e.whennotoverridden);theformerissimplytranslatedintothelatter.

ThefollowingmethodsareoverloadedinP4Python,P4PerlandP4Ruby

Method Additionalfunctionality

require “P4” p4 = P4.new p4.api_level = 64 p4.connect p4.at_exception_level(P4::RAISE_ERRORS) do p4.run(“sync”, “...”) end p4.disconnect

12

run_filelog() ReturnsDepotFilearrayrun_login() Takesp4.passwordastheinputrun_password(old,new) Setsthepasswordwithoutpromptingrun_resolve() Canusearesolverobject(Perl,Python)orablock(Ruby)run_submit() Canhaveachangeformargument

Youcangettheoriginalfunctionalityofeachcommandbackbyusingtheform“run(‘command’)”,forexample“run(‘filelog’)”insteadof“run_filelog()”.

EveryP4.DepotFileobjectcontainsalistofrevisionsoftypeP4.Revision.RevisionscanalsocontainalistofintegrationsoftypeP4.Integration.Detailsaredescribedinthedocumentation.Themethod

run_filelog()isoverwrittentoprovideamoreuser‐friendlyoutput.

Tofindoutmoreabout“run_resolve()”andtheresolverobject,pleaseconsulttheP4Scriptmanual.

2.5.4 FormhandlingTherearespecialcommandsforhandlingforms,suchas“change”,“branch”and“client”andallothercommandsthatnormallyopenaneditorfromthecommandlineandhave“‐o”and“‐i”optionsforredirectingoutputandinput.

Runningacommandsuchas“client”fromascriptasasimple“run”commandwouldopenaneditor,

whichisusuallynotwhatthescriptauthorintended.Insteadofrequiringuserstospecify“‐o”foroutputand“‐i”forinputforeveryspecform,thereareshortcutsthatprovidetheseoptionsautomatically.

Theseshortcutsare

Method Descriptionfetch_<form>(args) Equivalenttorun(“<form>”,‐o”,args)[0]save_<form>(spec,args) Equivalenttorun(“<form>”,“‐i”,“args”)withp4.input=specparse_<form>(textdoc) ParsesatextdocandconvertsitintoaSpecobjectformat_<form>(spec) Takesaspecandconvertsitintobackintoasinglestringdocumentdelete_<form>(args) Equivalenttorun(“<form>”,“‐d”,args)

ThenamingconventionforPerlisP4.Fetch<Form>(),forexample,P4.FetchClient().

ThespecobjectreturnedorexpectedasanargumentisofclassP4.Spec,whichisasubclassofthe

standardhashdictionaryclassforPerl,PythonandRuby.Thespecispre‐parsedandstoredaskey‐valuepairsinthehashdictionary.Thisgreatlysimplifieshandlingofforms.

P4.Specgeneratesasetofmethods(orattributesinPython)thatmakeiteasiertosetandretrieveindividualvaluesofaSpecobject.Thenameofthemethod(orattribute)isthelowercasenameofa

specfieldprecededbyanunderscore.Forexample,thefieldnameoftherootdirectoryforaclientworkspaceis“Root”.Theequivalentgeneratedattributeisthen“_root”.

Acoupleofexampleswillmakethisclearer.

HereisanexampleofhowtheclientworkspaceoptionscanbemodifiedwithafewlinesofPython:

13

Thefirstlineretrievesthespecoftheclientworkspace“myws”.Thesecondlineaccessthefield

“Options”,replaces“normdir”with“rmdir”andwritesthenewoptionsback.Thislinedemonstratesthatfieldscanbeaccessedeitherbyprovidingthekeytothestandardhashdictionarygetoperator,herethesquarebrackets[],orthroughthegeneratedattribute“_options”.

Finally,thenewspecissavesbacktotheserver.

Hereisanotherexample,thistimeinRuby,thatsubmitsachangebyretrievingachangespec,

settingthedescriptionandthenusingtheoverloaded“run_submit()”method:

Finally,anexampleinPerlthatsetstheoptionsforalabelto“locked”:

Theparse,formatanddeletemethodswillbedemonstratedinchapter4.

cl = p4.fetch_client(“myws”) cl._options = cl[“Options”].replace(“normdir”, “rmdir”) p4.save_client(cl)

ch = p4.fetch_change ch._description = “My latest changes.” p4.run_submit(ch)

$label = $p4->FetchLabel($labelName); $label->_options("locked"); $p4->SaveLabel($label);

14

3 P4.MapclassTheP4.Mapclasswasaddedtothescriptinginterfacesfor2008.2.P4.MapisawrapperaroundtherecentlyexposedmappinginterfaceintheP4API.

ThemappingAPIallowsaprogrammertousethesamemappingfacilitiesandalgorithmsthe

Perforceserverappliesforclientworkspaces,branchesandallothermaps.ItisnotnecessarytobeconnectedtoaPerforceservertousethemappinginterface;allcalculationsareperformedintheapplication.

Amaptranslatesalefthandside(lhs)intoarighthandside(rhs).Forexample,themap

//depot/MAIN/... //bruno_ws/MAIN/...

translatesthefile“//depot/MAIN/main.c”into“//bruno_ws/MAIN/main.c”.Mapsfollowthesame

rulesasaclientworkspacesmappingwithallfeaturesprovided,includingexclusionaryandoverlaymappingandreordering.

TheP4.MapconstructorandtheP4.Map.insert()methodacceptdifferentwaysofdescribingthemapobject:

‐ Asinglestringwithlhsandrhsseparatedbywhitespace.

‐ Twoseparatestringarguments.Thisisusefulifeithersidehaswhitespacesinthepath.‐ Anarrayofstringsforseveralmappinglines

Bothconstructorandinsert()methodcanalsotakeasinglestringrepresentingthelhs.Inthiscase,theleftandrighthandsideareidentical.

Somemethodsaredescribedbelow:

Method Descriptioninsert(arg) Addalineorsetoflinestothemappingclear() Removesalllinesfromthemaptranslate(pattern) Translatesthepatternfromlhstorhsreverse() Returnsamapwithlhsandrhsreversedincludes(pattern) Returnstrueifthepatterncanbemappedjoin(map1,map2) Classmethod,joinstomapstogether

Herearesomeexamplesonhowtousethemappinginterface:

import P4 map = P4.Map( [“//depot/src/... //ws/src/...”, “//depot/doc/... //ws/doc/...”] ) map.insert( “//with spaces/...”, “//ws/nospaces/...” ) map.insert( “-//depot/doc/excluded/... //ws/doc/excluded” ) map.includes( “//depot/doc/excluded/doc.txt” ) # => False map.includes( “//with spaces/hello.txt” ) # => True map.translate( “//with spaces/hello.txt” ) # => “//ws/nospaces/hello.txt”

15

Asyoucansee,inordertousethemapfromrighttoleft,youneedtocreateareversemapfirst,and

performthetranslationoperationonthatreversemap.

Thejoinclassmethodisusedtocombinetwomapstogether,creatingaproductofbothmappings.Themapsuseddonothavetobeofthesamesizeforthejointosucceed,astheseexamplesdemonstrate:

Inthefirstexample,map3iscreatedtoreducetheopenmapofmap1downtotwospecificdirectories.

Inthesecondexample,theresultmap5providesadirectmappingfromthedepotpathstothelocaldirectories.ThisisnotunlikethePerforceServeritselfmapsfilesfromthedepotsyntaxtothelocal

syntax.

Conclusion:

ThemappinginterfaceallowsuserstoemulatehowthePerforceServerperformsmappingswithouthavingtoconnecttotheserver.Wewillseeanexampleinchapter4.4.

map2 = map.reverse() # //ws/src/... //depot/src/... # //ws/doc/... //depot/doc/... # //ws/nospaces/... //with spaces/... # -//ws/doc/excluded/... //depot/doc/excluded/... map2.translate( “//ws/src/main.c” ) # => //depot/src/main.c map2.clear() map2 # => P4.Map object: (empty)

map1 = P4.Map( “//depot/...”, “//ws/...” ) map2 = P4.Map() map2.insert( “//depot/src/...” ) map2.insert( “//depot/doc/...” ) map3 = P4.Map.join( map2, map1 ) # //depot/src/... //ws/src/... # //depot/doc/... //ws/doc/... map4 = P4.Map( “//ws/...”, “/home/chris/works/...” ) map5 = P4.Map( map3, map4 ) # //depot/src/... /home/chris/works/src/... # //depot/doc/... /home/chris/works/doc/...

16

4 Examplesfromthewild

4.1 OverviewThePerforcescriptingAPIscanbeusedinawiderangeofsituations.Inthissection,wewillpresentsomeexamplesonhowtheseAPIscanbeusedtosolvesomecommonproblemslikemaintenanceclean‐up,form‐manipulatingtriggersandchangetriggers.Intheend,wewillpresentalarger

application.

TheseexamplesareprovidedinPython,butcanbetranslatedwithoutgreatefforttoPerlandRuby.ThealgorithmsandP4methodsusedstaythesame.

4.2 ScriptexampleAsimplescripttocleanupworkspacesolderthan6monthscouldbewrittenlikethis:

Thisscriptsimplyretrievesallclientworkspacedescriptionsfromtheserverandprocesseseachclient.Ifthetimeaclientworkspacewasaccessedthelasttimeismorethan26weeksago(agood

approximationof6months),wesimplyforceadeleteofthisworkspace.

InthisexamplewealsoseetheuseoftheP4.delete_<form>()method,whichissimplyaconvenientshortcutofthecommand“p4<form>‐d”.

Anactuallydeployedscriptwouldprobablycheckthedescriptionofeachworkspacetolookfora

“DONODELETEME”markerorsomeotherwaytoidentifyworkspacesthatshouldbeexempt.

4.3 Triggerexamples

4.3.1 Form­outtrigger:SettingtemplatesfornewclientsForm‐outtriggersprovetobeagoodexampletoseetheparse_<form>()andformat_<form>()methodsinaction.

Aform‐outtriggerdoesnotreceivetheformspec,butinsteadthefilenameofthetemporaryfile

generatedbytheserver.Insteadofusingregularexpressionsandsearchandreplacestatementstomodifythistemporaryfiletoprovidethedefaultsettingsrequiredbythistrigger,wearegoingtoconvertthefilecontentintoaP4.Specobject,manipulatethisobjectinthiseasilyaccessibleform

andthenconverttheobjectbackintoitsoriginalfileformat.

Butbeforewecandothat,weneedtosolveanotherproblemfirst,typicalforform‐triggers:howdoweknowthatthisformiscreatedthefirsttime?Inmostcases,wedonotwanttopreventthemodificationofexistingforms.

from P4 import P4 from time import time p4 = P4() p4.connect() for c in p4.run_clients(): age = time.time() – int(c.[‘Access’]) if age > 86400 * 7 * 26: # 26 weeks p4.delete_client(“-f”, c.[‘client’]) p4.disconnect()

17

Forclients,jobs,labelsandbranches,wecannow(2008.2)usethenew“‐e”optiontosearchforthenameoftheform.Foraform‐outtriggercalledonanewform,thenamedoesnotexistyetinthe

databaseandthereforethesearchwillfail.

Forusersandgroups,weneedtoloadallexistingusersorgroupsintothescriptandthensearchforthenamegiventoourformtriggerscript.Thiscangetpotentiallyexpensiveiftherearemanyusersorgroupsontheserveralready.

Forform‐outtriggersonnewclients,thereexistsanothertrickthatcanbeusedonPerforceServers

before2008.2:

Thecommand“p4info”intaggedformwill,foranunknownclient,returntheclientname“*unknown*”.Wecanusethistoouradvantage:

Nowthatweknowthatwearedealingwithanewclient,wecanproceedtoloadthetemporaryfilegeneratedbytheserverintomemoryandconvertitintoaP4.Specobject.Thenwewillsetthe

defaultsandwritethetemporaryfileback.

4.3.2 Changetrigger:AtriggerbaseclassChange‐submitandChange‐contenttriggerstendtofollowthesamepattern:

Thetriggeriscalledwiththechangenumber.Itthenloadsthechangespecintomemoryviaa“p4describe”.Thetriggerscriptprocessesthechangeinformation,suchasthedescription,thelist

offilesortheassociatedjobs,anddetermineswhethertoletthechangepassorwhethertorejectit.

Insteadofcodingthesamebasicmechanismeverytime,youcoulddefineasinglebaseclassthatperformsallthecommonoperationsandleavesittothederivedclasstovalidatethechange.

import P4, sys p4 = P4() p4.client = sys.argv[1] # the client name passed to the form trigger p4.connect() clientInfo = p4.run_info()[0][‘clientName’] if clientInfo != ‘*unknown*’: # client already exists p4.disconnect() sys.exit(0) # trigger exits successfully w/o modifying form

filename = sys.argv[2] # the filename passed to our form trigger with open(filename, “r”) as f: clientAsString = f.read() client = p4.parse_client(clientAsString) client._options = myDefaultOptions client._submitoptions = MyDefaultSubmitOptions clientAsString = p4.format_client(client) with open(filename, “w”) as f: f.write(clientAsString) p4.disconnect() sys.exit(0)

18

SuchbasetriggershavebeenwrittenforP4PythonandP4Ruby.TheyarenotpartofthestandardinstallationbutcanbedownloadedfromthepublicPerforceserverattheselocations:

• //guest/sven_erik_knop/triggers

P4PythonversionP4Triggers.py

• //guest/tony_smith/perforce/P4Rubylib/triggers/P4RubyversionP4Triggers.rb

ThereiscurrentlynochangetriggerbaseclassforPerl.

TocreateatriggerusingtheP4Triggerbaseclass,extendtheP4Triggerclassandoverrideatleastthevalidate()method.YouroverwrittenversionshouldreturnaBooleanindicatingwhethertolet

thechangepassornot.

ThereisanexampleforeachPython(CheckCaseTrigger.py)andRuby(checkcase.rb),thatshowhowtousethesebaseclasses.

Thetriggerexamplecanbeusedtoensurethatnodirectories(Ruby)ordirectoriesandfiles(Python)aresubmittedthatalreadyexistinadifferentcasespelling.ThePythonversioncanalsodealwith

UnicodePerforceServers.

4.4 Applicationexample:P4BucketAnexampleforacompleteapplicationwrittenusingP4PythonisP4Bucket.

ThescriptanditsdocumentationcanbedownloadedfromthepublicPerforceserverhere:

//guest/sven_erik_knop/p4bucket

ThescriptallowsadministratorstoarchivebinaryfilesstoredinPerforcetoabucket.Abucketissimplyanameandafilelocation.Archivingleavesaplaceholderfile(a“ghost”)inthePerforce

depotthatexplainsthatthefilehasbeenarchivedandwhenthishappened.

Archivedfilescanthenberestoredagainfromthebucket.ThehistoryofthefileinthePerforceMetadataisunaffectedbythis,butthedigestofthearchivedfileisadjustedtomakesurethata“p4verify”doesnotshowupanyerrors.

Comparesthistothealternativeofsettingthe+S<n>filetypeattribute.Whenanewrevisionis

created,theserverwillpurgethepreviousrevisionbydeletingthefilefromthedepotirrevocably.Thereisnorestore.

P4BucketrequiresPython2.5or2.6andP4Python2008.2orhighertorun,sinceitmakesuseofthenewP4.Mapclass.

HerearesomeexcerptsfromtheP4BucketscriptthatdemonstratehowthescriptingAPIisused.

4.4.1 CreatingthemapThefirstexamplesolvestheproblemofhowtomapadepotfilenametothephysicallocationofthisfileinthedepotdirectories.ThisisdoneusingtheP4.Map.Themapisinitializedusingthedepot

nameasthelefthandsideandthephysicallocationastherighthandside.

19

ThephysicallocationofthedepotiseitherinsidetheP4ROOTdirectory,inwhichcasethe“map”fieldwillbearelativepath,oroutsideofP4ROOT;the“map”fieldthencontainsanabsolutepath.

IfthemapisrelativetoP4ROOT,weneedtoaddtheserverrootdirectorytoittogettheabsolute

locationofthedepot.

4.4.2 TranslatingtheargumentsintofilelocationsThenextstepisthentoconvertthefileargumentspassedtotheP4Bucketscriptbytheuserintopairofdepotfilenameandphysicallocation.

Thisisdoneintwosteps.First,thescriptperformsa“files‐a”onthefileargumentstoretrieveallfile

revisions.Theresultisfilteredtoincludeonlyeligiblefilerevisions,thatis,onlyfilerevisionsthatdonothavetheaction“delete”andareoftype“binaryfullfile”or“binarycompressedfile”.

Thiscandidatelististhenusedastheargumenttoanfstatcommandthatretrievesallthenecessaryinformationonthisfile,suchasthelibrarianinformation,thedigest,theattributesandthelistof

lazycopiescreatedfromthisinstance.

Iftherevisionhasnotbeenarchivedalreadyandifitisnottheheadrevision,thesourceoflazycopiesorisalazycopyitself,wecanthentranslatethelogicallocationofthefiletothephysicallocationthroughourmap.

4.4.3 SettingofattributesandrecalculationofthedigestFinally,oncethefileismovedtothebucketandreplacedwiththeplaceholder,wesetsomeattributestothisrevisiontorememberwherethefilewasarchivedandbywhomandwhen.Then

werecalculatethedigestofthisrevisiontoensurethata“p4verify”doesnotflagthearchivedrevisionas“BAD!”

# build the map from depot to file location serverRoot = p4.run_info()[0][‘serverRoot’] depotMap = P4.Map() for depot in p4.run_depots(): if depot["type"] != "remote": map = depot["map"] # depotname/... if not absolute_path(map): map = serverRoot + "/" + map depotMap.insert( "//%s/..." % depot["name"], map )

c = [] # candidate list for f in p4.run_files(“-a”, pattern): if candidate(f): c.append(f[“depotFile”]+“#”+f[“rev”]) if len(c) > 0: for s in p4.run_fstat(“-Oazcl”, c):

if archiveable(s): d = depotMap.translate(s[“lbrFile”])

20

# thisRev contains the full name of the depot file revision # rev contains only the revision including the “#” symbol p4.run_attribute("-f", "-n", "archiveUser", "-v", self.p4.user, thisRev) p4.run_attribute("-f", "-n", "archiveDate", "-v", now, thisRev) p4.run_attribute("-f", "-n", "archiveDigest", "-v", afile["digest"], thisRev) p4.run_attribute("-f", "-n", "archiveBucket", "-v", name, thisRev) # reverify to set the correct digest for the replaced afile # need the form file#rev,#rev, otherwise all previous revs get verified as well p4.run_verify("-v", thisRev + "," + rev)

21

5 Outlookandconclusion

5.1 OutlookWhatisgoingtohappentotheP4Scriptinterfacesinthefuture?

WethinktheinterfaceisnowverystableandexpectonlyminorchangesinthefuturetothecoreAPI,mostofwhichwillbebackwardscompatible.

Onesmallupcomingchangeisthesupportforblocks(inRuby)orcontexts(inPython).ForP4Ruby,therealreadyexistssuchamethod:P4#at_exception_level(level).Whenpassedablockofcode,

thatcodeisrunundertheexception_leveldefinedintheargument,thenthelevelisresettoitsoldvalue.

AsimilarconstructisavailableinPython:

Weplantosupportthiskindofconstructforconnect(),exception_levelandtaggedmodefornow.

Theremightalsobescopeforotherlanguageextension,shouldthedemandarise.

5.2 ConclusionsP4Perl,P4RubyandP4PythonarewellestablishedtoolsthatarefullysupportedbyPerforce.

Thereisawiderangeofuses,fromsimplescriptstofullapplications.

Examplescanbefoundinthepublicdepot.

with p4.at_exception_level(P4.RAISE_ERRORS): p4.run_sync(“...”)