Unix Linux Introduction

1FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

TheUnixandGNU/Linuxcommandline

TheUnixandGNU/Linux

commandline

MichaelOpdenackerThomasPetazzoniFreeElectrons

Copyright2009,FreeElectrons.CreativeCommonsBYSA3.0licenseLatestupdate:Jul15,2010,Documentsources,updatesandtranslations:http://freeelectrons.com/docs/commandlineCorrections,suggestions,contributionsandtranslationsarewelcome!


Commandmementosheet

Itisausefulcompaniontothispresentation.

Examplesforthemostusefulcommandsaregiveninjustonesheet.

SuggestionsforuseStickthissheetonyourwall,useitasdesktopwallpaper,makeitamousemat,printitonclothing,sliceitintobookmarks...

CautionStoreawayfrommice!

Getitonhttp://freeelectrons.com/docs/commandline


TrainingContents(1)

Shells,filesystemandfilehandling

Everythingisafile

GNU/Linuxfilesystemstructure

Commandlineinterpreters

Handlingfilesanddirectories

Displaying,scanningandsortingfiles

Symbolicandhardlink

Fileaccessrights


Trainingcontents(2)

StandardI/O,redirections,pipes

Standardinputandoutput,redirectingtofiles

Pipes:redirectingstandardoutputtoothercommands

Standarderror


TrainingContents(3)

Taskcontrol

Fullcontrolontasks

Executinginbackground,suspending,resumingandaborting

Listofactivetasks

Killingprocesses

Environmentvariables

PATHenvironmentvariables

Shellaliases,.bashrcfile


Trainingcontents(4)

Miscellaneous

Texteditors

Compressingandarchiving

Printingfiles

Comparingfilesanddirectories

Lookingforfiles

Gettinginformationaboutusers



Unixfilesystem


Everythingisafile

Regularfiles

DirectoriesDirectoriesarejustfileslistingasetoffiles

SymboliclinksFilesreferringtothenameofanotherfile

DevicesandperipheralsReadandwritefromdevicesaswithregularfiles

PipesUsedtocascadeprogramscat*.log|greperror

SocketsInterprocesscommunication

AlmosteverythinginUnixisafile!


Filenames

FilenamefeaturessincethebeginningofUnix

Casesensitive

Noobviouslengthlimit

Cancontainanycharacter(includingwhitespace,except/).Filetypesstoredinthefile(magicnumbers).Filenameextensionsnotneededandnotinterpreted.Justusedforuserconvenience.

Filenameexamples:README .bashrc WindowsBuglistindex.htm index.html index.html.old


Filepaths

Apathisasequenceofnesteddirectorieswithafileordirectoryattheend,separatedbythe/character

Relativepath:documents/fun/microsoft_jokes.htmlRelativetothecurrentdirectory

Absolutepath:/home/bill/bugs/crash9402031614568

/:rootdirectory.Startofabsolutepathsforallfilesonthesystem(evenforfilesonremovabledevicesornetworkshared).


GNU/Linuxfilesystemstructure(1)

Notimposedbythesystem.Canvaryfromonesystemtotheother,evenbetweentwoGNU/Linuxinstallations!

/ Rootdirectory/bin/ Basic,essentialsystemcommands/boot/ Kernelimages,initrdandconfigurationfiles/dev/ Filesrepresentingdevices

/dev/hda:firstIDEharddisk/etc/ Systemconfigurationfiles/home/ Userdirectories/lib/ Basicsystemsharedlibraries



/lost+found Corruptfilesthesystemtriedtorecover/media Mountpointsforremovablemedia:

/media/usbdisk,/media/cdrom/mnt/ Mountpointsfortemporarilymountedfilesystems/opt/ Specifictoolsinstalledbythesysadmin

/usr/local/oftenusedinstead/proc/ Accesstosysteminformation

/proc/cpuinfo,/proc/version.../root/ rootuserhomedirectory/sbin/ Administratoronlycommands/sys/ Systemanddevicecontrols

(cpufrequency,devicepower,etc.)



/tmp/ Temporaryfiles/usr/ Regularusertools(notessentialtothesystem)

/usr/bin/,/usr/lib/,/usr/sbin.../usr/local/ Specificsoftwareinstalledbythesysadmin

(oftenpreferredto/opt/)/var/ Datausedbythesystemorsystemservers

/var/log/,/var/spool/mail(incomingmail),/var/spool/lpd(printjobs)...

TheUnixfilesystemstructureisdefinedbytheFilesystemHierarchyStandard(FHS):http://www.pathname.com/fhs/



Shellsandfilehandling


Commandlineinterpreters

Shells:toolstoexecuteusercommands

Calledshellsbecausetheyhidethedetailsontheunderlyingoperatingsystemundertheshell'ssurface.

Commandsareinputinatextterminal,eitherawindowinagraphicalenvironmentoratextonlyconsole.

Resultsarealsodisplayedontheterminal.Nographicsareneededatall.

Shellscanbescripted:providealltheresourcestowritecomplexprograms(variable,conditionals,iterations...)


Wellknownshells

Mostfamousandpopularshells

sh:TheBourneshell(obsolete)Traditional,basicshellfoundonUnixsystems,bySteveBourne.

csh:TheCshell(obsolete)OncepopularshellwithaClikesyntax

tcsh:TheTCshell(stillverypopular)ACshellcompatibleimplementationwithevolvedfeatures(commandcompletion,historyeditingandmore...)

bash:TheBourneAgainshell(mostpopular)Animprovedimplementationofshwithlotsofaddedfeaturestoo.


fish:agreatnewshell

TheFriendlyInteractiveSHellhttp://www.fishshell.org/

Standardfeatures:history,commandandfilecompletion...

Brandnewfeatures:commandoptioncompletion,commandcompletionwithshortdescription,syntaxhighlighting..

Easiertoanyopenfiles:openbuiltincommand.

Muchsimplerandconsistentsyntax(notPOSIXcompliant)Makesiteasiertocreateshellscripts.

Commandlinebeginnerscanlearnmuchfaster!Evenexperiencedusersshouldfindthisshellveryconvenient.


lscommand

lsa(all)Listsallthefiles(including.*files)

lsl(long)Longlisting(type,date,size,owner,permissions)

lst(time)Liststhemostrecentfilesfirst

lsS(size)Liststhebiggestfilesfirst

lsr(reverse)Reversesthesortorder

lsltr(optionscanbecombined)Longlisting,mostrecentfilesattheend

Liststhefilesinthecurrentdirectory,inalphanumericorder,exceptfilesstartingwiththe.character.


Filenamepatternsubstitutions

Betterintroducedbyexamples!

ls*txtTheshellfirstreplaces*txtbyallthefileanddirectorynamesendingbytxt(including.txt),exceptthosestartingwith.,andthenexecutesthelscommandline.

lsd.*Listsallthefilesanddirectoriesstartingwith.dtellslsnottodisplaythecontentsofdirectories.

cat?.logDisplaysallthefileswhichnamesstartby1characterandendby.log


Specialdirectories(1)

./

Thecurrentdirectory.Usefulforcommandstakingadirectoryargument.Alsosometimesusefultoruncommandsinthecurrentdirectory(seelater).

So./readme.txtandreadme.txtareequivalent.

../

Theparent(enclosing)directory.Alwaysbelongstothe.directory(seelsa).Onlyreferencetotheparentdirectory.

Typicalusage:cd..


Specialdirectories(2)

~/

Notaspecialdirectoryindeed.Shellsjustsubstituteitbythehomedirectoryofthecurrentuser.

Cannotbeusedinmostprograms,asitisnotarealdirectory.

~sydney/

Similarly,substitutedbyshellsbythehomedirectoryofthesydneyuser.


Thecdandpwdcommands

cdChangesthecurrentdirectoryto.

cdGetsbacktothepreviouscurrentdirectory.

pwdDisplaysthecurrentdirectory("workingdirectory").


Thecpcommand

cpCopiesthesourcefiletothetarget.

cpfile1file2file3...dirCopiesthefilestothetargetdirectory(lastargument).

cpi(interactive)Asksforuserconfirmationifthetargetfilealreadyexists

cpr(recursive)Copiesthewholedirectory.


mvandrmcommands

mv (move)Renamesthegivenfileordirectory.

mvi(interactive)Ifthenewfilealreadyexits,asksforuserconfirm

rmfile1file2file3... (remove)Removesthegivenfiles.

rmi(interactive)Alwaysaskforuserconfirm.

rmrdir1dir2dir3(recursive)Removesthegivendirectorieswithalltheircontents.


Creatingandremovingdirectories

mkdirdir1dir2dir3...(makedir)Createsdirectorieswiththegivennames.

rmdirdir1dir2dir3...(removedir)RemovesthegivendirectoriesSafe:onlyworkswhendirectoriesandempty.Alternative:rmr(doesn'tneedemptydirectories).


Displayingfilecontents

Severalwaysofdisplayingthecontentsoffiles.

catfile1file2file3...(concatenate)Concatenatesandoutputsthecontentsofthegivenfiles.

morefile1file2file3...Aftereachpage,askstheusertohitakeytocontinue.Canalsojumptothefirstoccurrenceofakeyword(/command).

lessfile1file2file3...Doesmorethanmorewithless.Doesn'treadthewholefilebeforestarting.Supportsbackwardmovementinthefile(?command).


Theheadandtailcommands

head[]Displaysthefirstlines(or10bydefault)ofthegivenfile.Doesn'thavetoopenthewholefiletodothis!

tail[]Displaysthelastlines(or10bydefault)ofthegivenfile.NoneedtoloadthewholefileinRAM!Veryusefulforhugefiles.

tailf(follow)Displaysthelast10linesofthegivenfileandcontinuestodisplaynewlineswhentheyareappendedtothefile.Veryusefultofollowthechangesinalogfile,forexample.

Examplesheadwindows_bugs.txttailfoutlook_vulnerabilities.txt


Thegrepcommand

grepScansthegivenfilesanddisplaysthelineswhichmatchthegivenpattern.

greperror*.logDisplaysallthelinescontainingerrorinthe*.logfiles

grepierror*.logSame,butcaseinsensitive

greprierror.Same,butrecursivelyinallthefilesin.anditssubdirectories

grepvinfo*.logOutputsallthelinesinthefilesexceptthosecontaininginfo.


Thesortcommand

sortSortsthelinesinthegivenfileincharacterorderandoutputsthem.

sortrSame,butinreverseorder.

sortruu:unique.Same,butjustoutputsidenticallinesonce.

Morepossibilitiesdescribedlater!


Thesedcommand

sedisaStreamEDitor

Itparsestextfilesandimplementsaprogramminglanguagetoapplytransformationsonthetext.

Oneofthemostcommonusageofsedistextreplacement,whichreliesonregularexpressions

sede's/abc/def/'testfilewillreplaceeverystringabcbydefinthefiletestfileanddisplaytheresultonthestandardoutput.

sed's/^[\t]*//'testfilewillremoveanytabulationorspaceatthebeginningofaline

sed's/^|$[^|]*$|$[^|]*$|$/\1>\2/'testfilereplacelineslike |string1|string2|by string1>string2


sed:regularexpressions

RegularexpressionsareusefulinmanyUnixtools,notonlysed.Theyallowtomatchtheinputtextagainstanexpression.

.matchesanycharacter

[]matchesanycharacterlistedinsidethebrackets

[^]matchesanycharacternotlistedinsidethebrackets

^matchesthebeginningoftheline

$matchestheendoftheline

*matchesthepreviouselementzeroormoretimes,+matchesthepreviouselementoneormoretimes,?matchesthepreviouselementzerooronetime

definesasubexpressionthatcanbelaterrecalledbyusing\n,wherenisthenumberofthesubexpressionintheregularexpression

Moreathttp://www.regularexpressions.info/


Symboliclinks

Asymboliclinkisaspecialfilewhichisjustareferencetothenameofanotherone(fileordirectory):

Usefultoreducediskusageandcomplexitywhen2fileshavethesamecontent.

Example:anakin_skywalker_biography>darth_vador_biography

Howtoidentifysymboliclinks:

lsldisplays>andthelinkedfilename.

GNUlsdisplayslinkswithadifferentcolor.


Creatingsymboliclinks

Tocreateasymboliclink(sameorderasincp):lnsfile_namelink_name

Tocreatealinkwithtoafileinanotherdirectory,withthesamename:lns../README.txt

Tocreatemultiplelinksatonceinagivendirectory:lnsfile1file2file3...dir

Toremovealink:rmlink_nameOfcourse,thisdoesn'tremovethelinkedfile!


Hardlinks

Thedefaultbehaviorforlnistocreatehardlinks

Ahardlinktoafileisaregularfilewithexactlythesamephysicalcontents

Whiletheystillsavespace,hardlinkscan'tbedistinguishedfromtheoriginalfiles.

Ifyouremovetheoriginalfile,thereisnoimpactonthehardlinkcontents.

Thecontentsareremovedwhentherearenomorefiles(hardlinks)tothem.


Filesnamesandinodes

Makeshardandsymbolic(soft)linkseasiertounderstand!

Inode

File HardlinkSoftlink

Inodeinterface

Filenameinterface

Filesystem

Users

rmrm

Inode



Commanddocumentation


Commandhelp

SomeUnixcommandsandmostGNU/Linuxcommandsofferatleastonehelpargument:

h(ismostlyusedtointroduce1characteroptions)

help(isalwaysusedtointroducethecorrespondinglongoptionname,whichmakesscriptseasiertounderstand)

Youalsooftengetashortsummaryofoptionswhenyouinputaninvalidargument.


Manualpages

manDisplaysoneorseveralmanualpagesfor

manman

MostavailablemanualpagesareaboutUnixcommands,butsomearealsoaboutCfunctions,headersordatastructures,orevenaboutsystemconfigurationfiles!

manstdio.h

manfstab(for/etc/fstab)

ManualpagefilesarelookedforinthedirectoriesspecifiedbytheMANPATHenvironmentvariable.


Infopages

InGNU,manpagesarebeingreplacedbyinfopages.Somemanualpageseventelltorefertoinfopagesinstead.

info

infofeatures:

Documentationstructuredinsections(nodes)andsubsections(subnodes)

Possibilitytonavigateinthisstructure:top,next,prev,up

InfopagesgeneratedfromthesametexinfosourceastheHTMLdocumentationpages


SearchingtheInternetforresources(2)

Lookingfordocumentation

Lookfororpagetofindthetoolorprojecthomepageandthenfindthelatestdocumentationresources.

Lookfordocumentationormanualinyourfavoritesearchengine.

Lookingforgenerictechnicalinformation

WikiPedia:http://wikipedia.orgLotsofusefuldefinitionsincomputerscience.Arealencyclopedia!Opentoanyone'scontributions.


SearchingtheInternetforresources(1)

Investigatingissues

Mostforumsandmailinglistarchivesarepublic,andareindexedonaveryfrequentbasisbyGoogle.

Ifyouinvestigateanerrormessage,copyitverbatiminthesearchform,enclosedindoublequotes(errormessage).Lotsofchancesthatsomebodyelsealreadyfacedthesameissue.

Don'tforgettouseGoogleGroups:http://groups.google.com/Thissiteindexesmorethan20yearsofnewsgroupsmessages.



Usersandpermissions


Fileaccessrights

3typesofaccessrights

Readaccess(r)

Writeaccess(w)

Executerights(x)

3typesofaccesslevels

User(u):fortheownerofthefile

Group(g):eachfilealsohasagroupattribute,correspondingtoagivenlistofusers

Others(o):forallotherusers

Uselsltocheckfileaccessrights


Accessrightconstraints

xissufficienttoexecutebinariesBothxandrandrequiredforshellscripts.

Bothrandxpermissionsneededinpracticefordirectories:rtolistthecontents,xtoaccessthecontents.

Youcan'trename,remove,copyfilesinadirectoryifyoudon'thavewaccesstothisdirectory.

Ifyouhavewaccesstoadirectory,youCANremoveafileevenifyoudon'thavewriteaccesstothisfile(rememberthatadirectoryisjustafiledescribingalistoffiles).Thisevenletsyoumodify(remove+recreate)afileevenwithoutwaccesstoit.


Accessrightsexamples

rwrrReadableandwritableforfileowner,onlyreadableforothers

rwrReadableandwritableforfileowner,onlyreadableforusersbelongingtothefilegroup.

drwxDirectoryonlyaccessiblebyitsowner

rxFileexecutablebyothersbutneitherbyyourfriendsnorbyyourself.Niceprotectionsforatrap...


chmod:changingpermissions

chmod2formatsforpermissions:

Octalformat(abc):a,b,c=r*4+w*2+x(r,w,x:booleans)Example:chmod644(rwforu,rforgando)

Orsymbolicformat.Easytounderstandbyexamples:chmodgo+r:addreadpermissionstogroupandothers.chmoduw:removewritepermissionsfromuser.chmodax:(a:all)removeexecutepermissionfromall.


Morechmod(1)

chmodRa+rXlinux/Makeslinuxandeverythinginitavailabletoeveryone!

R:applychangesrecursively

X:x,butonlyfordirectoriesandfilesalreadyexecutableVeryusefultoopenrecursiveaccesstodirectories,withoutaddingexecutionrightstoallfiles.


Morechmod(2)

chmoda+t/tmp

t:(sticky).Specialpermissionfordirectories,allowingonlythedirectoryandfileownertodeleteafileinadirectory.

Usefulfordirectorieswithwriteaccesstoanyone,like/tmp.

Displayedbylslwithatcharacter.


Fileownership

Particularlyusefulin(embedded)systemdevelopmentwhenyoucreatefilesforanothersystem.

chownRsco/home/linux/src(R:recursive)Makesuserscothenewownerofallthefilesin/home/linux/src.

chgrpRempire/home/askywalkerMakesempirethenewgroupofeverythingin/home/askywalker.

chownRborg:aliensusss_entreprise/chowncanbeusedtochangetheownerandgroupatthesametime.


Bewareofthedarksideofroot

rootuserprivilegesareonlyneededforveryspecifictaskswithsecurityrisks:mounting,creatingdevicefiles,loadingdrivers,startingnetworking,changingfileownership,packageupgrades...

Evenifyouhavetherootpassword,yourregularaccountshouldbesufficientfor99.9%ofyourtasks(unlessyouareasystemadministrator).

Inatrainingsession,itisacceptabletouseroot.Inreallife,youmaynotevenhaveaccesstothisaccount,orputyoursystemsanddataatriskifyoudo.


Usingtherootaccount

Incaseyoureallywanttouseroot...

Ifyouhavetherootpassword:su(switchuser)

Inmoderndistributions,thesudocommandgivesyouaccesstosomerootprivilegeswithyourownuserpassword.Example:sudomount/dev/hda4/home



StandardI/O,redirections,pipes


Standardoutput

Moreaboutcommandoutput

Allthecommandsoutputtingtextonyourterminaldoitbywritingtotheirstandardoutput.

Standardoutputcanbewritten(redirected)toafileusingthe>symbol

Standardoutputcanbeappendedtoanexistingfileusingthe>>symbol


Standardoutputredirectionexamples

ls~saddam/*>~gwb/weapons_mass_destruction.txt

catobiwan_kenobi.txt>starwars_biographies.txtcathan_solo.txt>>starwars_biographies.txt

echoREADME:Nosuchfileordirectory>READMEUsefulwayofcreatingafilewithoutatexteditor.NiceUnixjoketoointhiscase.


Standardinput

Moreaboutcommandinput

Lotsofcommands,whennotgiveninputarguments,cantaketheirinputfromstandardinput.

sortwindowslinux[Ctrl][D]linuxwindows

sort


Pipes

Unixpipesareveryusefultoredirectthestandardoutputofacommandtothestandardinputofanotherone.

Examples

cat*.log|grepierror|sort

greprierror.|grepvignored|sortu\>serious_errors.log

cat/home/*/homework.txt|grepmark|more

ThisoneofthemostpowerfulfeaturesinUnixshells!


Theteecommand

tee[a]file

Theteecommandcanbeusedtosendstandardoutputtothescreenandtoafilesimultaneously.

make|teebuild.logRunsthemakecommandandstoresitsoutputtobuild.log.

makeinstall|teeabuild.logRunsthemakeinstallcommandandappendsitsoutputtobuild.log.


Standarderror

Errormessagesareusuallyoutput(iftheprogramiswellwritten)tostandarderrorinsteadofstandardoutput.

Standarderrorcanberedirectedthrough2>or2>>

Example:catf1f2nofile>newfile2>errfile

Note:1isthedescriptorforstandardoutput,so1>isequivalentto>.

Canredirectbothstandardoutputandstandarderrortothesamefileusing&>:catf1f2nofile&>wholefile


Theyescommand

Usefultofillstandardinputwithalwaysthesamestring.

yes|Keepsfillingthestandardinputofwith(ybydefault).

Examplesyes|rmrdir/bank>yesno|credit_applicantyes""|makeoldconfig(equivalenttohitting[Enter]toacceptalldefaultsettings)


Specialdevices(1)

Devicefileswithaspecialbehaviororcontents

/dev/nullThedatasink!Discardsalldatawrittentothisfile.Usefultogetridofunwantedoutput,typicallyloginformation:mplayerblack_adder_4th.avi&>/dev/null

/dev/zeroReadsfromthisfilealwaysreturn\0charactersUsefultocreateafilefilledwithzeros:ddif=/dev/zeroof=disk.imgbs=1kcount=2048

Seemannullormanzerofordetails


Specialdevices(2)

/dev/randomReturnsrandombyteswhenread.Mainlyusedbycryptographicprograms.Usesinterruptsfromsomedevicedriversassourcesoftruerandomness(entropy).Readscanbeblockeduntilenoughentropyisgathered.

/dev/urandomForprogramsforwhichpseudorandomnumbersarefine.Alwaysgeneratesrandombytes,evenifnotenoughentropyisavailable(inwhichcaseitispossible,thoughstilldifficult,topredictfuturebytesequencesfrompastones).

Seemanrandomfordetails.


Specialdevices(3)

/dev/fullMimicsafulldevice.Usefultocheckthatyourapplicationproperlyhandlesthiskindofsituation.

Seemanfullfordetails.



Taskcontrol


Fullcontrolontasks

Sincethebeginning,Unixsupportstruepreemptivemultitasking.

Abilitytorunmanytasksinparallel,andabortthemeveniftheycorrupttheirownstateanddata.

Abilitytochoosewhichprogramsyourun.

Abilitytochoosewhichinputyourprogramstakes,andwheretheiroutputgoes.


Processes

EverythinginUnixisafileEverythinginUnixthatisnotafileisaprocess

Processes

Instancesofarunningprograms

Severalinstancesofthesameprogramcanrunatthesametime

Dataassociatedtoprocesses:Openfiles,allocatedmemory,stack,processid,parent,priority,state...


Runningjobsinbackground

Sameusagethroughoutalltheshells

Useful

Forcommandlinejobswhichoutputcanbeexaminedlater,especiallyfortimeconsumingones.

Tostartgraphicalapplicationsfromthecommandlineandthencontinuewiththemouse.

Startingatask:add&attheendofyourline:

find_prince_charmingcutecleverrich&


Backgroundjobcontrol

jobsReturnsthelistofbackgroundjobsfromthesameshell

[1]Running~/bin/find_meaning_of_lifewithoutgod&[2]+Runningmakemistakes&

fgfg%Putsthelast/nthbackgroundjobinforegroundmode

Movingthecurrenttaskinbackgroundmode:[Ctrl]Zbg

kill%Abortsthenthjob.


Jobcontrolexample

>jobs[1]Running~/bin/find_meaning_of_lifewithoutgod&[2]+Runningmakemistakes&

>fgmakemistakes

>[Ctrl]Z[2]+Stoppedmakemistakes

>bg[2]+makemistakes&

>kill%1[1]+Terminated~/bin/find_meaning_of_lifewithoutgod


Listingallprocesses

...whatevershell,scriptorprocesstheyarestartedfrom

psuxListsalltheprocessesbelongingtothecurrentuser

psaux(Note:psedfonSystemVsystems)Listsalltheprocessesrunningonthesystempsaux|grepbart|grepbashUSERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMANDbart30390.00.259161380pts/2S14:350:00/bin/bashbart31340.00.253881380pts/3S14:360:00/bin/bashbart31900.00.263681360pts/4S14:370:00/bin/bashbart34160.00.000pts/2RW15:070:00[bash]

PID: ProcessidVSZ: Virtualprocesssize(code+data+stack)RSS: Processresidentsize:numberofKBcurrentlyinRAMTTY: TerminalSTAT: Status:R(Runnable),S(Sleep),W(paging),Z(Zombie)...


Liveprocessactivity

topDisplaysmostimportantprocesses,sortedbycpupercentage

top15:44:33up1:11,5users,loadaverage:0.98,0.61,0.59Tasks:81total,5running,76sleeping,0stopped,0zombieCpu(s):92.7%us,5.3%sy,0.0%ni,0.0%id,1.7%wa,0.3%hi,0.0%siMem:515344ktotal,512384kused,2960kfree,20464kbuffersSwap:1044184ktotal,0kused,1044184kfree,277660kcached

PIDUSERPRNIVIRTRESSHRS%CPU%MEMTIME+COMMAND3809jdoe250625639321312R93.80.80:21.49bunzip22769root160157m80m90mR2.716.05:21.01X3006jdoe1503092815m27mS0.33.00:22.40kdeinit3008jdoe16056248924468S0.30.20:06.59autorun3034jdoe1502676412m24mS0.32.50:12.68kscd3810jdoe16028929161620R0.30.20:00.06top

YoucanchangethesortingorderbytypingM:Memoryusage,P:%CPU,T:Time.

Youcankillataskbytypingkandtheprocessid.


Killingprocesses(1)

killSendsanabortsignaltothegivenprocesses.Letsprocessessavedataandexitbythemselves.Shouldbeusedfirst.Example:kill3039313431903416

kill9Sendsanimmediateterminationsignal.Thesystemitselfterminatestheprocesses.Usefulwhenaprocessisreallystuck(doesn'tanswertokill1).

kill91Killsalltheprocessesofthecurrentuser.1:meansallprocesses.


Killingprocesses(2)

killall[]Killsallthejobsrunning.Example:killallbash

xkillLetsyoukillagraphicalapplicationbyclickingonit!Veryquick!Convenientwhenyoudon'tknowtheapplicationcommandname.


Recoveringfromstuckgraphics

Ifyourgraphicalsessionisstuckandyoucannolongertypeinyourterminals,don'treboot!

Itisverylikelythatyoursystemisstillfine.Trytoaccessatextconsolebypressingthe[Ctrl][Alt][F1]keys(or[F2],[F3]formoretextconsoles)

Inthetextconsole,youcantrytokilltheguiltyapplication.

Oncethisisdone,youcangobacktothegraphicsessionbypressing[Ctrl][Alt][F5]or[Ctrl][Alt][F7](dependingonyourdistribution)

Ifyoucan'tidentifythestuckprogram,youcanalsokillallyourprocesses:kill91Youarethenbroughtbacktotheloginscreen.


Sequentialcommands

Cantypethenextcommandinyourterminalevenwhenthecurrentoneisnotover.

Canseparatecommandswiththe;symbol:echoIlovethee;sleep10;echonot

Conditionals:use||(or)or&&(and):moreGod||echoSorry,Goddoesn'texistRunsechoonlyifthefirstcommandfails

ls~sd6&&cat~sd6/*>~sydney/recipes.txtOnlycatsthedirectorycontentsifthelscommandsucceeds(meansreadaccess).


Quoting(1)

Double(")quotescanbeusedtopreventtheshellfrominterpretingspacesasargumentseparators,aswellastopreventfilenamepatternexpansion.

>echo"HelloWorld"HelloWorld

>echo"Youareloggedas$USER"Youareloggedasbgates

>echo*.logfind_prince_charming.logcosmetic_buys.log

>echo"*.log"*.log


Quoting(2)

Singlequotesbringasimilarfunctionality,butwhatisbetweenquotesisneversubstituted

>echo'Youareloggedas$USER'Youareloggedas$USER

Backquotes(`)canbeusedtocallacommandwithinanother

>cd/lib/modules/`unamer`;pwd/lib/modules/2.6.91.6_FC2

Backquotescanbeusedwithindoublequotes

>echo"YouareusingLinux`unamer`"YouareusingLinux2.6.91.6_FC2


Measuringelapsedtime

timefind_expensive_housingnear

real0m2.304s(actualelapsedtime)user0m0.449s(CPUtimerunningprogramcode)sys0m0.106s(CPUtimerunningsystemcalls)

real=user+sys+waitingwaiting=I/Owaitingtime+idletime(runningothertasks)


Environmentvariables

Shellslettheuserdefinevariables.Theycanbereusedinshellcommands.Convention:lowercasenames

Youcanalsodefineenvironmentvariables:variablesthatarealsovisiblewithinscriptsorexecutablescalledfromtheshell.Convention:uppercasenames.

envListsalldefinedenvironmentvariablesandtheirvalue.


Shellvariablesexamples

Shellvariables(bash)

projdir=/home/marshall/coolstufflsla$projdir;cd$projdir

Environmentvariables(bash)

cd$HOME

exportDEBUG=1./find_extraterrestrial_life(displaysdebuginformationifDEBUGisset)


Mainstandardenvironmentvariables

LD_LIBRARY_PATH

SharedlibrarysearchpathDISPLAY

ScreenidtodisplayX(graphical)applicationson.EDITOR

Defaulteditor(vi,emacs...)

HOMECurrentuserhomedirectory

HOSTNAMENameofthelocalmachine

MANPATHManualpagesearchpath

PATHCommandsearchpath

PRINTER

Defaultprintername

SHELLCurrentshellname

TERMCurrentterminaltype

USER

Currentusername

Usedbylotsofapplications!


PATHenvironmentvariables

PATHSpecifiestheshellsearchorderforcommands

/home/acox/bin:/usr/local/bin:/usr/kerberos/bin:/usr/bin:/bin:/usr/X11R6/bin:/bin:/usr/bin

LD_LIBRARY_PATHSpecifiesthesharedlibrary(binarycodelibrariessharedbyapplications,liketheClibrary)searchorderforld

/usr/local/lib:/usr/lib:/lib:/usr/X11R6/lib

MANPATHSpecifiesthesearchorderformanualpages

/usr/local/man:/usr/share/man


PATHusagewarning

Itisstronglyrecommendednottohavethe.directoryinyourPATHenvironmentvariable,inparticularnotatthebeginning:

Acrackercouldplaceamaliciouslsfileinyourdirectories.Itwouldgetexecutedwhenyourunlsinthisdirectoryandcoulddonaughtythingstoyourdata.

Ifyouhaveanexecutablefilecalledtestinadirectory,thiswilloverridethedefaulttestprogramandsomescriptswillstopworkingproperly.

Eachtimeyoucdtoanewdirectory,theshellwillwastetimeupdatingitslistofavailablecommands.

Callyourlocalcommandsasfollows:./test


Alias

Shellsletyoudefinecommandaliases:shortcutsforcommandsyouuseveryfrequently.

Examples

aliasls='lsla'Usefultoalwaysruncommandswithdefaultarguments.

aliasrm='rmi'Usefultomakermalwaysaskforconfirmation.

aliasfrd='find_rambaldi_deviceasaprisky'Usefultoreplaceverylongandfrequentcommands.

aliascia='./home/sydney/env/cia.sh'Usefultosetanenvironmentinaquickway(.isashellcommandtoexecutethecontentofashellscript).


Thewhichcommand

Beforeyourunacommand,whichtellsyouwhereitisfound

bash>whichlsaliasls='lscolor=tty'/bin/ls

tcsh>whichlsls:aliasedtolscolor=tty

bash>whichalias/usr/bin/which:noaliasin(/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin)

tcsh>whichaliasalias:shellbuiltincommand.


~/.bashrcfile

~/.bashrcShellscriptreadeachtimeabashshellisstarted

Youcanusethisfiletodefine

Yourdefaultenvironmentvariables(PATH,EDITOR...).

Youraliases.

Yourprompt(seethebashmanualfordetails).

Agreetingmessage.


Commandediting

Youcanusetheleftandrightarrowkeystomovethecursorinthecurrentcommand.

Youcanuse[Ctrl][a]togotothebeginningoftheline,and[Ctrl][e]togototheend.

Youcanusetheupanddownarrowstoselectearliercommands.

Youcanuse[Ctrl][r]tosearchinsidethehistoryofpreviouscommands.


Commandhistory(1)

historyDisplaysthelatestcommandsthatyouranandtheirnumber.Youcancopyandpastecommandstrings.

Youcanrecallthelatestcommand:!!

Youcanrecallacommandbyitsnumber!1003

Youcanrecallthelatestcommandmatchingastartingstring:!cat


Commandhistory(2)

Youcanmakesubstitutionsonthelatestcommand:^more^less

Youcanrunanothercommandwiththesamearguments:more!*



MiscellaneousTexteditors


Texteditors

GraphicaltexteditorsFineformostneeds

nedit

Emacs,Xemacs

Kate,Gedit

TextonlytexteditorsOftenneededforsysadminsandgreatforpowerusers

vi,vim

nano


Thenedittexteditor

http://www.nedit.org/

Besttexteditorfornonvioremacsexperts

Featurehighlights:

Veryeasytextselectionandmoving

Syntaxhighlightingformostlanguagesandformats.Canbetailoredforyourownlogfiles,tohighlightparticularerrorsandwarnings.

Easytocustomizethroughmenus

Notinstalledbydefaultbyalldistributions


neditscreenshot


Emacs/Xemacs

EmacsandXemacsareprettysimilar(uptoyourpreference)

Extremelypowerfultexteditorfeatures

Greatforpowerusers

Lessergonomicthannedit

Nonstandardshortcuts

Muchmorethanatexteditor(games,email,shell,browser).

Somepowercommandshavetobelearnt.


Kateandgedit

Kateisapowerfultexteditordedicatedtoprogrammingactivities,forKDE

http://kate.kde.org

GeditisatexteditorfortheGnomeenvironment

http://projects.gnome.org/gedit/


vi

TextmodetexteditoravailableinallUnixsystems.Createdbeforecomputerswithmiceappeared.

Difficulttolearnforbeginnersusedtographicaltexteditors.

Veryproductiveforpowerusers.

Oftencan'tbereplacedtoeditfilesinsystemadministrationorinEmbeddedSystems,whenyoujusthaveatextconsole.


vimviimproved

viimplementationnowfoundinmostGNU/Linuxhostsystems

Implementslotsoffeaturesavailableinmoderneditors:syntaxhighlighting,commandhistory,help,unlimitedundoandmuchmuchmore.

Coolfeatureexample:candirectlyopencompressedtextfiles.

ComeswithaGTKgraphicalinterface(gvim)

Unfortunately,notfreesoftware(becauseofasmallrestrictioninfreedomtomakechanges)


vibasiccommands

Thoughviisextremelypowerful,itsmain30commandsareeasytolearnandaresufficientfor99%ofeveryone'sneeds!

Youcanalsotakethequicktutorialbyrunningvimtutor.

Getourvimementosheetifyoudidn'tgetitwiththiscourse:http://freeelectrons.com/docs/commandline


GNUnano

http://www.nanoeditor.org/

Anothersmalltextonly,mousefreetexteditor.

AnenhancedPicoclone(nonfreeeditorinPine)

Friendlyandeasiertolearnforbeginnersthankstoonscreencommandsummaries.

Availableinbinarypackagesforseveralplatforms.

Analternativetoviinembeddedsystems.However,notavailableasabusyboxbuiltin.


GNUnanoscreenshot



MiscellaneousCompressingandarchiving


Measuringdiskusage

Caution:differentfromfilesize!

duh(diskusage)h:returnssizeondiskofthegivenfile,inhumanreadableformat:K(kilobytes),M(megabytes)orG(gigabytes),.Withouth,dureturnstherawnumberofdiskblocksusedbythefile(hardtoread).NotethatthehoptiononlyexistsinGNUdu.

dushs:returnsthesumofdiskusageofallthefilesinthegivendirectory.


Measuringdiskspace

dfhReturnsdiskusageandfreespaceforthefilesystemcontainingthegivendirectory.Similarly,thehoptiononlyexistsinGNUdf.

Example:>dfh.FilesystemSizeUsedAvailUse%Mountedon/dev/hda59.2G7.1G1.8G81%/

dfhReturnsdiskspaceinformationforallfilesystemsavailableinthesystem.Whenerrorshappen,usefultolookforfullfilesystems.


Compressinganddecompressing

Veryusefulforshrinkinghugefilesandsavingspace

g[un]zipGNUzipcompressionutility.Creates.gzfiles.Ordinaryperformance(similartoZip).

b[un]zip2Morerecentandeffectivecompressionutility.Creates.bz2files.Usually2025%betterthangzip.

[un]lzmaMuchbettercompressionratiothanbzip2(upto10to20%).Compatiblecommandlineoptions.


Archiving(1)

Usefultobackuporreleaseasetoffileswithin1file

tar:originallytapearchive

Creatinganarchive:tarcvfc:createv:verbose.Usefultofollowarchivingprogress.f:file.Archivecreatedinfile(tapeusedotherwise).

Example:tarcvf/backup/home.tar/homebzip2/backup/home.tar


Archiving(2)

Viewingthecontentsofanarchiveorintegritycheck:tartvft:test

Extractingallthefilesfromanarchive:tarxvf

Extractingjustafewfilesfromanarchive:tarxvfFilesordirectoriesaregivenwithpathsrelativetothearchiverootdirectory.


ExtraoptionsinGNUtar

tar=gtar=GNUtaronGNU/LinuxCancompressanduncompressarchivesonthefly.UsefultoavoidcreatinghugeintermediatefilesMuchsimplertodothanwithtarandbzip2!

joption:[un]compressesontheflywithbzip2

zoption:[un]compressesontheflywithgzip

lzmaoption:[un]compressesontheflywithlzma

Examples(whichonewillyouremember?)gtarjcvfbills_bugs.tar.bz2bills_bugs

tarcvfbills_bugs|bzip2>bills_bugs.tar.bz2


Checkingfileintegrity

Verylowcostsolutiontocheckfileintegrity

md5sumFC3i386disk*.iso>MD5SUMComputesaMD5(MessageDigestAlgorithm5)128bitchecksumofthegivenfiles.Usuallyredirectedtoafile.

Exampleoutput:db8c7254beeb4f6b891d1ed3f689b412FC3i386disc1.iso2c11674cf429fe570445afd9d5ff564eFC3i386disc2.isof88f6ab5947ca41f3cf31db04487279bFC3i386disc3.iso6331c00aa3e8c088cc365eeb7ef230eaFC3i386disc4.iso

md5sumcMD5SUMCheckstheintegrityofthefilesinMD5SUMbycomparingtheiractualMD5checksumwiththeiroriginalone.



MiscellaneousPrinting


Unixprinting

Multiuser,multijob,multiclient,multiprinterInUnix/Linux,printingcommandsdon'treallyprint.Theysendjobstoprintingqueues,possiblyonthelocalmachine,onnetworkprintingserversoronnetworkprinters.

Printerindependentsystem:PrintserversonlyacceptjobsinPostScriptortext.Printerdriversontheservertakecareoftheconversiontoeachprintersownformat.

Robustsystem:Rebootasystem,itwillcontinuetoprintpendingjobs.


Printingcommands

Usefulenvironmentvariable:PRINTERSetsthedefaultprinteronthesystem.Example:exportPRINTER=lp

lpr[P]SendsthegivenfilestothespecifiedprintingqueueThefilesmustbeintextorPostScriptformat.Otherwise,youonlyprintgarbage.

a2ps[P]AnytoPostScriptconvertsmanyformatstoPostScriptandsendtheoutputtothespecifiedqueue.Usefulfeatures:severalpages/sheet,pagenumbering,infoframe...


Printjobcontrol

lpq[P]Listsalltheprintjobsinthegivenordefaultqueue.

lpisnotreadyRankOwnerJobFile(s)TotalSize1stasloane84nsa_windows_backdoors.ps60416bytes2ndamoore85gw_bush_iraq_mistakes.ps65024000bytes

cancel[]Removesthegivenjobnumberfromthedefaultqueue.


UsingPostScriptandPDFfiles

ViewingaPostScriptfile

PostScriptviewersexist,buttheirqualityisprettypoor.

BetterconverttoPDFwithps2pdf:ps2pdfdecss_algorithm.psxpdfdecss_algorithm.pdf&

PrintingaPDFfile

Youdon'tneedtoopenaPDFreader!

BetterconverttoPostScriptwithpdf2ps:pdf2psrambaldi_artifacts_for_dummies.pdflprrambaldi_artifacts_for_dummies.ps



MiscellaneousSynchronizingfiles


Smartdirectorycopywithrsync

rsync(remotesync)hasbeendesignedtokeepinsyncdirectorieson2machineswithalowbandwidthconnection.

Onlycopiesfilesthathavechanged.Fileswiththesamesizearecomparedbychecksums.

Onlytransferstheblocksthatdifferwithinafile!

Cancompressthetransferredblocks

Preservessymboliclinksandfilepermissions:alsoveryusefulforcopiesonthesamemachine.

Canworkthroughssh(secureremoteshell).Veryusefultoupdatethecontentsofawebsite,forexample.


rsyncexamples(1)

rsynca/home/arvin/sd6_agents//home/sydney/misc/

a:archivemode.EquivalenttorlptgoD...easywaytotellyouwantrecursionandwanttopreservealmosteverything.

rsyncPavdelete/home/steve/ideas//home/bill/my_ideas/

P:partial(keeppartiallytransferredfiles)andprogress(showprogressduringtransfer)

delete:deletefilesinthetargetwhichdon'texistinthesource.

Caution:directorynamesshouldendwith/.Otherwise,yougetamy_ideas/ideas/directoryatthedestination.


rsyncexamples(2)

Copyingtoaremotemachine

rsyncPav/home/bill/legal/arguments/\[email protected]:/home/legal/arguments/

Userbillwillbepromptedforapassword.

Copyingfromaremotemachinethroughssh

[email protected]:/prod/beer/\fridge/homer/beer/

Userhomerwillbepromptedforhissshkeypassword.



MiscellaneousComparingfilesanddirectories


Comparingfilesanddirectories

difffile1file2Reportsthedifferencesbetween2files,ornothingifthefilesareidentical.

diffrdir1/dir2/Reportsallthedifferencesbetweenfileswiththesamenameinthe2directories.

Thesedifferencescanbesavedinafileusingtheredirection,andthenlaterreappliedusingthepatchcommand.

Toinvestigatedifferencesindetail,betterusegraphicaltools!


tkdiff

http://tkdiff.sourceforge.net/Usefultooltocomparefilesandmergedifferences


kompare

AnothernicetooltocomparefilesandmergedifferencesPartofthekdesdkpackage(FedoraCore)


gvimdiff

Anothernicetooltoviewdifferencesinfiles

AvailableinmostdistributionswithgvimApparentlynotusingdiff.Noissuewithfileswithbinarysections!



MiscellaneousLookingforfiles


Thefindcommand

Betterexplainedbyafewexamples!

find.name*.pdfListsallthe*.pdffilesinthecurrent(.)directoryorsubdirectories.Youneedthedoublequotestopreventtheshellfromexpandingthe*character.

finddocsname"*.pdf"execxpdf{}';'Findsallthe*.pdffilesinthedocsdirectoryanddisplaysoneaftertheother.

Manymorepossibilitiesavailable!However,theabove2examplescovermostneeds.


Thelocatecommand

Muchfasterregularexpressionsearchalternativetofind

locatekeysListsallthefilesonyoursystemwithkeysintheirname.

locate*.pdfListsallthe*.pdffilesavailableonthewholemachine

locate/home/fridge/*beer*Listsallthe*beer*filesinthegivendirectory(absolutepath)

locateismuchfasterbecauseitindexesallfilesinadedicateddatabase,whichisupdatedonaregularbasis.

findisbettertosearchthroughrecentlycreatedfiles.



MiscellaneousVariouscommands


Gettinginformationaboutusers

whoListsalltheusersloggedonthesystem.

whoamiTellswhatuserIamloggedas.

groupsTellswhichgroupsIbelongto.

groupsTellswhichgroupsbelongsto.

fingerTellsmoredetails(realname,etc)aboutDisabledinsomesystems(securityreasons).


Changingusers

Youdonothavetologouttologonanotheruseraccount!

suhyde(Rare)Changetothehydeaccount,butkeepingtheenvironmentvariablesettingsoftheoriginaluser.

sujekyll(Morefrequent)Logonthejekyllaccount,withexactlythesamesettingsasthisnewuser.

suWhennoargumentisgiven,itmeanstherootuser.


Thewgetcommand

Insteadofdownloadingfilesfromyourbrowser,justcopyandpastetheirURLanddownloadthemwithwget!

wgetmainfeatures

httpandftpsupport

Canresumeinterrupteddownloads

Candownloadentiresitesoratleastcheckforbadlinks

Veryusefulinscriptsorwhennographicsareavailable(systemadministration,embeddedsystems)

Proxysupport(http_proxyandftp_proxyenv.variables)


wgetexamples

wgetc\http://microsoft.com/customers/dogs/winxp4dogs.zipContinuesaninterrupteddownload.

wgetmhttp://lwn.net/Mirrorsasite.

wgetrnphttp://www.xml.com/ldd/chapter/book/Recursivelydownloadsanonlinebookforofflineaccess.np:"noparent".Onlyfollowslinksinthecurrentdirectory.


Misccommands(1)

sleep60Waitsfor60seconds(doesn'tconsumesystemresources).

wcreport.txt(wordcount)438211518302report.txtCountsthenumberoflines,wordsandcharactersinafileorinstandardinput.


Misccommands(2)

bc("basiccalculator?")bcisahandybutfullfeaturedcalculator.Evenincludesaprogramminglanguage!Usetheloptiontohavefloatingpointsupport.

dateReturnsthecurrentdate.Usefulinscriptstorecordwhencommandsstartedorcompleted.


Checksumcommands

Achecksumorhashsumisafixedsizedatumcomputedfromanarbitraryblockofdigitaldataforthepurposeofdetectingaccidentalerrorsthatmayhavebeenintroducedduringitstransmissionsorstorage.http://en.wikipedia.org/wiki/Checksum

TheMD5hashalgorithmisimplementedinthemd5sumcommand$md5sumpatch2.6.24.7.bz20c1c5d6d8cd82e18d62406d2f34d1d38patch2.6.24.7.bz2

TheSHAalgorithmisimplementedintheshaXsum(sha1sum,sha256sum,etc.)

Theintegrityofseveralfilescanbeverifiedagainstafilelistingthechecksumsusingthecoption.


Systemadministration

Seeourpresentationaboutsystemadministrationbasics:

Networksetup

Creatingandmountingfilesystems

Accessingadministrator(root)priviledges

Packagemanagement

Alsoavailableonhttp://freeelectrons.com/docs/commandline



Applicationdevelopment


Compilingsimpleapplications

ThecompilerusedforallLinuxsystemsisGCChttp://gcc.gnu.org

Tocompileasinglefileapplication,developedinC:gccotesttest.c

Willgenerateatestbinary,fromthetest.csourcefile

ForC++:g++otesttest.cc

TheWalloptionenablesmorewarnings

Tocompilesourcesfilestoobjectfilesandlinktheapplication:gccctest1.cgccctest2.cgccotesttest1.otest2.o

gccautomaticallycallsthelinkerld


Usinglibraries(1)

OnanyLinuxsystem,aClibraryisavailableandoffersalargesetofAPIsforapplicationdevelopment.

Seehttp://www.gnu.org/software/libc/manual/

OutsideoftheClibrary,thousandsofotherlibrariesareavailableforgraphicprogramming,multimedia,networking,scientificcomputations,andmoroe.

Mostlibrariesarealreadyavailableaspackagesinyourdistribution,ingeneralintwopackages

libfooisthepackagecontainingthelibraryitself.Thispackageisrequiredtoexecutealreadycompiledapplications,butnotsufficienttobuildnewapplications

libfoodevisthepackagecontainingtheheadersandotherconfigurationsfilesandtoolsneededtobuildnewapplicationsrelyingonlibfoo.


Usinglibraries(2)

Inyoursourcecode,includetheproperheaderfilesofthelibrary

Usually#includeor#include

Theseheadersarepresentin/usr/include/

Refertothedocumentationofthelibraryfordetails,availablein/usr/share/doc//,ontheWeb...orintheheaderfiles!

Tocompileyourapplicationwiththelibrary,theeasiestsolutionistousepkgconfig,whichissupportedbymostlibrariestoday:gccotesttest.c$(pkgconfigcflagslibs)

Bydefault,theapplicationaredynamicallylinkedwiththelibraries

Thelibrariesmustbepresentin/lib/fortheapplicationtowork

Usethelddcommandtoseewhichlibrariesareneededbyanapplication


MakeandMakefiles

Thecompilationprocesscanbeautomatedusingthemaketool.

makereadsafilecalledMakefilefromthecurrentdirectory,andexecutestherulesdescribedinthisfile

Everyruleishasatargetname,acolon,andalistofdependencies,andthelistofcommandstogeneratethetargetfromthedependenciestarget:dep1dep2

command1command2command3

Whensimplyrunningmake,thedefaulttargetthatisgeneratedisall.Atargetisonlyregeneratedifdependencieshavechanged.

Seehttp://www.gnu.org/software/make/manual/


SimpleMakefileexample

CFLAGS=Wall

all:test

test:t1.ot2.ogcco$@$^$(CFLAGS)

clean:$(RM)ftest

install:$(CP)test/usr/bin

Variablescanbedefinedandlaterexpanededwith$(VARNAME)

Thedefaulttargetallsimplydependsonthetesttarget.

Thetesttargetdependsont1.oandt2.o.Oncethesefilesaregenerated,thegcccommandisexecuted.$@isthetargetname$^isthenameofalldependencies.The.ofilesaregeneratedusingimplicitdependencies,knownbymake.

Thesetargetsareexecutedbyrunningmakecleanandmakeinstall


Buildsystems

Makefilesarenice,buttheydon'teasilyalloweasyadaptationtothedifferentbuildenvironmentanddifferentbuildoptions

Moreelaboratedbuildsystemshavebeendeveloped

Autotools(automake,autoconf),basedonMakefilesandshellscripts.Eventhoughtheyareoldandalittlebitdifficulttounderstand,theyarethemostpopularbuildsystemforfreesoftwarepackages.

CMake,anewer,cleanerbuildsystem

SconcsandWaf,otherbuildsystemsbasedonPython

Thetypicalstepstocompileaautotoolsbasedpackageare./configuremakesudomakeinstall


Debugging

TheofficialdebuggerthatcomeswiththeGNUdevelopmenttoolsisgdb.

Seehttp://sourceware.org/gdb/download/onlinedocs/gdb.html

Anapplicationmustbecompiledwiththegoptiontobeproperlydebugged.Thisoptionaddsdebugginginformationtotheapplicationbinarygccotesttest.cg

Theapplicationcanthenberuninsidethegdbdebugger:gdbtest

Orthedebuggercanbeattachedtotheapplicationwhileitisrunning:gdbtestpPID

WherePIDistheprocessIDoftherunningapplication


Usinggdb

gdbisatextbaseddebugger,withacommandlineinterfacelikeashell,providingdedicatedcommands.Someoftheimportantcommandsare:

break(b)tosetabreakpointinthecode.Canbeusedwithafunctionnameoralocationinthesourcecode,oranabsolutememoryaddress.

print(p)toprintthevalueofavariable.Usedwithavariablename,evenifit'sacomplexone(whichinvolvesdereferencingstructures,forexample)

ctocontinuetheexecutionuntilthenextbreakpoint.

next(n)toexecuteonlythenextlineofcode(stepoveranyfunctioncall)andstep(s)toexecuteonlythenextlineofcode(stepintoanyfunctioncall)

backtrace(bt)todisplaythefunctioncallstack


gdbsamplesession

thomas@surf:/tmp$gccotesttest.cgthomas@surf:/tmp$gdbtestGNUgdb6.8debian[...)(gdb)breakfooBreakpoint1at0x80483c7:filetest.c,line5.(gdb)runStartingprogram:/tmp/test2

Breakpoint1,foo(a=2,b=3)attest.c:55 returna+b;(gdb)pa$1=2(gdb)pb$2=3(gdb)cContinuing.foo=5

Programexitednormally.


ddd,thegraphicalgdb

Thereareseveralgraphicalfrontendstogdb.Apopularoneisddd.

Allowstonavigateinthesourcecodewhiledebugging

Allowstosetbreakpoints,inspectandchangevariablevaluedirectlybylookingatthesourcecode

http://www.gnu.org/software/ddd/



VersioncontrolsystemCVS


Versioncontrol(1)

Activitythatconsistsinmaintainingthefullhistoryofprojectfiles

Everyversionofeveryfileisrecorded

Itallowsto:

Createaknowledgedatabaseabouttheprojectcodebase

Revertincaseofmistake

Reviewthemodificationsintroducedbetweentwodifferentversions

Workasateamonasingleproject

Createparalleldevelopmentorreleasebranches

Tagpreviousversionsofaproject


Versioncontrol(2)

Usefulfor:

Sourcecode,ofcourse

Documentation,translationstrings,compilationscripts,configurationfiles,etc.

Binarydatacanbeversioned,butthefunctionalitieswillbelimited.Besttousetextonlyfileformats.

Generatedfilesshouldneverbeversioned.

Versioncontrolisabsolutelynecessarywhenworkingasateamonaproject

Versioncontrolsystemsareoneofthefundamentaltoolsusedforopensourcedevelopment

Versioncontrolisstillveryusefulwhenworkingaloneonaproject.


Toolsandapproaches

Alotofversioncontrolsystemsavailable

Proprietary:Perforce,Synergy,StarTeam,BitKeeper,ClearCase

Freeandopensource:CVS,Subversion,Git,Mercurial,Arch,Monotone

Inthefree/opensolutions,twomainapproaches

Theclassicalone,thecentralizedapproach,implementedbyCVSandSubversion

Anewerone,thedecentralizedordistributedapproach,implementedinGit,Mercurial,ArchorMonotone.


Centralizedapproach

Thecentralizedapproachisbasedonthepresenceofaserverhostingarepository.

Therepositorycontainsthehistoryofallfilesandactsasthereferenceforallparticipantsintheproject.

Typicalclient/serverapproach.

Repository

Client Client


Conflictmanagement(1)

Whentwopersonsareworkingatthesametimeonthesamefile,whenthechangesaresenttotheserver,aconflictcanoccurifthemodificationsarenotcompatible.

A

A A

Linus Richard

A' A''

RepositoryA'A''


Conflictmanagement(2)

Twosolutionstosolvetheconflictproblem

Thelockingapproach:preventingtwodeveloperstoworkonthesamefile.

Themergeapproach:theseconddevelopersendinghischangestotheserverisresponsibleformerginghischangeswiththeotherdeveloperschangesalreadypresentontheserver.

Inthefree/opensourceprojects,thetraditionalmodelisthemergeone.Itavoidstheburdenofhandlingthelockingandscalesbetterwithabiggerteam.

ButCVSandSubversionbothimplementadvisorylocking:adevelopermustvoluntarilylockthefilesheisgoingtoworkwith.


Conflictresolutionbyfusion

A

A

Linus Richard

AA' A''

A''

A''A*

A*

A*


CVS

FreeSoftware

Usedtobethemostwidelyusedfreeversioncontrolsystem.NowreplacedbySubversion.

Stillusedbybigprojects,butalotoflargeprojects(KDE,Apache,Eclipse)havemovedtoSubversion.

Homepage:http://www.nongnu.org/cvs/

Documentation:http://ximbiot.com/cvs/manual/


CVSusage

Themainclientisacommandlineone,implementedinthecvscommand

cvshelpgivesthelistofavailablecommands

add,admin,annotate,checkout,commit,diff,edit,editors,export,history,import,init,log,login,logout,ls,pserver,rannotate,rdiff,release,remove,rlog,rls,rtag,server,status,tag,unedit,update,version,watch,watchers

cvshelpcommandgivesthehelpofcommand.

Graphicalinterfaces

IntegrationinVim,Emacs,Anjuta,DevC++,Eclipse,Kdevelop,etc.

Graphicalclients:Cervisia(forKDE),gcvs(forGnome),CrossVC,etc.

Webclients:ViewVC, cvsweb,etc.


Createtheworkingcopy

Thefirststeptoworkonanexistingprojectistocreateaworkingcopy.Aworkingcopyisacopyonthedevelopermachineofthefulltreeoftheprojectsourcecode.

Thedeveloperwillworkonthiscopyofthesourcecode,asusual(filesaredirectlyaccessibleandeditable).

cvsdrepositoryaddresscheckoutmodule

Createinthecurrentdirectoryaworkingcopyofthemodulemodulelocatedinthegivenrepository

Arepositoryaddresscanbe:

Alocalpath,like:local:/home/user/cvsrepo/

TheaddressofaCVSpserver,like:pserver:user@host:/var/cvsrepo/

UsingSSH,bysettingtheenvironmentvariableCVS_RSHtossh,andusinganaddresslike:ext:user@host:/var/cvsrepo/


Simpleusage

Workingcopycreationtobedoneonceforall!

Modificationsoftheprojectfiles

Filesaremodifieddirectly,asusual,nothingspecialisnecessary.CVSknowsthecontentsoftheoriginalversionofthefilesandisabletocomputethelistofmodificationsmadebythedeveloper.

EverydirectorycontainsaCVSdirectorythatshouldnotbemodified.

Sendthemodificationstotheserver:commit

cvscommit

Runsatexteditortowritethemessagethatdescribesthecommit.Thismessagewillbepreservedforeverinthehistory.Veryimportanttodescribeyourchangeindetails!

Fetchthemodificationsmadebyotherdevelopers:update

cvsupdate


Example

$cvsd:local:/tmp/repo/checkoutprojectcvscheckout:UpdatingprojectUproject/READMEUproject/a.cUproject/b.c$cdproject/project$viREADMEproject$cvscommitm"Addinginfos"cvscommit:Examining./tmp/repo/project/README,v


Filemanagement

Whennewfilesordirectoriesarecreatedinsidetheproject,theyarenotautomaticallytakenintoaccountbyCVS

Theymustbeexplicitlyaddedwiththeaddcommand

cvsaddfile1.c

Thenewfileisnotpropagatedtotherepositoryuntilthenextcommit

Samethingwithdirectories

Filesanddirectoriescanberemovedwiththeremovecommand

IssuesfixedinSubversion

Filesanddirectoriescannotberenamedwithoutlosingthehistory

DirectoryremovalisbadlyhandledinCVS


Example

project$vic.cproject$cvsaddc.ccvsadd:schedulingfile`c.c'foradditioncvsadd:use`cvscommit'toaddthisfilepermanentlyproject$cvscommitm"Newfilec.c"cvscommit:Examining./tmp/repo/project/c.c,v


Stateoftheworkingcopy

Togetinformationsaboutthestateoftheworking,usethestatuscommand

Willtelltheversionofthefile,iftheyhavebeenlocallymodified,etc.

Toseethemodificationsmadetotheprojectandnotcommittedyet,usethediffcommand

Repository

Workingcopydiff

checkout update


Diffcommandexample

project$cvsdiffcvsdiff:Diffing.Index:c.c========================================================RCSfile:/tmp/repo/project/c.c,vretrievingrevision1.1diffur1.1c.cc.c 28Jan200916:48:110000 1.1+++c.c 28Jan200916:50:410000@@1+1,2@@Test.+Test2.


Usingthehistory

Seethecommitmessages

cvslog

cvslogfile.c

Seethedifferencesbetweentwopastrevisions

cvsdiffr1.1r1.2file.c

WithCVS,theversioningisdoneonaperfilebasis:acommitisnotanentitythatcanbereferredtooncecompleted.Itmakestheusageofthehistoryverycomplicated.ThisisfixedinSubversion.

Seewhomadeachange

cvsannotate

Agraphicalinterface,eitheragraphicalclientorawebclient,willbeveryusefultonavigateandusethehistoryefficiently


ConflictmanagementinCVS(1)

Afterchangingafile,wetrytocommitthechangetotheserver$cvscommitm"Modification"cvscommit:Examining.cvscommit:Uptodatecheckfailedfor`main.c'cvs[commitaborted]:correctaboveerrorsfirst!

Thefilehasbeenmodifiedontheserversinceourlastupdate.Sowemustmakemergethemodificationswehavedonewiththemodificationsoftheotherdevelopers,byupdatingourworkingcopy.$cvsupdatecvsupdate:Updating.RCSfile:/tmp/repo/project/main.c,vretrievingrevision1.1retrievingrevision1.2Mergingdifferencesbetween1.1and1.2intomain.crcsmerge:warning:conflictsduringmergecvsupdate:conflictsfoundinmain.cCmain.c


ConflictmanagementinCVS(2)

Insidemain.c

#include

intmain(void){1.2return0;}

Ourmodification!

Themodificationmadebytheotherdeveloper


ConflictresolutioninCVS

Theconflictmustbemanuallyresolvedbythedeveloper,noautomatedtoolcanresolvesuchconflicts

Theconflictisresolveddirectlyinthefilebyremovingthemarkersandmergingthemodifications(eitherselectingoneofthemorcreatinganewversionbasedonbothmodifications)

Onceresolved,thefilecanbecommittedasusual

project$cvscommitm"InFrench"cvscommit:Examining./tmp/repo/project/main.c,v


Repositoryinitialization

Arepositorymustbeinitializedusingtheinitcommand:

mkdir/home/user/cvsrepo

cvsd:local:/home/user/cvsrepoinit

Onceinitialized,itcanbeaccessedlocallyorremotelythroughSSH.Remoteaccessthroughpserverwillrequireadditionalconfiguration,seetheCVSdocumentation.

Ifalreadyexistingprojectshavetobeimportedintherepository,usetheimportcommand

cdproject

cvsd:local:/home/user/cvsrepoimportmodulenamevendortagreleasetag

vendortag,symbolicnameforthebranch

releasetag,symbolicnamefortherelease


Tagsandbranches

Tagsallowtoidentifyagivenversionoftheprojectthroughameaningfulsymbolicname

Usefulforexampleiftheversionhasbeendeliveredtothetestteamortoacustomer.

Tagsarecreatedusingthecvstagcommand.

Branchesallowtocreateparallelflowofdevelopments

Maintenanceofapreviousrelease

Developmentofexperimentalnewfeatures

Createdusingtheboptionofthecvstagcommand

Branchingandmergingisrelativelycomplicated,andfallsoutsidethescopeofthistraining.


Connectionwithothertools

Versioncontrol

Mail

BugtrackingBuildandtest

IRC

Sendamailateverycommit,withthediff.Allowotherstoreview,andnotifiesthemofthechanges..

Sendamessageateverycommit.Notifiesofthechanges.

Ateverycommit,abuildandtestmachinbuildsandteststheproject.

Linkthecommitswiththebugsandthebugswiththecommits.


WhySubversion?

BecauseCVShasmanydrawbacksandshortcoming

Notpossibletosimplyrenamefilesanddirectorieswhilepreservingthehistory

Noatomiccommitsidentifyingthecommitasawhole,makingitdifficulttonavigateinthehistory,revertcommits,andisthesourceofrepositoryincoherencyincaseofcrashes

Poorbranchingandmergingcapabilities

Inusage,SubversionisverysimilartoCVS:thecommandsareexactlythesame.

FixesallCVSshortcomings

Andprovidesinterestingbranchingandmergingfeaturessuchasmergetracking.

Manylargescaleprojectsalreadymadetheswitch.


Distributedversioncontrolsystems

Sinceafewyears,anewgenerationofversioncontrolsystems

Distributedversioncontrolinsteadofacentralizedapproach

Principles

Notechnicallycentralrepository,everycopyisarepository

Alldeveloperscancreatelocalbranches,sharethesebrancheswithotherdeveloperswithoutaskingacentralauthority.

Advancedbranchingandmergingcapabilities.

Moreandmorecommonlyusedinfreesoftwareprojects(Linuxkernel,X.org,etc.)

Mostcommonlyusedtools:Git,Mercurial

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Relateddocuments

Allourtechnicalpresentationsonhttp://freeelectrons.com/docs

LinuxkernelDevicedriversArchitecturespecificsEmbeddedLinuxsystemdevelopment

FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com

Howtohelp

Youcanhelpustoimproveandmaintainthisdocument...

Bysendingcorrections,suggestions,contributionsandtranslations

Byaskingyourorganizationtoorderdevelopment,consultingandtrainingservicesperformedbytheauthorsofthesedocuments(seehttp://freeelectrons.com/).

Bysharingthisdocumentwithyourfriends,colleaguesandwiththelocalFreeSoftwarecommunity.

Byaddinglinksonyourwebsitetoouronlinematerials,toincreasetheirvisibilityinsearchengineresults.

CustomDevelopment

SystemintegrationEmbeddedLinuxdemosandprototypesSystemoptimizationApplicationandinterfacedevelopment

FreeElectronsOurservices

EmbeddedLinuxTraining

Allmaterialsreleasedwithafreelicense!

UnixandGNU/LinuxbasicsLinuxkernelanddriversdevelopmentRealtimeLinux,uClinuxDevelopmentandprofilingtoolsLightweighttoolsforembeddedsystemsRootfilesystemcreationAudioandmultimediaSystemoptimization

Consultingandtechnicalsupport

HelpindecisionmakingSystemarchitectureSystemdesignandperformancereviewDevelopmenttoolandapplicationsupportInvestigatingissuesandfixingtoolbugs

Linuxkernel

LinuxdevicedriversBoardsupportcodeMainstreamingkernelcodeKerneldebugging