1.11.2

1.2.11.2.21.3

1.3.11.4

1.4.11.4.21.4.31.5

1.5.11.5.21.5.31.5.41.6

1.6.11.6.21.6.31.7

1.7.11.7.21.7.31.7.41.7.51.8

1.8.11.8.21.8.31.9

1.9.11.9.21.9.31.9.4

TableofContents

IntroductionAboutThisDocument

CourseOverviewCourseResources

FME2018OverviewNewFeaturesin2018

FeatureCachingCachingFeaturesPartialRunsExercise:EmergencyResponseProject

CollapsibleBookmarksCollapsingBookmarksCollapsibleBookmarkBestPracticeCollapsibleBookmarksandCachesExercise:EmergencyReponseProject

ToleranceandPrecisionToleranceParametersToleranceExamplesExercise:ToleranceandDataCleaning

MicrosoftFormatsFormatsandTransformersMicrosoftWordMicrosoftPowerPointMicrosoftExcelExercise:VancouverCommunityMappingProject

FeatureJoinerFeatureJoinerModesMultipleJoinMatchesExercise:CrimeAnalysisProject

CourseWrap-UpProductInformationandResourcesCommunityInformationandResourcesFeedbackandCertificatesThankYou

2

UpgradingtoFMEDesktop2018TrainingCourse

Thisisatrainingmanualforusersupgradingtothe2018versionofSafeSoftware’sFMEDesktopapplication.

Thetrainingwillintroducenewconceptsandterminology,andhelpstudentsbecomefamiliarwithmethodologythathaschangedsincethepreviousrelease.

CourseStructure

Thecourseismadeupofvarioussections,including:

Ageneraloverviewof2018AuthoringworkflowswiththePartialRuntoolsCollapsiblebookmarksandrelatedfunctionalityPrecisionandtoleranceinspatialtransformationWritingandstylingMicrosoftdataformatsUsingthenewFeatureJoinertransformer

CurrentStatus

Thecurrentstatusofthismanualis:COMPLETE:thismanualcanbeusedfortraining.

NB:Evenforcompletedcontent,SafeSoftwareInc.assumesnoresponsibilityforanyerrorsinthisdocumentortheirconsequences,andreservestherighttomakeimprovementsandchangestothisdocumentwithoutnotice.Seethefulllicensingagreementforfurtherdetails.

Introduction

3

AboutThisDocument

Thismanualisatrainingcourseforusersupgradingtothe2018versionofSafeSoftware’sFMEDesktopapplication.

LookoutforresidentsoftheCityofInteropolis,whowillappearfromtime-to-timetogiveyouadviceanddispenseFME-relatedwisdom.Infact,herecomessomeonenow:

Mr.E.Dict(AttorneyofFMELaw)says...OnbehalfoftheCityofInteropolis,welcometothistrainingcourse.Hereisthestandardlegalinformationaboutthistrainingdocumentandthedatasetsusedduringthecourse.

Besuretoreadit,particularlyifyou'rethinkingaboutre-usingormodifyingthiscontent.

LicensingandWarranty

Permissionisherebygrantedtouse,modifyanddistributetheFMETutorialsandrelateddataanddocumentation(collectively,the“Tutorials”),subjecttothefollowingrestrictions:

1. TheoriginoftheTutorialsandanyassociatedFME®softwaremustnotbemisrepresented.

2. RedistributionsinoriginalormodifiedformmustincludeSafeSoftware’scopyrightnoticeandanyapplicableDataSource(s)notices.

3. YoumaynotsuggestthatanymodifiedversionoftheTutorialsisendorsedorapprovedbySafeSoftwareInc.

4. Redistributionsinoriginalormodifiedformmustincludeadisclaimersimilartothatbelowwhich:(a)statesthattheTutorialsareprovided“as-is”;(b)disclaimsanywarranties;and(c)waivesanyliabilityclaims.

SafeSoftwareInc.makesnowarrantyeitherexpressedorimplied,including,butnotlimitedto,anyimpliedwarrantiesofmerchantability,non-infringement,orfitnessforaparticularpurposeregardingtheseTutorials,andmakessuchTutorialsavailablesolelyonan“as-is”basis.InnoeventshallSafeSoftwareInc.beliabletoanyonefordirect,indirect,special,collateral,incidental,orconsequentialdamagesinconnectionwithorarisingoutoftheuse,modificationordistributionoftheseTutorials.

Thismanualdescribesthefunctionalityanduseofthesoftwareatthetimeofpublication.Thesoftwaredescribedherein,andthedescriptionsthemselves,aresubjecttochangewithoutnotice.

DataSources

CityofVancouver

Unlessotherwisestated,thedatausedhereoriginatesfromopendatamadeavailablebytheCityofVancouver,BritishColumbia.ItcontainsinformationlicensedundertheOpenGovernmentLicense-Vancouver.

AboutThisDocument

4

Others

ForwardSortationAreas:StatisticsCanada,2011CensusDigitalBoundaryFiles,2013.Reproducedanddistributedonan"asis"basiswiththepermissionofStatisticsCanada.©ThisdataincludesinformationcopiedwithpermissionfromCanadaPostCorporation.

DigitalElevationModel:GeoBase

FireHallData:Someattributedataadaptedfromcontent©2013byWikipedia,usedunderaCreativeCommonsAttribution-ShareAlikelicense

StanleyParkGPSTrail:UsedwithkindpermissionofVancouverTrails.com.

OpenStreetMapDatasets:©OpenStreetMapcontributors.

ContainsinformationlicensedundertheOpenGovernmentLicence–BritishColumbia

Copyright

©2005–2018SafeSoftwareInc.Allrightsarereserved.

Revisions

Everyefforthasbeenmadetoensuretheaccuracyofthisdocument.SafeSoftwareInc.regretsanyerrorsandomissionsthatmayoccurandwouldappreciatebeinginformedofanyerrorsfound.SafeSoftwareInc.willcorrectanysucherrorsandomissionsinasubsequentversion,asfeasible.Pleasecontactusat:

SafeSoftwareInc.Phone:604-501-9985Email:train@safe.comWeb:safe.comGitHub:github.com/safesoftware/FMETraining

SafeSoftwareInc.assumesnoresponsibilityforanyerrorsinthisdocumentortheirconsequences,andreservestherighttomakeimprovementsandchangestothisdocumentwithoutnotice.

Trademarks

FME isaregisteredtrademarkofSafeSoftwareInc.Allbrandorproductnamesaretrademarksorregisteredtrademarksoftheirrespectivecompaniesororganizations.

DocumentInformation

DocumentName:UpgradingtoFMEDesktop2018.0TrainingCourse

®

®

AboutThisDocument

5

CourseOverview

Thetrainingwillintroducenewconceptsandterminology,andhelpstudentsbecomefamiliarwithmethodologythathaschangedsincethepreviousrelease.

CourseStructure

Thecourseismadeupofvarioussections,including:

Ageneraloverviewof2018AuthoringworkflowswiththePartialRuntoolsCollapsiblebookmarksandrelatedfunctionalityPrecisionandtoleranceinspatialtransformationWritingandstylingMicrosoftdataformatsUsingthenewFeatureJoinertransformer

AbouttheManual

TheUpgradingtoFMEDesktop2018TrainingmanualnotonlyformsthebasisforthistrainingcoursebutisalsousefulreferencematerialforfutureworkyoumayundertakewithFME.

AllscreenshotsinthesematerialsweretakenusingFMEonWindowsServer2016.Thefontsused(especiallyinscreenshotsofthelogwindow)mayberesizedorotherwisechangedforimprovedlegibility.

CourseOverview

6

CourseResources

Anumberofsampledatasetsandworkspaceswillbeusedinthiscourse.

OnYourTrainingComputer

ThedatausedinthistrainingcourseisbasedonopendatafromtheCityofVancouver,Canada.

MostexercisesaskyoutoassumetheroleofacityplanneratthefictionalcityofInteropolisandtosolveaparticularproblemusingthisdata.

Whetherit'salocalcomputeroravirtualcomputerhostedinthecloud,you'llfindresourcesfortheexamplesandexercisesinthemanualatthefollowinglocations:

Location Resource

C:\FMEData2018\Data DatasetsusedbytheCityofInteropolis

C:\FMEData2018\Resources\DesktopBasic Otherresourcesusedinthetraining

C:\FMEData2018\Workspaces\DesktopBasic Workspacesusedinthestudentexercises

C:\FMEData2018\Output Thelocationinwhichtowriteexerciseoutput

<documents>\FME\Workspaces ThedefaultlocationtosaveFMEworkspaces

YoushouldalsofindFMEpre-installed,plusadigitalcopyofthismanual.

Pleasealertyourinstructorifanyitemismissingfromyoursetup.

YoucanfindthelatestversionofFMEDesktopandFMEServerforWindows,Mac,andLinux-togetherwiththelatestBetaversions-ontheSafeSoftwarewebsite.

CourseEtiquette

Foronlinecourses,pleaseconsiderotherstudentsandtestyourvirtualmachineconnectionbeforethecoursestarts.Theinstructorcannothelpdebugconnectionproblemsduringthecourse!

Forlivecourses,pleaserespectotherstudents’needsbykeepingnoisetoaminimumwhenusingamobilephoneorcheckinge-mail.

CourseResources

7

2018Overview

FME2018Overview

8

NewFeaturesin2018

FMEDesktop2018introducesmanynewtoolsandfunctionalitytobothFMEWorkbenchandtheFMEDataInspector.Thesetoolsmakewide-rangingchangestothemethodsworkspaceauthorsuseindesigningandcreatingworkspaces.

TopTools

Themostimportantnewtoolswillbecoveredduringthiscourse.Theyare:

CachesandPartialRunsTheabilityforFMEtocachedataateachpointofatranslation,andtore-usethatcacheddata.

CollapsibleBookmarksTheabilityforbookmarkstobecollapsedtosimpleinput/outputports,tohidethecontents.

PrecisionandToleranceParametersTheabilitytosettoleranceonanumberofspatialoperations,toworkaroundcoordinateprecisiondifferences.

MicrosoftFormatsTheabilitytowritedatatoMicrosoftPowerPointandMicrosoftWord,andtheadditionofstylingtransformersforallMicrosoftformats

FeatureJoinerASQL-basedtransformerforcarryingoutjoinsinamorepowerfulversionoftheFeatureMerger

OtherUpdates

Besidesthesetoptools,itisalsoworthmentioningotherhighlightsfromtheFME2018release:

NewFormats

TherearemultiplenewformatssupportedinFME2018,withover50newrasterformatsbasedonGDALtechnology.Someofthekeyformatsare:

AppleVenueFormat(AVF)ReaderandWriterBentleyi-ModelReaderEsriIndexed3DSceneLayer(I3S)LDAP(ActiveDirectory)ReaderPCD(PointCloudData)ReaderandWriterPDFReader(2D/Raster/Text)QlikDataExchangeWriterWellLogASCIIStandard(WLAS)Reader

ThereisalsoanFMEARformatandassociatedappontheiTunesstore.

NewTransformers

AsidefromMicrosoft-relatedfunctionality,newtransformersin2018mostlyrevolvearoundconnectingtowebservices:

HDFSConnector(Hadoop)ProjectWiseConnectorS3ConnectorSalesforceConnector

WorkbenchUpdates

FMEWorkbenchincludesanumberofupgrades-somesmall,somelarge-separatetowhatthiscoursecovers.Theseinclude:

Copy/pastetransformerparametersDynamiclogwindowfilteringFeatureWriteroutputportsNewgeometrytypesupport(clothoidsand3Darcs)SupportfornewGermanandAustraliancoordinatesystemdatumsTesterDateTimesupport

NewFeaturesin2018

9

ChangeRejectedFeaturehandlingwitharight-click

Annotationwordwrapsbydefault.BookmarksfilledbydefaultNewWindowsrenderingoptionsforhighDPIdisplaysandwebviews

PlacenewannotationdirecttothecanvasusingCtrl+K

FurtherReading

FormoreinformationonwhatelseisnewandgreatinFME2018,seetheFME2018unveilingwebinar,theFMEDesktopDeepDivewebinar,andtheFMEServerDeepDivewebinar

NewFeaturesin2018

10

NewFeaturesin2018

11

FeatureCaching

FeatureCaching

12

FeatureCaching

InpreviousversionsofFME,itwaspossibletorunaworkspaceusingRunwithFullInspection.

FullInspectionmeantthedatawassavedateachstepoftheworkspaceandsowasavailableforimmediateinspection.

InFME2018thefunctionalityaroundthistoolchanged,andsothenamewaschangedtoo,toRunwithFeatureCaching:

Withthisoptionactive,FMEgeneratescachesateverystepofthetranslationwhentheworkspaceisrun:

Thecachesareindicatedbythesmalliconsoneachobject.

InspectingCachedData

Sometimesit'simportanttobeabletoinspectdataatanystepofthetranslation.AddinganInspectortransformerateverystepwouldbetiresome,soinsteadtheDataCachingoptioncanbeused.

Cacheddatacanbeinspectedbysimplyclickingontheicononaparticularobject.

SavingCachedData

Becauseworkspacescanbeworkedonoverseveralsessions,andbecauseyoumightwanttotransferaworkspacewithcachestoanotheruser,savingaworkspaceasatemplatealsogivestheoptiontosavethedatacaches:

CachingFeatures

13

Thatwayanyworkspacecreatedfromthetemplateautomaticallygainsaccesstothecachesfromthepreviousrun.

TIPIt'scertainlyquickertosetup"RunwithDataCaching"thantomanuallyaddInspectortransformers.However,beawarethatcachingdataobviouslycausesthetranslationtobeslower,andtousesystemresourcessuchasdiskspace.

Datacachingisveryusefulwhiledevelopingaworkspace,butshouldbeturnedoffbeforeputtingaworkspaceintoproduction.

CachingFeatures

14

CachedDataStatus

Whenthecacheonatransformerorfeaturetypeisgreen,itmeansthatthecacheisup-to-date;i.e.thecacherepresentsthedatathatwouldbeproducedweretheworkspacetoberunatthatpoint.

Heretheworkspacecreatesadatetimeattributeandtheworkspaceisruntofillthecaches:

However,whentheworkspaceisedited,cachescanchangecolortoyellow.Thismeansthecacheisnolongerup-to-date;i.e.thecachenolongerrepresentsthedatathatwouldbeproducedweretheworkspacetoberun.

HeretheDateTimeStamperhasbeenchangedtocreateadateonlyattribute:

Allcachesafter(downstreamof)theDateTimeStamperturntoyellowtoindicatetheynolongerrepresenttheworkspace.Butcachesbefore(upstreamof)theDateTimeStamperremaingreen,becausetheyarenolongeraffected.

ReusingCachedFeatures

ThemainreasonthefunctionalitychangednamefromFullInspectiontoFeatureCaching,isthatthecacheddatacannowbeusedforpurposesotherthaninspection.

Whenitistimetorunaworkspace,datacanbeautomaticallyretrievedfromgreencachesanduseddownstreamofthatpoint.Forexample,intheaboveworkspacedatadoesnotneedtobereadfromthesourcedatasetbecauseitisalreadyavailableinanup-to-datecache.

Thisisachievedbyclickingonthetransformeryouwishtorun,andusingoneofthetwopop-upactions:

PartialRuns

15

ThetwooptionsareRuntoThisandRunfromThis.

RunToThisrunstheworkspaceupto,andincluding,thechosentransformer.It'sperfectforwhenthattransformerhasbeenedited,andyouwanttoseewhattheresultthattransformeraloneis.

RunFromThis(shortcutF6)runstheworkspacefromthechosentransformeronwards.It'sperfectforwhenthattransformerhasbeenedited,andyouwanttoseewhatthefinalworkspaceresultswillbe.

Hoveringoveroneoftheseoptionscausesthesectionofworkspacetoberuntohighlightingreen:

HeretheAttributeFiltertransformerisbeingrun,thedatabeingfetchedfromthepriorcache.However,becausethepriorcacheisyellow,andout-of-date,FMEwillinformtheuserthatitmustgobackfurthertoobtainavalidcacheandruntheextratransformersinbetween:

PartialRuns

16

Iftheauthorwishestore-runtheentireworkspace,theycandothatusingtheusualrunbutton(orF5),whichwillre-runtheworkspaceand(assumingtheFeatureCachingoptionisturnedon)refillallofthecaches.

TIPAJunctiontransformercanplayausefulpartinPartialRuns.Forexample,heretheworkspaceissplitintotwostreams.IfIwishtorunbothstreamssimultaneously,thentheonlycommandwouldbeRunFromThisontheSpatialSorter.

ButthatmeanstheSpatialSorterwillalsoberun.IfIwanttorunbothstreamsbutnottheSpatialSorter,thenaJunctiontransformerintheappropriatelocationwillallowmetodothat.IcanRunFromThisontheJunctiontransformeritself,whichhaslittleoverhead.

WhyReuseCachedData?

Theobviousreasonforusingcacheddataisoneofperformance.Iftheworkspacecanusedatathatwascachedinaprevioustransformer,thenthattransformationdoesnotneedtooccuragain.Whenbuildingaworkspaceonetransformeratatime,theworkspacecanbetestedateachstepandonlythecurrenttransformerneedstobeexecuted.

Similarlythiscanpreventexcessreadingofdata.Ifdataisbeingreadfromadatabaseonanetwork,forexample,thenacacheofdatapreventsFMEfromhavingtomakesubsequentcallstothatdatabase.Itsavesonbothnetworktrafficandquerytime.

Additionally,thisisagreattooltouseforcachingtheresultsofwebservices.Forexample,recordsbeingsenttoawebserviceforgeocodingmightincuranactualmonetarycostforthatservice.ButnowFMEisabletocachethatdataandreuseit,avoidingtheneedtorepeatedlyusethatwebservice.

MsAnalystsays...Obviouslydatacachingtakesnoaccountofwhatmighthavechangedinthesourcedataset.Itonlyknowswhathasbeenchangedintheworkspace.It'suptotheworkspaceauthortobeawareofifthesourcedatasethaschangedandthereforeneedstoberefreshedwithanewrun.

PartialRuns

17

PartialRuns

18

Exercise CachingandPartialRuns

Data Roads(AutoCADDWG)FirehallsandZones(GML)

OverallGoal CreatetimeanddistanceisolineswiththeNetworkCostCalculator

Demonstrates Datacachingandpartialruns

StartWorkspace None

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Caching-Ex1-Complete.fmwt

Toassistinplanningemergencyservicesforthecity,youhavebeentaskedwithassessingemergencyresponsetimesbycreatingadatasetofisochrones-contourlinesdepictingtraveltimefromeachfirehall-formedicalemergencies.

1)AddFirehallReaderStartFMEWorkbench.SelectReaders>AddReaderfromthemenubar.Settheparametersasfollows:

ReaderFormat GML(GeographyMarkupLanguage)

ReaderDataset C:\FMEData2018\Data\Emergency\FireHallsWithZones.gml

Whenprompted,selecttheFireHallsandMedicZonesfeaturetypestoaddtotheworkspace:

2)AddRoadsReaderOncemoreselectReaders>AddReaderfromthemenubar.Thistimesettheparametersasfollows:

ReaderFormat AutodeskAutoCADDWG/DXF

ReaderDataset C:\FMEData2018\Data\Transportation\CompleteRoads.dwg

Whenprompted,selectallofthefeaturetypestoaddtotheworkspace(i.e.wewantallroadtypes)leavingoutonlylayer"0".

3)RunWorkspaceEnsurefeaturecachingisturnedon(Run>RunwithFeatureCaching)andruntheworkspace.Thedatawillbereadandcached.Thismeans:

Ourpre-translationinspectioncanbedonehereWedon'thavetoreaditseparatelyintheDataInspector

ThedataisnowreadyforusebylaterstagesoftheworkspaceWhenwerunthewholeworkspace,FMEwon'thavetore-readthedata

Selectallofthefeaturetypesintheworkspace,right-clickoneofthem,andchooseInspectCachedFeatures(shortcutCtrl+I):

Exercise:EmergencyResponseProject

19

Nowwecaninspectthedatatoseewhatwearedealingwith.

4)AddTesterForthisexercisewe'llworkonasinglefirehall,FireHallnumber4.SoaddaTestertransformerconnectedtotheFireHallsfeaturetype.Setituptofilteroutotherfirehalls.Thesimplestwayistocheckfor(andkeep)featureswhoseNameattributeendsinthenumber4:

ClickontheTesterandpresstheRunFromThisbutton(shortcutF6).TheTestertransformerwillrunusingthecacheofdatafromthefeaturetypetowhichitisconnected.

ClickonthegreencacheiconforthePassedoutputport.TheDataInspectorwillopenandshowasinglefirehallfeature.Itshouldbehallnumber4.

5)AddNetworkCostCalculatorNowwe'lladdaNetworkCostCalculatortransformer.ThebestwayofdoingthisistoselectalloftheroadfeaturetypesandthenuseQuickAddbytypingNCCandselectingthattransformer.Thiswaythetransformerwillbeaddedandallroadfeaturetypesconnectedautomatically.

ThenconnectTester:PassedtotheSourceinputport:

Exercise:EmergencyResponseProject

20

Rearrangetheobjectsasnecessarysoyouavoidoverlappingconnections(ormovetheinputportsup/downinstead).

TIPNoticethateventhoughwehaven'truntheNetworkCostCalculatoryet,wehavefeaturecountsontheconnectionsintoit.They'recoloredgreytoshowthatthishasn'thappened,butthat'swhatthecountswouldbe.

6)SetNetworkCostCalculatorParametersInspecttheNetworkCostCalculatorparameters.SetOutputOptimalCostAsto"Z-Values".SetSnapSourcePointsto"Yes"andsetaSnappingToleranceof100:

TheseparameterswilloutputthecostsasZvaluesoneachnetworkline,andwillsnapthesourcefirehalltothenetworkifitdoesn'tsitexactlyuponit.

RuntheNetworkCostCalculatorbyclickingitandchoosingRunToThis.

Exercise:EmergencyResponseProject

21

TIPBecausewecurrentlyhaveallthedatacached,andtherearenotransformersafterthis,itdoesn'tyetmatterifwechooseRunFromThisorRunToThis.Bothachievethesameresultwiththesameperformance.

InspecttheConnectedoutputcache.Ifyouqueryaroadfeatureyou'llseethatithasZvaluesoneachvertex.TheZvaluerepresentstheroaddistancetothefirehall.

7)AddContourGeneratorTovisualizetheseresults,addaContourGeneratortransformerconnectedtotheNetworkCostCalculator:Connectedoutputport.TheContourGeneratorporttoconnectisPoints/Lines:

OpentheContourGeneratorparametersdialog.SettheSurfaceToleranceto10andtheOutputContourIntervalto50.RuntheContourGeneratortransformerandinspecttheContoursoutputcache:

TheoutputisasetofIsodistancelinesshowingthedistance(byroad)fromthechosenfirehall.

8)AddExpressionEvaluatorTocreateIsochrones(linesofequaltraveltime)weneedtospecifythespeedatwhichavehiclecantravel.Obviouslythisvariesdependingonthetypeofroad,terrain,weather,andintersectioncrossings;butaquicksearchshowsthattheaveragespeedofafiretruckinacityis55kph(35mph).

So,addanExpressionEvaluatortransformertotheworkspace,betweentheAutoCADRoadsfeaturetypesandtheNetworkCostCalculatortransformerLineport.ThequickestwaytodothisistoclickonthegreenLinearrowandtypethenameoftheExpressionEvaluator.Thatwaythetransformerwillbeinsertedinalloftheconnectionssimultaneously:

Exercise:EmergencyResponseProject

22

Noticehowthecaches"downstream"fromthenewtransformerchangetoyellow.Theyarestalebecausetheynolongerrepresenttheresultstheworkspacewouldgenerate.

9)SetUpExpressionEvaluatorInspecttheparametersfortheExpressionEvaluator.SetupanexpressiontocalculateanattributecalledTravelTime.Theexpressionshouldbe:

@Length()/916.666666

@Length()isanFMEfunctionthatmeasuresthelengthofeachroadlink.

916.66666comesfrom(speed(metres-per-km/seconds-per-minute))whichis(55(1000/60))

10)AdjustNetworkCostCalculatorNowinspecttheparametersfortheNetworkCostCalculatoragain.

ChangetheWeightTypeparameterfrom"ByLength"to"ByTwoAttributes".SelectTravelTimeastheattributeforboththeforwardandreverseweight:

11)AdjustContourGeneratorNowinspecttheparametersfortheContourGeneratoragain.

ChangetheSurfaceToleranceto1andtheOutputContourIntervalto1(i.e.eachcontourwillrepresent1minuteoftraveltime).

Nowwemustre-runtheworkspace.Youcanchoosetoeither:

RunFromThisontheExpressionEvaluator

Exercise:EmergencyResponseProject

23

RunToThisontheContourGeneratorRunFromThisontheContourGenerator

Thesamesectionsofworkspacewillhighlightasyouhoveroverthefirsttwooptions.ThethirdoptionwillonlyhighlighttheContourGeneratoritself,andwhenchosenwillwarnthatpriortransformersneedtobeincludedastheydon'tyethavevalidcaches.

InspectthenewContourGeneratoroutputcache:

Theresultwillbeaseriesofisochronesrepresentingtraveltimesforemergencyvehiclesfromfirehallnumber4.

12)SaveasTemplateWe'llcontinuethisexerciseattheendofthenextchapter.Incaseyoucloseandre-opentheworkspaceinthemeantime,we'llsaveitasatemplate.Thatwaywecankeepthecaches.

SoselectSave>SaveAsTemplatefromthemenubar.Whenpromptedenteratemplatename.BesuretochecktheoptiontoIncludeFeatureCaches:

Nowwhenwere-opentheworkspaceusingthistemplate,thedatawillalreadybecachedandreadytouse.

Exercise:EmergencyResponseProject

24

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

TurnonFeatureCachingInspectallcacheddatainaworkspaceCarryoutapartialrunusingRunFromHereandRunToHereDecidewhenRunToHereorRunFromHereismostappropriateSaveatemplatefilewithFeatureCachesincluded

Exercise:EmergencyResponseProject

25

CollapsibleBookmarks

CollapsibleBookmarks

26

Bookmarks

Asweknow,abookmark,likeitsreal-worldnamesake,isameansofputtingamarkerdownforeasyaccess.

WithFMEthebookmarkcoversanareaofworkspacethatisusuallycarryingoutaspecifictask,soausercanpickitoutofalargersetoftransformersandmovetoitwithrelativeease.

Asoneuserhasputit,bookmarksarelikeparagraphsforyourworkspace!

Theaboveworkspaceillustratesnicelyhowtomarkupdifferentsectionsofaworkspaceusingbookmarks.Asyoucansee,it'spermittedtosubdividebookmarksfurtherbynestingonebookmarkinsideanother.

ButinFME2018thereisfunctionalitytotakethisonestepfurther...

CollapsingBookmarks

InFME2018,eachbookmarkhasasmalliconinthetop-leftcornerthatallowsittobecollapsed:

Collapsingabookmarkmeansitiscompresseddowntothesizeofasingletransformer,displayingnoneofthecontentsexceptforwheredataentersorexitsthebookmark:

CollapsingBookmarks

27

Clickingtheiconasecondtimere-opensthebookmarktoitsprevioussize.

Thisfunctionalityallowslargesectionsofworkspacetoberenderedinamuchsmallerarea,andonlyopenedupwheneditingisrequired.

Forexamplethesectionofworkspacedisplayedabovemightbereducedtothis:

Thisisobviouslyahugesavingofspaceonalargecanvas.

CollapsingBookmarksvsCustomTransformers

It'seasytoseecollapsingbookmarksasareplacementforcustomtransformers,especiallywherethecustomtransformerwasintendedonlyasameanstosavespaceoncanvas:

However,collapsiblebookmarksarenotsuchagoodreplacementwherethecontentisintendedtobereusedinmultiplelocations.That'sbecauseeachcustomtransformerinheritsitsbehaviourfromamastercustomtransformerdefinition.Editingthedefinitionupdatesallthecustomtransformers.

Thesameisnottrueofacollapsedbookmark.Thereisnomasterdefinitionandeditingthecontentsofabookmarkdoesnotautomaticallyupdatethecontentsinanyotherbookmarks,evenifitwasadirectcopy.

Additionally,acollapsedbookmarkcannotbesharedwithotherusersanddoesn'thavetheabilitytosetpublishedparameters,inthesamewaythatacustomtransformerwould.

Sowhileacollapsedbookmarkcanreplacesomeaspectsofacustomtransformer,itcan'tfullyreplacethem.

CollapsingBookmarks

28

CollapsedBookmarkStyle

Withnewcollapsiblebookmarks,thereareaspectsofBestPracticeandstylethatneedtobeconsidered.

RenamingPorts

Theinputandoutputportsoncollapsedbookmarkscanberenamedtohelpclarifythedataenteringandexiting:

It'snotobviousthatthiscapabilityexists,butinfactitcanbeaccessedinthreedifferentways!

Double-clickingtheobjectSelectingtheobjectandpressingF2UsingtheRenameoptiononthecontextmenu(aspicturedabove)

Theobviousbenefithereisoneofclarity.Byknowingwhateachinputandoutputportisfor,extraconnectionscanbemadetoitwithouthavingtoexpandthebookmarkcontents.

RearrangingBookmarks

Althoughyoucancollapseonebookmarkatatime,anoptionontheNavigatorwindowallowsyoutocollapseallbookmarksatonce:

Doingsoillustrateshowmuchspacecanbesavedinasinglecanvas:

CollapsibleBookmarkBestPractice

29

Ofcourseithelpstothenmovethebookmarksclosertogethertoreusethatspace:

Butbesidesreusingthespace,thereisanotherreasonforthisreorganization.

Whenbookmarksarepositionedclosetoeachother,re-openingacollapsedbookmarkadjuststhelayoutoftheworkspace,movingothertransformersorbookmarksoutofthewaysothatitscontentsareshownwithoutoverlap.Re-closingthebookmarkcausestheoppositetooccur.

Forexample,intheabovescreenshot:

CollapsibleBookmarkBestPractice

30

...ifanybookmarkisexpanded,thenthebookmarksarounditaremovedtoonesidetoaccommodatethenewspacerequirements.Whenthebookmarkiscollapsedagain,thereversetakesplace,togivethesamecompactlayoutasbefore.

CollapsibleBookmarkBestPractice

31

CollapsedBookmarksandCaches

Collapsiblebookmarkshavebeendesignedverycarefullysothattheyinteractwithcachesandpartialruns.

RunningaBookmark

Thepop-upoptionsforapartialrun,aswellasappearingontransformers,arealsopresentonbookmarks:

TheoptionsarecalledRunToContainedandRunFromContained,andappearwhetherthebookmarkiscollapsedorexpanded.

Thisobviouslymakesiteasiertorunasectionofworkspace,particularlywherethebookmarkiscollapsed.

ImprovingPerformance

Whenanauthorcreatesaverylargeworkspace,asisusuallythecasewhentherearemanybookmarks,runningwithfeaturecachingturnedoncouldcreatealargeamountofunwantedcaches:

NoticeintheabovescreenshotthateverytransformerinthePrepareDataforMatchingbookmarkisbeingcached.

Toavoidthis,featurecachinghasbeendesignedtoavoidcachingdataforeverytransformerinsideacollapsedbookmark.Whenaworkspaceisrunalltransformersinthecollapsedbookmarkoperateasusual,buttheonlydatacachedisfortransformersrepresentedbyanoutputportonthebookmark:

Wecanseethatbyexpandingthebookmarkaftertheworkspacehasfinishedrunning:

CollapsibleBookmarksandCaches

32

Thissavesconsiderabletimeandresourceswhenworkingwithalargeworkspace.

Dr.Workbenchsays...Obviouslyyoudon'twanttoputaworkspaceintoproductionwhencachingisturnedon,regardlessofwhetheryourbookmarksarecollapsed.Thistechniqueisonlyrecommendedforuseinthedesign,authoring,andtestingphasesofworkspacecreation.

CollapsibleBookmarksandCaches

33

Exercise CachingandPartialRuns

Data Roads(AutoCADDWG)FirehallsandZones(GML)

OverallGoal CreatetimeanddistanceisolineswiththeNetworkCostCalculator

Demonstrates CollapsibleBookmarks

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Begin.fmwt

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Complete.fmwt

Herewe'llcontinuetheexercisefromtheCachingchapter.Ourgoalistoassistinplanningemergencyservicesbycreatingadatasetofisochrones-contourlinesdepictingtraveltimefromeachfirehall-formedicalemergencies.

1)StartWorkbenchStartFMEWorkbenchand,ifnecessary,openthetemplatefileforthepreviousexercise.Youcanfindacopyofitat:C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Begin.fmwt

Theworkspaceworksfine,butitneedsacoupleofadjustments,plusitisindireneedofbestpracticestyle.

2)AddBookmarkThefirstthingwecandoismaketheworkspacecanvastidierbybookmarkingalltheroadfeaturetypesandcollapsingitintoasmallerspace.

So,selectallofthefeaturetypesfortheAutoCADroadsreader.PressCtrl+B(orselectInsertBookmarkfromthetoolbar).NamethebookmarkAutoCADRoadsReader:

Exercise:EmergencyReponseProject

34

Ifwecollapsethebookmarknow(tryitandsee)we'llstillgetmultipleoutputsandconnections.Wedon'treallyneedthis.SoaddasingleJunctiontransformerbetweenalltheconnectionsandtheExpressionEvaluatorandresizethebookmark(expandingitfirstifnecessary)tofittheJunction:

Collapsethebookmarkandnowtherewillbejustasingleoutputport.Double-clicktheporttorenameit.NameitRoads:

Thishasobviouslysavedquitealotofspaceinthecanvas.

Exercise:EmergencyReponseProject

35

3)AddClipperToreturntothetaskathand,thecurrentworkspaceoutputcoversallofthecityofVancouver,whereasweonlyneedittoshowcontoursfortheareaofresponsibilityforfirehallnumber4.

ToclipthedataaddaClippertransformertotheworkspace.ConnectthecontourstotheClippeeinputportandtheMedicZonefeaturetypetotheClipperport:

Rearrangetheobjects/portstoensuretherearenooverlappingconnections.

4)DuplicateTesterTheMedicZonesdatacoversallofthecitytoo,andweonlyneedfirehallnumber4.WecanfilterthiswithaTester;infactwiththesametestastheexistingTester.

So,selecttheexistingTesterandpressCtrl+D(RightClick>Duplicate)toduplicateit.DroptheduplicateTesterintotheMedicZones-Clipperconnection:

Butdon'truntheworkspaceyet...

5)AddBookmarkIfweruntheworkspacenowthereareanumberofyellowcachesthatwillbeupdatedandsaved.However,wedon'treallyneedthese.Cachesareusefulfortestingasectionofworkspace,butwealreadyknowthemainpartofthisworkspaceiscorrect.

So,addabookmarkaroundthefourmainprocessingtransformers(ExpressionEvaluator,Tester,NetworkCostCalculator,ContourGenerator):

Nowcollapsethebookmark,renameitsports,andrearrangecontentaroundit:

Exercise:EmergencyReponseProject

36

Noticethatthereisnowonlyasinglecacheforthewholebookmark.

6)RunWorkspaceRe-runtheworkspacebyselectingtheClippertransformerandchoosingRunToThis.

Wearenowsavingcachespacebecauseonlyonecacheiscreatedforthebookmarkedparts.Expand(andre-collapse)theIsochronebookmarktoprovethis.TheContourGeneratorhascachesbecauseitisthelasttransformer.TheTesterhascaches,butonlybecauseitalreadyhadthem:

Noticethatasyouexpand/collapsethebookmark,otherobjectsmoveinresponsetokeepfromoverlapping.

NowinspecttheClipper:Insidecache.Youshouldseethattheisochronesareclippedtotheareaofresponsibilityforfirehall4.

7)SaveWorkspaceRe-savetheworkspaceasatemplate-includingcaches-soitisreadyforfurthereditingifrequired.

TIPToseesomeadvancedtasksforthisproject,checkouttheGISTutorialontheFMEKnowledgeCentre.

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

AddandcollapsebookmarksAddaJunctiontoreducethenumberofbookmarkoutputportsRenamebookmarkoutputportsRearrangeobjectsaroundabookmarksothattheymovewhenthebookmarkiscollapsed/expandedCollapseabookmarktosavecachingexcessdata

Exercise:EmergencyReponseProject

37

Exercise:EmergencyReponseProject

38

PrecisionandTolerance

ToleranceandPrecision

39

ToleranceParameters

FME2018introducesatoleranceparametertomanyexistingtransformers,andonenewtransformer:

AreaGapAndOverlapCleanerAreaOnAreaOverlayerClipperDissolverFeatureMergerGeneralizerIntersectorLineOnAreaOverlayerLineOnLineOverlayerSnapperTilerTopologyBuilder

TheAreaGapAndOverlapCleaneris,ofcourse,thenewtransformer.It'sadirectreplacementfortheSliverRemover.TheSnapperalreadyhadatoleranceparameter,butthathasbeenupdatedtousethisnewfunctionality.

Sowhythesetransformers?Becauseit'sasetoftransformersthatuseacommonpieceofcode.Thatcodewasupdatedtoallowatolerancetobeappliedandwasexposedasanewparameter:

Thetoleranceparametershavethreedifferentoptions:

NoneAutomaticCustomValue

NonemeansthetransformeroperatesasitalwaysdidinFME;withoutanytoleranceapplied.Theoperationwillbecarriedoutusingstrictmatchesofcoordinates.Thisisthedefaultvalue(for2018.0atleast).

AutomaticmeansFMEwilldecideuponatolerance.HereFMEistryingtoresolveminorissueswithcoordinates,suchascoordinateprecisionandrounding.

CustomValuemeanstheuserwillenteratolerance.Theuseristryingtoresolveanyissuestheybelievecanbefixedthroughtolerance,suchasbadgeometry.

CollapsedPorts

SometransformerswiththenewtolerancesettinghaveaCollapsedoutputport.Thisisforfeatureswhosesizeissmallerthanthetolerancevalue,andwhichthereforecollapsetoasinglepoint.

TIPBecausetheAutomaticmodeisnewertechnology,itsperformanceinmanycasesisactuallyfasterthanhavingnotoleranceatall!

ToleranceParameters

40

ToleranceParameters

41

ExampleGeometryProblems

Ingeneral,datacannotjoin,meet,orintersectbecauseofoneoftwoissues.

Firstlythegeometryofthedatacanhavesimplybeencreatedbadly:

Inthatexamplethetwolinesaremeanttoconnect,butinsteadhaveagapofaround10metres.

Thesecondsourceofproblemsisgeometrythathasverytinyimperfectionsduetoissuessuchascoordinaterounding:

Inthiscasethetwolinesaremeanttoconnect,butinsteadthereisaverysmallgapof0.00000001metres.Thissortoferrorisverydifficulttolocatevisually,becauseofitssmallsize.

Suchanissuemightarisebecauseofcoordinaterounding(coordinatesareroundedtoaprecisionthatleadstogaps),becauseofdifferentdataformats(whereeachformatstorescoordinatesinslightlydifferentways),orperhapsbecauseofsomeotherautomatedprocessthathasbeencarriedout.

NoTolerance

Whentoleranceissettonone,thentheabovefeaturesarenotconsideredamatch.

Forexample,inaTopologyBuilder,thegapsineachcasewouldleadFMEtoassumethateachgeometryisaroadthatendswithoutaconnection.Therewouldbetwonodeswithagapbetweenthem:

Dependingonthedatathismayormaynotbethecorrectresult.Withagapofonly0.00000001metreswecansafelyassumeitisnotcorrectinthisexample.

ToleranceExamples

42

AutomaticTolerance

Whentoleranceissettoautomatic,thenFMEwillassessthedataandcomeupwithareasonabletolerancetosolveminorproblems.Forexample,inaTopologyBuilderFMEwouldcorrectlydeterminethatthesmallergapisanerrorthatshouldbecorrected:

However,itwillnotfixthegrossererrorthatcausedthelargertolerancegap:

That'sbecausethereisnowayFMEcouldreasonablydefinethatasaproblem,orfixitwithoutpossiblyalteringothersimilargaps.

CustomTolerance

Whenthetoleranceissettoacustomvaluethentheauthoristryingtoresolveissuesatascaleoftheirownchoosing.Forexample,intheTopologyBuildertheycouldsetatoleranceof10.Thatwouldsolveboththesmallissuesandthelargerone:

Butitmightalsomistakenlyfixissueswheretwopointsarelegitimatelywithin10metresofeachother,likeatNode2.

Soingeneralcasesit'sbesttouseautomatictolerance,unlessyoucanbesurethatacustomtolerancewillworkasyouwantitto.

ToleranceExamples

43

Exercise ToleranceandDataCleaningData PropertyBoundaries(MicroStationDGN)

OverallGoal Cleanlinedatainordertobuildpropertypolygons

Demonstrates Intersection,DataCleaning,andTolerance

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Tolerance-Ex1-Begin.fmwt

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Tolerance-Ex1-Complete.fmwt

Forasmallexercise,you'vebeenaskedtoturnasetoflinework(representingpropertyboundaries)intopolygonfeatures.ThisiseasilydonewiththeAreaBuilder.However...theremaybesomedatacleaningrequirementsthataren'tassimple.

1)StartWorkbenchStartFMEWorkbenchandopenthestartingworkspacetemplate.

TheworkspaceincludesareadertoreadasetofMicroStationDGNfeatures.ItalsoincludesabookmarkedsetoftransformersthatwerepreviouslyusedtocleandatainFME2017:

It'ssooldthattheyevenincludeInspectortransformers!Howquaint!

2)AddAreaBuilderFirstlylet'signorethatbookmarkedsectionandsimplyaddanAreaBuildertransformerconnectedtothereaderfeaturetype(ParcelLines):

Turnonfeaturecachingandrunthatpartoftheworkspace.InspectbothAreaBuildercaches:

Exercise:ToleranceandDataCleaning

44

Noticethatthereareseveral"unused"features.Whereafeatureisunuseditdidnotcreatetheproperboundaryandtheareacoverstwo(ormore)properties.

3)MoveAreaBuilderTransformerWenowknowthedatawon'tbuildproperpolygonswithoutsomecleaning.SodisconnecttheAreaBuildertransformerandmoveittotheendoftheworkspace,connectedinplaceoftheCleanNetworkInspectorfromthebookmark:

DeleteanyotherInspectortransformerstoo,connectthesourcedatatotheSnappertransformerinthebookmark,andruntheworkspace.

Inspecttheoutput.TherewillbeperfectlyformedpolygonsandnoIncompletefeatures.

Sowe'vefinishedtheproject.Except...surelythereisabetterwayin2018?

4)DeleteBookmarkandTransformersLet'sstartoutbydeletingthe2017bookmarkandallitscontents,plusthenewAreaBuildertransformer,sothatallwe'releftwithisthereaderfeaturetype.That'sjusttocleartheworkspaceofclutter.

5)AddIntersectorandAreaBuildertransformersPlaceanIntersectortransformer(ithasthenewToleranceparameterandsomayhelpus)plusanewAreaBuildertransformer:

Exercise:ToleranceandDataCleaning

45

InspecttheIntersector'sparameters.MakesurethetoleranceparameterissettoAutomaticandre-runtheworkspace.Inspecttheoutput:

TherearesomeIncompletefeaturesbutnotasmanybadareas.TheIntersectorhasobviouslydonesomegood.However,thetolerancerequiredisjusttoomuchforFMEtohandleautomatically.

6)SetManualToleranceInspecttheIntersectorparametersagainandthistimesettheToleranceparametertoacustomvalueof0.2.Re-runtheworkspace.Theresultsarebetter,butnotquitecomplete.

Setthetoleranceparameterto0.3andtryagain.

Thistimetheareafeaturesbuildcorrectly.AnyIncompletefeaturesaresmalloverlapsandcanbeignored.

Sowehavereplicatedtheresultsofthe2017workspacebyreplacingallofthebookmarkcontentswithasimpleIntersectortransformer!Infacttheresultsarebetterbecausethe2017workspaceproducedsomeodditiesthatshouldn'texistinthedata(that'swhyithadmoreareasoutputthanin2018).

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

ApplyanAutomatictoleranceontheIntersectortransformerApplyaCustomtoleranceontheIntersectortransformer

Exercise:ToleranceandDataCleaning

46

MicrosoftFormatsandTransformers

MicrosoftFormats

47

MicrosoftFormatsandTransformers

FME2018introducesanewsetoftransformersandformatsforMicrosoft-relatedformats.Theseare:

Reader Writer Transformer

Excel Existing Existing New:ExcelStyler

Word None New New:MSWordStyler

PowerPoint None New New:PowerPointStyler

Inshort,FME2018hasnewwritersforWordandPowerPoint,andnewstylingtransformersforallthreeformats.AnExcelreaderandwriterhasexistedinFMEforalongtime,butthewriterhasnewformattingoptionsthatcanbefulfilledusingtheExcelStyler.

FormatsandTransformers

48

MicrosoftWord

TheMicrosoftWordwriterinFMEallowswritingfeaturestoWorddocuments.Thewriteronlyhasasingleparameter,thattodefineaWordtemplatetouse.

ThedatasenttothewritermustalsofirstbestyledusingtheMSWordStylertransformer.

MSWordStyler

TheMSWordStylertransformerallowsthecreationofvariouscontenttypes:

Thiswayadocumentcanbebuiltupfromseveralcomponents,withheadings,images,lists,tables,andplaintext(paragraphs).Asadifferentcontenttypeisselected,theavailableparameterschangeinthedialog.Forexample,forTablecontenttheparameterslooklikethis:

...withtheoutputofthatlookinglikethis:

MicrosoftWord

49

MapfeaturescannotbewrittendirectlytotheMicrosoftWordwriter,butcanbewrittenasanimagebyfirstrasterizingthedatawithanImageRasterizerorMapnikRasterizertransformer.

WriteOrder

Asyoumightexpect,featuresarewrittentotheWorddocumentintheordertheyarrive,soaworkspaceauthorcancontrolthedocumentbycontrollingtheorderoffeaturesusingaSortertransformer.

MicrosoftWord

50

MicrosoftPowerPoint

TheMicrosoftPowerPointwriterinFMEallowswritingfeaturestoPowerPointpresentations.LiketheWordformat,thePowerPointwriteronlyhasasingleparameter,thattodefineatemplatetouse.

Aseparateslideinthepresentationiscreatedforeachfeature.

ThedatasenttothewritermustalsofirstbestyledusingthePowerPointStylertransformer.

PowerPointStyler

ThePowerPointStylertransformerallowsthecreationofvariousslidetypes:

Thiswayapresentationcanbebuiltupfromseveralcomponents,withcharts,maps,images,tables,andotherplaintextcontent.Asadifferentcontenttypeisselected,theavailableparameterschangeinthedialog.Forexample,forTablecontenttheparameterslooklikethis:

...withtheoutputofthatlookinglikethis:

UnliketheWordwriter,MapfeaturescanbewrittendirectlytotheMicrosoftPowerPointwriter.Featurescanbegroupedtogetherinoneslideandeachfeaturebecomesavectorobjectinthatslide.

ThePowerPointStylertransformeralsohasaparametertocontroltheorderoffeaturesbeingwritten.IfthisisleftasAutomatic,thenallthefeaturesgoingtotheMicrosoftPowerPointWriterwillbewrittentoapresentationinthesameordertheyarrive,meaningtheslideswillbeinthatorder.

MicrosoftPowerPoint

51

SettingthisparameteronlymakessensewherethereismorethanonePowerPointStylertransformer.Ifanumberisenteredintothisfield,thenthefeaturesaresortedbythatnumberonwriting.Forexample,atitleslidecouldbegiventhenumber1toallowittobefirstinthepresentation.

Ifmorethanonefeaturehasthesameslideorder,thentheyarewritteninthesameordertheyarrive.

TIPThetemplatefileforaPowerPointcanbedesignedintheMasterslideviewinPowerPoint.Forexample,settingabackgroundimageinthetemplatecouldleadFMEtoproduceamapslidelikethis:

ParkImagebyOCParks,CCBY-ND2.0

Noticehowamapslideallowsthecreationoflabelsoneachfeature(alsotosetthefeaturecolor)andthecreationofa"caption"textboxintheupper-left.

MicrosoftPowerPoint

52

MicrosoftExcel

FMEhashadaMicrosoftExcelreaderandwriterforanumberofyears.However,newin2018thewriterhasgainedanumberofnewformattingoptions,andtheseoptionscanbecontrolledwiththenewExcelStylertransformer.

ExcelStyler

TheExcelStylertransformerallowsstylingtobeappliedtoeitherindividualcellsortoentirerows:

Itallowssettingoffont,fill,border,andtextalignmentstyles:

Asshownabove,parameterscanbesetbyattributes;sofeaturescanbecoloredonthebasisofanattributeandthatattributeusedtosetthecolorintheExcelStyler,givingsomethinglikethis:

MicrosoftExcel

53

...wherecolorisappliedonthebasisofastreetnameattribute.

TheExcelStyleralsoallowsforsettingcellformat(fornumbers)andcellprotection(whetheritishiddenorlocked).

MicrosoftExcel

54

Exercise ThePowerPointWriter

Data Libraries(EsriGeodatabase)LandBoundary(EsriShapefile)

OverallGoal CreateaPowerPointpresentationshowingvariousitemsofinterestinthecityofVancouver

Demonstrates Microsoft-relatedtransformersandformats

StartWorkspace None

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Microsoft-Ex1-Complete.fmw

Aspartofaconferenceoncommunitymapping,youhavebeenaskedtoprovideapresentationondifferentaspectsofcommunityinthecityofVancouver.YourealizethisisagreatchancetobothcarryoutthetaskanddemonstrateFME'spotentialforsuchprojects.

We'llbeginthetaskbycreatingaslideaboutlibraries.Sincewe'venotusedthisformatortransformerbefore,thiswillbeajointvoyageofexploration!

NB:Ifyoudon'thaveaPowerPointvieweronyourcomputer,thenyoucanviewfilesonlineonsuchsitesasonlinedocumentviewer.com

1)StartWorkbenchStartFMEWorkbench.Generateaworkspaceasfollows:

ReaderFormat EsriGeodatabase(FileGeodbOpenAPI)

ReaderDataset C:\FMEData2018\Data\CommunityMapping\CommunityMap.gdb

ReaderParameters TableList:Libraries

WriterFormat MicrosoftPowerPoint

WriterDataset C:\FMEData2018\Output\Training\VancouverCommunity.pptx

WriterParameters Template:C:\FMEData2018\Resources\DesktopBasic\VancouverTemplate.pptx

2)AddPowerPointStylerTransformerIfweranthetranslationnow,wewouldn'tgetanyslidesintheoutput.That'sbecauseallthefeaturesneedtobestylizedforthePowerPointwriterfirst.So,addaPowerPointStylertransformerbetweenthereaderandwriterfeaturetype.

Inspectthetransformerparameters.Set:

SlideType:MapSlideSlideSubtype:MapWithCaptionTitle:VancouverLibrariesCaption:AMapofLibrariesinVancouver

Nextsetthelabelparameterstolabeleachobjectwiththelibraryname.UseafontandcolorofyourchoosingandmakesureWrapLabelTextissettoNo:

Exercise:VancouverCommunityMappingProject

55

ClickOKtoclosethedialogandacceptthechanges.

TIPThePowerPointStyleralsohasaparameterforHyperlink.Ifyoulike,settheHyperlinkparametertopointtotheattributeLibraryURL.ThatwayalltheoutputfeatureswillhavehyperlinksinthePowerPointfile.

3)Add2DEllipseReplacerTransformerApointfeaturewon'tshowupverywellinaPowerPointslide,solet'sturnthelibrariesintosomethinglarger.

Adda2DEllipseReplacertransformerbeforethePowerPointStyler.Setituptocreateanellipseofsize200x200:

Acceptthechanges,turnonfeaturecaching(ifnotalready)andruntheworkspace.Theresultwillbeaseriesofslides,eachwithonelibrary:

Exercise:VancouverCommunityMappingProject

56

We'dpreferasingleslidewithalllibraries,sowe'llhavetochangethat.

4)AddAttributeCreatorTransformerThePowerPointStylertransformerhasanoptiontogroupobjects,butweneedanattributetogroupthemby.SoplaceanAttributeCreatortransformerbetweenthe2DEllipseCreatorandthePowerPointStyler(it'simportantforittogohereforalaterstep).

Usethetransformertocreateanattribute(callitGroup)withafixedvalue(say,1).

NowinspectthePowerPointStylerparametersagain.Nearthebottomofthedialog,underFeatureHandling,setCreateSlideFromtoMultipleFeatures,andsetSlideGroupIDtotheGroupattribute:

Nowre-runtheworkspace.Theoutputshouldbeasingleslidewithallthelibrariesonit:

5)AddShapefileReaderThemapiscorrect,butitreallyneedsabackgroundtothelibraryfeatures.We'llcreatethatwithadatasetoftheVancouverlandboundary.SoselectReader>AddReaderfromthemenubarandaddthefollowing:

ReaderFormat EsriShapefile

ReaderDataset C:\FMEData2018\Data\Boundaries\LandBoundary\VancouverLandBoundary.shp

Exercise:VancouverCommunityMappingProject

57

ConnectthenewlyaddedfeaturetypetotheAttributeCreatorinputport:

WeconnectittotheAttributeCreatorportsothatitgetsthesameattributevalueandappearsonthesameslideasthelibraryfeatures.

TIPIf(instep4)youweretemptedtouseanexistingattributeratherthancreateanewone,thisiswhywedidn't.Wewantsomethingthatwillworkformultipledatasets.It'salsowhyitwasimportanttoputtheAttributeCreatorafterthe2DEllipseReplacer,becausewedon'twanttoturnotherdataintoellipsestoo,justthelibraries.

6)AddSorterIfweruntheworkspacenowyouwillsee,however,thatallisnotwell.Thelandboundaryfeatureoverlaysthelibraries.Tohandlethiswemustsortthedatainsuchasorderthatthelibrariesarrivefirst.ThiswecandowithaSortertransformer.

Wecouldalsocreateattributesforeachdatatype,butwe'llavoidthatbyreusingtheformatattributefme_feature_type.

Soopentheparametersdialogforafeaturetype(itdoesn'tmatterwhich),clicktheFormatAttributestab,andputacheck-markagainstfme_feature_type:

ClickOKtoclosethisandputaSorterintotheworkspacejustbeforethePowerPointStyler.Setituptosortbyfme_feature_type,alphabetic,descending:

Exercise:VancouverCommunityMappingProject

58

Nowwhentheworkspaceisrun,weshouldgetlibraryfeaturesabovethecityboundary.

7)AddFeatureColorSetterThemapisnowvisible,butitwouldbebetterifwecouldgetastandardcolorforeachlibraryobject.WecansetcolorinsidethePowerPointStyler,butthatwouldcauseallfeaturestobethesamecolor,includingthecityboundary.

So,placeaFeatureColorSettertransformerbeforetheSortertransformer.Openitsparametersandsetituptocreaterandomcoloring,butbasedontheattributefme_feature_type:

Thismeansthateachdifferentfeaturetypewillgetitsowncolor.InthePowerPointStylertransformerlocatetheForegroundFillColorparameterandsetittofme_fill_color:

TIPThistechniqueislikecreatingagroup-bywithcolor.It'salsoagreattechniquetousefortheExcelStylertransformer,whereyouwanttocreatedifferentcolorsperrow,butbasedonanattributevalue.

Exercise:VancouverCommunityMappingProject

59

8)AddTitleS lideOnelastthing:wereallyneedatitleslideforthepresentation.Thisiseasilyachieved.PlaceaCreatorontothecanvas,connectittoasecondPowerPointStyler,andconnectthattotheoutput:

OpentheparametersdialogforthesecondPowerPointStyler.Setitasfollows:

SlideType:TitleSlideTitle:TheVancouverCommunitySubtitle:AseriesofFME-generatedmapsandchartsSlideOrder:1

TheSlideOrderparameterisimportantbecausewewantthistobethefirstslideinthepresentation.Yes,wecouldrelyonthisfeaturebeingwrittenfirst,butthat'shardertocontrol.AlsovisitthefirstPowerPointStylertransformerandsetitsSlideOrderto2,toensureitcomessecondinthepresentation.

Re-runtheworkspaceandchecktheoutput.Youshouldnowhavetwoslides(onetitleslide,oneforlibraries),withallslidesandfeaturesinthecorrectorderandwithuniquecolorsper"layer":

ThisformsagreatbasisforyourpresentationonthecommunitiesofVancouver!YoucouldevenstartexperimentingwithusingtheBookCountandCirculationattributesasthesizeoffeatureinthe2DEllipseReplacer,tocreateathematicmap.

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

CreatemapswiththePowerPointStylerReplacepointfeatureswithellipsesusingthe2DEllipseReplacerCreateslidegroupsformultiplefeaturesperPowerPointslideSortfeaturesintothecorrectorderforvisualizingonaslideSetfeaturecolorsonarandomgroup-bybasisSortslidesintothecorrectorderinapresentation

Exercise:VancouverCommunityMappingProject

60

Exercise:VancouverCommunityMappingProject

61

FeatureJoinerTransformer

FeatureJoiner

62

WhatistheFeatureJoiner?

TheFeatureJoinertransformerisnewforFME2018.It'saSQL-basedalternativetotheFeatureMergertransformer,thatjoinsdatatogetheronthebasisofcommonkeysandusingSQLterminologytodefinetheresults.

Onthecanvasitlookslikethis:

Andtheparametersdialoglookslikethis:

JoinModes

JoinedfeaturesexitthetransformerthroughtheJoinedport.WhathappenstotheremainingfeaturesiscontrolledbytheJoinModeparameter.

Therearethreejoinmodes:Inner,Left,andFull.

InInnermodefeatureswithoutajoinexitthrougheithertheUnjoinedLeftorUnjoinedRightport.

InLeftmode,LeftfeatureswithoutamatchstillexitthroughtheJoinedport.RightfeatureswithoutamatchexittheUnjoinedRightport.

InFullmode,bothLeftandRightfeatureswithoutamatchstillexitthroughtheJoinedport.

Wecanvisualizethisbehaviourwithadiagram:

Mode Description Depiction JoinedOutput UnjoinedLeft UnjoinedRight

Left

Leftfeatureslookforamatchandareoutputwhethertheyfindamatchor

AllmatchesplusunmatchedLeftfeatures

None UnusedRightfeatures

FeatureJoinerModes

63

not

Inner

Leftfeatureslookforamatchandareoutputiftheyfindone

Allmatchesonly UnmatchedLeftfeatures

UnusedRightfeatures

Full

BothLeftandRightfeaturesoutputthroughtheJoinedoutputport,whethertheyfindajoinornot

AllmatchesplusunmatchedLeftandRightfeatures

None None

FeatureJoinerModes

64

MultipleJoinMatches

EverytimeafeatureontheLeftmatchesoneontheRight,anewfeatureisoutputfromtheJoinedport.ThismeansthatwheretherearemultiplematchesforasingleLeft(Requestor)feature,therewillbemultipleoutputfeatures.

ThisiswhytheFeatureJoinerhasnoequivalenttotheFeatureMerger“handleduplicatesuppliers”parameter.Itdoesnotneedthis.ItjustcreatesanewJoinedfeatureforeachduplicatesupplier.

Tounderstandthis,let’stakealookatthiswithsomefeaturecountnumbersincluded:

Thereareeight(8)facilityfeaturesand1,463,442addresses.HowmanyfeatureswillexittheJoinedport?Well,itdepends.

Ifweassumeastrict1:1matchbetweeneachFacilityandanAddress,then8featureswillexitasJoined.Why?Becausewehave8matches.Dependingonthemodethecountwillbeoneofthese:

Joined UnjoinedLeft UnjoinedRight

Left 8 0 1,463,434

Inner 8 0 1,463,434

Full 1,463,442 0 0

However,wemightnothavesuchaclean1:1matchbetweenFacilityandAddresstable.Wecouldhave1:M,M:1,orevenM:M.

Let’ssayeachFacilityhastwomatchesintheAddresstable.Thenwegetthis:

Joined UnjoinedLeft UnjoinedRight

Left 16 0 1,463,426

Inner 16 0 1,463,426

Full 1,463,442 0 0

WhyarewegettingmorefeaturesoutthanweputintotheLeftport?Becausewearegettingonefeaturepermatch,andthereare16matches!

ThisisnotsomethingtheFeatureMergerwoulddo.Itwouldeitherignorethesecondmatchorcreatealist.ButthisisdesignedtobeaSQLequivalent,andsoitworksdifferently.Infact–inanextremecase–ifweassumedthateveryFacilityhasAddressID=1,andeveryAddresshasAddressID=1,theneveryfacilitywouldmatchtoeveryaddress!

Thatwouldgiveus8x1,463,442=11,707,536matches

IfyoudidwantthesameastheFeatureMergerwouldgive-onematchedrecordwithalistofmatches-thenyousimplyneedtocreatealistwithaListBuildergroupingbyyourjoinkeys.

MultipleJoinMatches

65

Exercise TheFeatureJoiner

Data Addresses(EsriGeodatabase)CrimeStatistics(MicrosoftExcel)

OverallGoal Joincrimestatisticstoeachaddressusingstreetnameandblocknumber

Demonstrates TheFeatureJoinertransformer

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\FeatureJoiner-Ex1-Begin.fmw

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\FeatureJoiner-Ex1-Complete.fmw

Yourtaskfortodayistomakeajoinbetweenaddresspointsandadatasetofcrimestatistics.Anexistingworkspacehasalreadybeensetuptopreparethedatabystandardizingtheattributesrequiredtomakeajoinkey.

1)OpenWorkspaceOpenthebeginningworkspacefortheexercise,FeatureJoiner-Ex1-Begin.fmw

Inspecttheworkspacecontents:

NoticethattheStringReplacerandtheStringCaseChangerinparticulararesetuptoadjustthePostalAddressattributes(200HighStreet)insuchawayastomatchthoseinthecrimedata(2XXHIGHSTREET).

AlsonoticethatthecrimedataisbeingreaddirectlyfromtheCityofVancouverwebsite.Weprobablywon'twanttoreadthatmoretimesthanisnecessary,sowillmakeuseofcaches.

TIPIf,forwhateverreason,youcan'taccessthedataontheCityofVancouverwebsite,changethereaderparameterontheExcelreadertopointto:C:\FMEData2018\Data\Emergency\crime_xlsx_all_years.zip

2)RunWorkspaceBeforeputtingdownanytransformers,turnonfeaturecachingandruntheworkspace.Wecan'tbesurehowlongitwilltaketofillthecaches,andwecancarryonbuildingtheworkspacewhileitruns.

3)PlaceAttributeManagerAFeatureJoinertransformerdoesnotallowuseofatexteditortoconstructajoinkey,thereforeanyjoinkeywewishtomakemustbedoneinanAttributeManager.

So,placeanAttributeManagertransformer,insidethePrepareAddressesbookmark,connectedtotheStringCaseChanger:

Exercise:CrimeAnalysisProject

66

OpentheAttributeManagerparametersdialog.CreateanewattributecalledJoinKey.Fortheattributevalueclickthedrop-downlistandchoosetoopentheTextEditor.Intheeditorenter:

@Trim(@Value(Number)@Value(Street))

...withaspacecharacterbetweentheNumberandStreetattributes:

4)PlaceFeatureJoinerAddaFeatureJoinertransformer,withthenewAttributeManagerconnectedtotheLeftinputportandtheCrimeDatafeaturetypeconnectedtotheRight:

Opentheparametersdialog.SettheJoinModetoInner,becausewewanttoonlykeepaddresseswherethereisamatchtocrimeonthesameblock.LeavetheothertwoJoinModeparameterstotheirdefault(UseLeft):

Exercise:CrimeAnalysisProject

67

NowselectthenewlycreatedJoinKeyattributeastheLeftJoinOnkey.ChoosetheexistingcrimedataattributeHUNDRED_BLOCKastheRightJoinOnkey:

Closethedialog.

5)RunWorkspaceRuntheworkspacebyselectingtheFeatureJoinerandchoosingRunToThis.Besidesnoticinghowfastthetranslationruns,noticethattherewillbe67,896matchedfeatures.That'sbecausetherearemultiplecrimesperblockandsomultiplematchesperaddress.

Inspectthecacheddataandyouwillnoticethatthereisonejoinperfeature,ratherthanalistsuchastheFeatureMergermightcreate.

6)AddAttributeValueMapperThereasonwewanttocarryoutthisjoinistoassesscrimeforeachaddress.We'lldothisbygivingavaluetoeachcrimethatiscommittedandcreatingasumforeachaddress.

So,placeanAttributeValueMappertransformerconnectedtotheFeatureJoiner:Joinedoutputport.OpentheparametersandsettheSourceAttributetoTYPEandtheDestinationAttributetoCrimeValue:

NowclicktheImportbutton.We'llusethistoimportalistofdifferenttypesofcrimefromthecrimedataset.Enter:

ReaderFormat MicrosoftExcel

ReaderDataset C:\FMEData2018\Data\Emergency\crime_xlsx_all_years.zip

ClickNext.Whenprompted(itwilltakeawhiletoscanthedataset)pick2017astheFeatureTypeandchangeImportFromtobeAttributeValues:

Exercise:CrimeAnalysisProject

68

NextselectTYPEastheSourceValue(noDestinationValueneedbeselected)andclickImport.FMEwillimportalistofdifferentcrimes.Giveeachcrimeavalue(sayfrom1-25)basedonwhatyouconsidertheseverityofthecrimetobe:

Don'tworry,it'sjustahypotheticalexercise.Youwon'tbemarkedonwhatvaluesyou'veawarded!

7)AddAggregatorNowweneedtogroupalljoinedfeaturesbytheiraddress,andaddtogetherthecrimevalues.AListBuilderandListSummermightbeonesolution,butherewe'lluseanAggregatortransformerbecauseitcandobothactionsinone.

SoplaceanAggregatorconnectedtotheAttributeValueMapper.Opentheparametersdialog.Set:

GroupBy:PSTLADDRESSCountAttribute:NumberOfCrimesAttributestoSum:CrimeValue

IfwewantedtocreatealistofcrimeswecoulddothatwiththeGenerateListoption,butwewon'tdothatfornow:

Exercise:CrimeAnalysisProject

69

NowruntheworkspaceusingRunToThisontheAggregator:

TheoutputoftheAggregator(4,271features)plustheUnJoinedLeftoutputontheFeatureJoiner(9,326)shouldequalthenumberofaddressesenteringtheFeatureJoiner(13,597).Itdoes,sowecantellwehavethecorrectnumberoffeatures.

8)AddCenterPointReplacerInspecttheAggregatorcache.Noticethateachfeatureisactuallymadeupofamultipointfeature.Thiswedon'tneed,soaddaCenterPointReplacertransformertotheendoftheworkspacetoreducethistoasinglepoint.

Nowwehaveourdesiredoutput:afeatureperaddresswithacountofthenumberofcrimesinthatblockandasumoftheirseverity.Withthisinformationwecould(ifwewanted)calculatetheaveragecrimeseverityperaddress/block,andmaybeevenmapthisdatawithperhapstheMapnikRasterizertransformer.

Savetheworkspaceasatemplate,includingthefeaturecaches,anditwillbereadyforyoutocontinuetheprojectthenexttimeyoustartFME.

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

CreateajoinkeyfortheFeatureJoinerUsetheFeatureJoinertransformertojointwosetsoffeaturesGroupmultiplejoinfeaturestogetherwithanAggregatorCreateasinglepointgeometryforgroupedjoins

Exercise:CrimeAnalysisProject

70

Exercise:CrimeAnalysisProject

71

CourseWrap-Up

AlthoughyourFMEtrainingisnowatanend,thereisagoodsupplyofexpertinformationavailableforfutureassistance.

CourseWrap-Up

72

ProductInformationandResources

SafeSoftwareWebSite

TheSafeSoftwarewebsiteistheofficialinformationsourceforallthingsFME.ItincludesinformationonFMEproducts,SafeSoftwareservices,FMEsolutions,FMEsupportandSafeSoftwareitself.

SafeSupportTeam

BehindFMEarepassionate,fun,andknowledgeableexperts,readytohelpyousucceed,withasupportteamphilosophybuiltontheprincipleofknowledgetransfer.

YoucanrequestproductsupportthroughaSupportCase(web/email)orusingaLiveChat.

YourLocalPartner

SafeSoftwarehaspartnersandresellersaroundtheworldtoprovideexpertiseandservicesinyourregionandyourlanguage.

YoucanfindalistofofficialpartnersontheSafeSoftwarePartnersPage.

ProductInformationandResources

73

SafeSoftwareBlog

TheSafeSoftwareblogprovidestechnicalinformationandgeneralthoughtsaboutFME,customers'usecases,andspatialdatainteroperability.Itincludesarticles,videos,andpodcasts.

FMEManualsandDocumentation

UsetheHelpfunctioninFMEWorkbenchtoaccesshelpandotherdocumentationforFMEDesktop.Alternatively,lookonourwebsiteundertheKnowledgeCentersection.

ProductInformationandResources

74

ProductInformationandResources

75

CommunityInformationandResources

SafeSoftwareactivelypromotesusersofFMEtobecomepartoftheFMECommunity.

TheFMEKnowledgeCenter

TheFMEKnowledgeCenterisourcommunitywebsite-aone-stopshopforallcommunityresources,plustoolsforbrowsingdocumentationanddownloads.

KnowledgeBase

TheFMEKnowledgeBasecontainsawealthofinformation;includingtips,tricks,examples,andFAQs.TherearesectionsonbothFMEDesktopandFMEServer,witharticlesontopicsfrominstallationandlicensingtothemostadvancedtranslationandtransformationtasks.

Q&AForum

FMEcommunitymemberspostFME-relatedmessages,askquestions,andshareinansweringotherusers’questions.Membersearn"reputation"and"badges"andthereisaleaderboardofthetop-participatingusers.ComeandseehowtheycanhelpwithyourFMEprojects!

IdeasExchange

FMEdevelopmentisverymuchuser-driven.TheIdeasExchangegivesusersthechancetoposttheirideasfornewFMEfunctionality,orimprovementstoexistingfunctionality,andallowseveryonetovoteontheproposedideas.Themorevotesanideagets,themorelikelyitistobeimplemented!

TheFMEChannel

ThisFMEYouTubechannelisforthosedemosthatcanonlybeproperlyappreciatedthroughascreencastormovie.Besidesthisthereareahostofexplanatoryandhelpfulmovies,includingrecordingsofmosttrainingandtutorials.

CommunityInformationandResources

76

CommunityInformationandResources

77

FeedbackandCertificates

Theformatofthistrainingcourseundergoesregularchangespromptedbycommentsandfeedbackfrompreviouscourses.

CourseFeedback

MissVectorsays...There'sonefinalsetofquestions–andthistimeyou'llbetellingmeiftheanswersarecorrectornot!

SafeSoftwaregreatlyvaluesfeedbackfromtrainingcourseattendeesandourfeedbackformisyourchancetotelluswhatyoureallythinkabouthowwellwe’remeetingyourtraininggoals.

Youcanfillinthefeedbackformnow,butyou'llalsoberemindedbyemailshortlyafteryourcourse.SafeSoftware'spartnerswhocarryouttrainingmayaskthatyoufillinaseparateform,butyoucanalsousetheofficialSafeSoftwareformifyouwish.

Certificates

Mr.E.Dict,(AttorneyofFMELaw)says...Inordertoproveyouhavetakenthistrainingcourse,acertificatewillbeemailedautomaticallytoanyonewhowasloggedonforthedurationofSafeSoftwarehostedcourses.

FeedbackandCertificates

78

ThankYou

ThankyouforattendingthisFMEtrainingcourse.

ThankYou

79