Table of Contents - Amazon S3...Course Overview The training will introduce new concepts and...

1.11.2

1.2.11.2.21.3

1.3.11.4

1.4.11.4.21.4.31.5

1.5.11.5.21.5.31.5.41.6

1.6.11.6.21.6.31.7

1.7.11.7.21.7.31.7.41.7.51.8

1.8.11.8.21.8.31.9

1.9.11.9.21.9.31.9.4

TableofContents

IntroductionAboutThisDocument

CourseOverviewCourseResources

FME2018OverviewNewFeaturesin2018

FeatureCachingCachingFeaturesPartialRunsExercise:EmergencyResponseProject

CollapsibleBookmarksCollapsingBookmarksCollapsibleBookmarkBestPracticeCollapsibleBookmarksandCachesExercise:EmergencyReponseProject

ToleranceandPrecisionToleranceParametersToleranceExamplesExercise:ToleranceandDataCleaning

MicrosoftFormatsFormatsandTransformersMicrosoftWordMicrosoftPowerPointMicrosoftExcelExercise:VancouverCommunityMappingProject

FeatureJoinerFeatureJoinerModesMultipleJoinMatchesExercise:CrimeAnalysisProject

CourseWrap-UpProductInformationandResourcesCommunityInformationandResourcesFeedbackandCertificatesThankYou

UpgradingtoFMEDesktop2018TrainingCourse

Thisisatrainingmanualforusersupgradingtothe2018versionofSafeSoftware’sFMEDesktopapplication.

Thetrainingwillintroducenewconceptsandterminology,andhelpstudentsbecomefamiliarwithmethodologythathaschangedsincethepreviousrelease.

CourseStructure

Thecourseismadeupofvarioussections,including:

Ageneraloverviewof2018AuthoringworkflowswiththePartialRuntoolsCollapsiblebookmarksandrelatedfunctionalityPrecisionandtoleranceinspatialtransformationWritingandstylingMicrosoftdataformatsUsingthenewFeatureJoinertransformer

CurrentStatus

Thecurrentstatusofthismanualis:COMPLETE:thismanualcanbeusedfortraining.

NB:Evenforcompletedcontent,SafeSoftwareInc.assumesnoresponsibilityforanyerrorsinthisdocumentortheirconsequences,andreservestherighttomakeimprovementsandchangestothisdocumentwithoutnotice.Seethefulllicensingagreementforfurtherdetails.

Introduction

AboutThisDocument

Thismanualisatrainingcourseforusersupgradingtothe2018versionofSafeSoftware’sFMEDesktopapplication.

LookoutforresidentsoftheCityofInteropolis,whowillappearfromtime-to-timetogiveyouadviceanddispenseFME-relatedwisdom.Infact,herecomessomeonenow:

Mr.E.Dict(AttorneyofFMELaw)says...OnbehalfoftheCityofInteropolis,welcometothistrainingcourse.Hereisthestandardlegalinformationaboutthistrainingdocumentandthedatasetsusedduringthecourse.

Besuretoreadit,particularlyifyou'rethinkingaboutre-usingormodifyingthiscontent.

LicensingandWarranty

Permissionisherebygrantedtouse,modifyanddistributetheFMETutorialsandrelateddataanddocumentation(collectively,the“Tutorials”),subjecttothefollowingrestrictions:

1. TheoriginoftheTutorialsandanyassociatedFME®softwaremustnotbemisrepresented.

2. RedistributionsinoriginalormodifiedformmustincludeSafeSoftware’scopyrightnoticeandanyapplicableDataSource(s)notices.

3. YoumaynotsuggestthatanymodifiedversionoftheTutorialsisendorsedorapprovedbySafeSoftwareInc.

4. Redistributionsinoriginalormodifiedformmustincludeadisclaimersimilartothatbelowwhich:(a)statesthattheTutorialsareprovided“as-is”;(b)disclaimsanywarranties;and(c)waivesanyliabilityclaims.

SafeSoftwareInc.makesnowarrantyeitherexpressedorimplied,including,butnotlimitedto,anyimpliedwarrantiesofmerchantability,non-infringement,orfitnessforaparticularpurposeregardingtheseTutorials,andmakessuchTutorialsavailablesolelyonan“as-is”basis.InnoeventshallSafeSoftwareInc.beliabletoanyonefordirect,indirect,special,collateral,incidental,orconsequentialdamagesinconnectionwithorarisingoutoftheuse,modificationordistributionoftheseTutorials.

Thismanualdescribesthefunctionalityanduseofthesoftwareatthetimeofpublication.Thesoftwaredescribedherein,andthedescriptionsthemselves,aresubjecttochangewithoutnotice.

DataSources

CityofVancouver

Unlessotherwisestated,thedatausedhereoriginatesfromopendatamadeavailablebytheCityofVancouver,BritishColumbia.ItcontainsinformationlicensedundertheOpenGovernmentLicense-Vancouver.

AboutThisDocument

Others

ForwardSortationAreas:StatisticsCanada,2011CensusDigitalBoundaryFiles,2013.Reproducedanddistributedonan"asis"basiswiththepermissionofStatisticsCanada.©ThisdataincludesinformationcopiedwithpermissionfromCanadaPostCorporation.

DigitalElevationModel:GeoBase

StanleyParkGPSTrail:UsedwithkindpermissionofVancouverTrails.com.

OpenStreetMapDatasets:©OpenStreetMapcontributors.

ContainsinformationlicensedundertheOpenGovernmentLicence–BritishColumbia

Copyright

Revisions

Everyefforthasbeenmadetoensuretheaccuracyofthisdocument.SafeSoftwareInc.regretsanyerrorsandomissionsthatmayoccurandwouldappreciatebeinginformedofanyerrorsfound.SafeSoftwareInc.willcorrectanysucherrorsandomissionsinasubsequentversion,asfeasible.Pleasecontactusat:

SafeSoftwareInc.Phone:604-501-9985Email:train@safe.comWeb:safe.comGitHub:github.com/safesoftware/FMETraining

SafeSoftwareInc.assumesnoresponsibilityforanyerrorsinthisdocumentortheirconsequences,andreservestherighttomakeimprovementsandchangestothisdocumentwithoutnotice.

Trademarks

FME isaregisteredtrademarkofSafeSoftwareInc.Allbrandorproductnamesaretrademarksorregisteredtrademarksoftheirrespectivecompaniesororganizations.

DocumentInformation

DocumentName:UpgradingtoFMEDesktop2018.0TrainingCourse

AboutThisDocument

CourseOverview

Thetrainingwillintroducenewconceptsandterminology,andhelpstudentsbecomefamiliarwithmethodologythathaschangedsincethepreviousrelease.

CourseStructure

Thecourseismadeupofvarioussections,including:

Ageneraloverviewof2018AuthoringworkflowswiththePartialRuntoolsCollapsiblebookmarksandrelatedfunctionalityPrecisionandtoleranceinspatialtransformationWritingandstylingMicrosoftdataformatsUsingthenewFeatureJoinertransformer

AbouttheManual

TheUpgradingtoFMEDesktop2018TrainingmanualnotonlyformsthebasisforthistrainingcoursebutisalsousefulreferencematerialforfutureworkyoumayundertakewithFME.

AllscreenshotsinthesematerialsweretakenusingFMEonWindowsServer2016.Thefontsused(especiallyinscreenshotsofthelogwindow)mayberesizedorotherwisechangedforimprovedlegibility.

CourseOverview

CourseResources

Anumberofsampledatasetsandworkspaceswillbeusedinthiscourse.

OnYourTrainingComputer

ThedatausedinthistrainingcourseisbasedonopendatafromtheCityofVancouver,Canada.

MostexercisesaskyoutoassumetheroleofacityplanneratthefictionalcityofInteropolisandtosolveaparticularproblemusingthisdata.

Whetherit'salocalcomputeroravirtualcomputerhostedinthecloud,you'llfindresourcesfortheexamplesandexercisesinthemanualatthefollowinglocations:

Location Resource

C:\FMEData2018\Data DatasetsusedbytheCityofInteropolis

C:\FMEData2018\Resources\DesktopBasic Otherresourcesusedinthetraining

C:\FMEData2018\Workspaces\DesktopBasic Workspacesusedinthestudentexercises

C:\FMEData2018\Output Thelocationinwhichtowriteexerciseoutput

<documents>\FME\Workspaces ThedefaultlocationtosaveFMEworkspaces

YoushouldalsofindFMEpre-installed,plusadigitalcopyofthismanual.

Pleasealertyourinstructorifanyitemismissingfromyoursetup.

YoucanfindthelatestversionofFMEDesktopandFMEServerforWindows,Mac,andLinux-togetherwiththelatestBetaversions-ontheSafeSoftwarewebsite.

CourseEtiquette

Foronlinecourses,pleaseconsiderotherstudentsandtestyourvirtualmachineconnectionbeforethecoursestarts.Theinstructorcannothelpdebugconnectionproblemsduringthecourse!

Forlivecourses,pleaserespectotherstudents’needsbykeepingnoisetoaminimumwhenusingamobilephoneorcheckinge-mail.

CourseResources

2018Overview

FME2018Overview

NewFeaturesin2018

FMEDesktop2018introducesmanynewtoolsandfunctionalitytobothFMEWorkbenchandtheFMEDataInspector.Thesetoolsmakewide-rangingchangestothemethodsworkspaceauthorsuseindesigningandcreatingworkspaces.

TopTools

Themostimportantnewtoolswillbecoveredduringthiscourse.Theyare:

CachesandPartialRunsTheabilityforFMEtocachedataateachpointofatranslation,andtore-usethatcacheddata.

CollapsibleBookmarksTheabilityforbookmarkstobecollapsedtosimpleinput/outputports,tohidethecontents.

PrecisionandToleranceParametersTheabilitytosettoleranceonanumberofspatialoperations,toworkaroundcoordinateprecisiondifferences.

MicrosoftFormatsTheabilitytowritedatatoMicrosoftPowerPointandMicrosoftWord,andtheadditionofstylingtransformersforallMicrosoftformats

FeatureJoinerASQL-basedtransformerforcarryingoutjoinsinamorepowerfulversionoftheFeatureMerger

OtherUpdates

Besidesthesetoptools,itisalsoworthmentioningotherhighlightsfromtheFME2018release:

NewFormats

TherearemultiplenewformatssupportedinFME2018,withover50newrasterformatsbasedonGDALtechnology.Someofthekeyformatsare:

AppleVenueFormat(AVF)ReaderandWriterBentleyi-ModelReaderEsriIndexed3DSceneLayer(I3S)LDAP(ActiveDirectory)ReaderPCD(PointCloudData)ReaderandWriterPDFReader(2D/Raster/Text)QlikDataExchangeWriterWellLogASCIIStandard(WLAS)Reader

ThereisalsoanFMEARformatandassociatedappontheiTunesstore.

NewTransformers

AsidefromMicrosoft-relatedfunctionality,newtransformersin2018mostlyrevolvearoundconnectingtowebservices:

HDFSConnector(Hadoop)ProjectWiseConnectorS3ConnectorSalesforceConnector

WorkbenchUpdates

FMEWorkbenchincludesanumberofupgrades-somesmall,somelarge-separatetowhatthiscoursecovers.Theseinclude:

Copy/pastetransformerparametersDynamiclogwindowfilteringFeatureWriteroutputportsNewgeometrytypesupport(clothoidsand3Darcs)SupportfornewGermanandAustraliancoordinatesystemdatumsTesterDateTimesupport

NewFeaturesin2018

ChangeRejectedFeaturehandlingwitharight-click

Annotationwordwrapsbydefault.BookmarksfilledbydefaultNewWindowsrenderingoptionsforhighDPIdisplaysandwebviews

PlacenewannotationdirecttothecanvasusingCtrl+K

FurtherReading

FormoreinformationonwhatelseisnewandgreatinFME2018,seetheFME2018unveilingwebinar,theFMEDesktopDeepDivewebinar,andtheFMEServerDeepDivewebinar

NewFeaturesin2018

FeatureCaching

InpreviousversionsofFME,itwaspossibletorunaworkspaceusingRunwithFullInspection.

FullInspectionmeantthedatawassavedateachstepoftheworkspaceandsowasavailableforimmediateinspection.

InFME2018thefunctionalityaroundthistoolchanged,andsothenamewaschangedtoo,toRunwithFeatureCaching:

Withthisoptionactive,FMEgeneratescachesateverystepofthetranslationwhentheworkspaceisrun:

Thecachesareindicatedbythesmalliconsoneachobject.

InspectingCachedData

Sometimesit'simportanttobeabletoinspectdataatanystepofthetranslation.AddinganInspectortransformerateverystepwouldbetiresome,soinsteadtheDataCachingoptioncanbeused.

Cacheddatacanbeinspectedbysimplyclickingontheicononaparticularobject.

SavingCachedData

Becauseworkspacescanbeworkedonoverseveralsessions,andbecauseyoumightwanttotransferaworkspacewithcachestoanotheruser,savingaworkspaceasatemplatealsogivestheoptiontosavethedatacaches:

CachingFeatures

Thatwayanyworkspacecreatedfromthetemplateautomaticallygainsaccesstothecachesfromthepreviousrun.

TIPIt'scertainlyquickertosetup"RunwithDataCaching"thantomanuallyaddInspectortransformers.However,beawarethatcachingdataobviouslycausesthetranslationtobeslower,andtousesystemresourcessuchasdiskspace.

Datacachingisveryusefulwhiledevelopingaworkspace,butshouldbeturnedoffbeforeputtingaworkspaceintoproduction.

CachingFeatures

CachedDataStatus

Whenthecacheonatransformerorfeaturetypeisgreen,itmeansthatthecacheisup-to-date;i.e.thecacherepresentsthedatathatwouldbeproducedweretheworkspacetoberunatthatpoint.

Heretheworkspacecreatesadatetimeattributeandtheworkspaceisruntofillthecaches:

However,whentheworkspaceisedited,cachescanchangecolortoyellow.Thismeansthecacheisnolongerup-to-date;i.e.thecachenolongerrepresentsthedatathatwouldbeproducedweretheworkspacetoberun.

HeretheDateTimeStamperhasbeenchangedtocreateadateonlyattribute:

Allcachesafter(downstreamof)theDateTimeStamperturntoyellowtoindicatetheynolongerrepresenttheworkspace.Butcachesbefore(upstreamof)theDateTimeStamperremaingreen,becausetheyarenolongeraffected.

ReusingCachedFeatures

ThemainreasonthefunctionalitychangednamefromFullInspectiontoFeatureCaching,isthatthecacheddatacannowbeusedforpurposesotherthaninspection.

Whenitistimetorunaworkspace,datacanbeautomaticallyretrievedfromgreencachesanduseddownstreamofthatpoint.Forexample,intheaboveworkspacedatadoesnotneedtobereadfromthesourcedatasetbecauseitisalreadyavailableinanup-to-datecache.

Thisisachievedbyclickingonthetransformeryouwishtorun,andusingoneofthetwopop-upactions:

PartialRuns

ThetwooptionsareRuntoThisandRunfromThis.

RunToThisrunstheworkspaceupto,andincluding,thechosentransformer.It'sperfectforwhenthattransformerhasbeenedited,andyouwanttoseewhattheresultthattransformeraloneis.

RunFromThis(shortcutF6)runstheworkspacefromthechosentransformeronwards.It'sperfectforwhenthattransformerhasbeenedited,andyouwanttoseewhatthefinalworkspaceresultswillbe.

Hoveringoveroneoftheseoptionscausesthesectionofworkspacetoberuntohighlightingreen:

HeretheAttributeFiltertransformerisbeingrun,thedatabeingfetchedfromthepriorcache.However,becausethepriorcacheisyellow,andout-of-date,FMEwillinformtheuserthatitmustgobackfurthertoobtainavalidcacheandruntheextratransformersinbetween:

PartialRuns

Iftheauthorwishestore-runtheentireworkspace,theycandothatusingtheusualrunbutton(orF5),whichwillre-runtheworkspaceand(assumingtheFeatureCachingoptionisturnedon)refillallofthecaches.

TIPAJunctiontransformercanplayausefulpartinPartialRuns.Forexample,heretheworkspaceissplitintotwostreams.IfIwishtorunbothstreamssimultaneously,thentheonlycommandwouldbeRunFromThisontheSpatialSorter.

ButthatmeanstheSpatialSorterwillalsoberun.IfIwanttorunbothstreamsbutnottheSpatialSorter,thenaJunctiontransformerintheappropriatelocationwillallowmetodothat.IcanRunFromThisontheJunctiontransformeritself,whichhaslittleoverhead.

WhyReuseCachedData?

Theobviousreasonforusingcacheddataisoneofperformance.Iftheworkspacecanusedatathatwascachedinaprevioustransformer,thenthattransformationdoesnotneedtooccuragain.Whenbuildingaworkspaceonetransformeratatime,theworkspacecanbetestedateachstepandonlythecurrenttransformerneedstobeexecuted.

Similarlythiscanpreventexcessreadingofdata.Ifdataisbeingreadfromadatabaseonanetwork,forexample,thenacacheofdatapreventsFMEfromhavingtomakesubsequentcallstothatdatabase.Itsavesonbothnetworktrafficandquerytime.

Additionally,thisisagreattooltouseforcachingtheresultsofwebservices.Forexample,recordsbeingsenttoawebserviceforgeocodingmightincuranactualmonetarycostforthatservice.ButnowFMEisabletocachethatdataandreuseit,avoidingtheneedtorepeatedlyusethatwebservice.

MsAnalystsays...Obviouslydatacachingtakesnoaccountofwhatmighthavechangedinthesourcedataset.Itonlyknowswhathasbeenchangedintheworkspace.It'suptotheworkspaceauthortobeawareofifthesourcedatasethaschangedandthereforeneedstoberefreshedwithanewrun.

PartialRuns

Exercise CachingandPartialRuns

Data Roads(AutoCADDWG)FirehallsandZones(GML)

OverallGoal CreatetimeanddistanceisolineswiththeNetworkCostCalculator

Demonstrates Datacachingandpartialruns

StartWorkspace None

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Caching-Ex1-Complete.fmwt

Toassistinplanningemergencyservicesforthecity,youhavebeentaskedwithassessingemergencyresponsetimesbycreatingadatasetofisochrones-contourlinesdepictingtraveltimefromeachfirehall-formedicalemergencies.

1)AddFirehallReaderStartFMEWorkbench.SelectReaders>AddReaderfromthemenubar.Settheparametersasfollows:

ReaderFormat GML(GeographyMarkupLanguage)

ReaderDataset C:\FMEData2018\Data\Emergency\FireHallsWithZones.gml

Whenprompted,selecttheFireHallsandMedicZonesfeaturetypestoaddtotheworkspace:

2)AddRoadsReaderOncemoreselectReaders>AddReaderfromthemenubar.Thistimesettheparametersasfollows:

ReaderFormat AutodeskAutoCADDWG/DXF

ReaderDataset C:\FMEData2018\Data\Transportation\CompleteRoads.dwg

Whenprompted,selectallofthefeaturetypestoaddtotheworkspace(i.e.wewantallroadtypes)leavingoutonlylayer"0".

3)RunWorkspaceEnsurefeaturecachingisturnedon(Run>RunwithFeatureCaching)andruntheworkspace.Thedatawillbereadandcached.Thismeans:

Ourpre-translationinspectioncanbedonehereWedon'thavetoreaditseparatelyintheDataInspector

ThedataisnowreadyforusebylaterstagesoftheworkspaceWhenwerunthewholeworkspace,FMEwon'thavetore-readthedata

Selectallofthefeaturetypesintheworkspace,right-clickoneofthem,andchooseInspectCachedFeatures(shortcutCtrl+I):

Exercise:EmergencyResponseProject

Nowwecaninspectthedatatoseewhatwearedealingwith.

4)AddTesterForthisexercisewe'llworkonasinglefirehall,FireHallnumber4.SoaddaTestertransformerconnectedtotheFireHallsfeaturetype.Setituptofilteroutotherfirehalls.Thesimplestwayistocheckfor(andkeep)featureswhoseNameattributeendsinthenumber4:

ClickontheTesterandpresstheRunFromThisbutton(shortcutF6).TheTestertransformerwillrunusingthecacheofdatafromthefeaturetypetowhichitisconnected.

ClickonthegreencacheiconforthePassedoutputport.TheDataInspectorwillopenandshowasinglefirehallfeature.Itshouldbehallnumber4.

5)AddNetworkCostCalculatorNowwe'lladdaNetworkCostCalculatortransformer.ThebestwayofdoingthisistoselectalloftheroadfeaturetypesandthenuseQuickAddbytypingNCCandselectingthattransformer.Thiswaythetransformerwillbeaddedandallroadfeaturetypesconnectedautomatically.

ThenconnectTester:PassedtotheSourceinputport:

Rearrangetheobjectsasnecessarysoyouavoidoverlappingconnections(ormovetheinputportsup/downinstead).

TIPNoticethateventhoughwehaven'truntheNetworkCostCalculatoryet,wehavefeaturecountsontheconnectionsintoit.They'recoloredgreytoshowthatthishasn'thappened,butthat'swhatthecountswouldbe.

6)SetNetworkCostCalculatorParametersInspecttheNetworkCostCalculatorparameters.SetOutputOptimalCostAsto"Z-Values".SetSnapSourcePointsto"Yes"andsetaSnappingToleranceof100:

TheseparameterswilloutputthecostsasZvaluesoneachnetworkline,andwillsnapthesourcefirehalltothenetworkifitdoesn'tsitexactlyuponit.

RuntheNetworkCostCalculatorbyclickingitandchoosingRunToThis.

TIPBecausewecurrentlyhaveallthedatacached,andtherearenotransformersafterthis,itdoesn'tyetmatterifwechooseRunFromThisorRunToThis.Bothachievethesameresultwiththesameperformance.

InspecttheConnectedoutputcache.Ifyouqueryaroadfeatureyou'llseethatithasZvaluesoneachvertex.TheZvaluerepresentstheroaddistancetothefirehall.

7)AddContourGeneratorTovisualizetheseresults,addaContourGeneratortransformerconnectedtotheNetworkCostCalculator:Connectedoutputport.TheContourGeneratorporttoconnectisPoints/Lines:

OpentheContourGeneratorparametersdialog.SettheSurfaceToleranceto10andtheOutputContourIntervalto50.RuntheContourGeneratortransformerandinspecttheContoursoutputcache:

TheoutputisasetofIsodistancelinesshowingthedistance(byroad)fromthechosenfirehall.

8)AddExpressionEvaluatorTocreateIsochrones(linesofequaltraveltime)weneedtospecifythespeedatwhichavehiclecantravel.Obviouslythisvariesdependingonthetypeofroad,terrain,weather,andintersectioncrossings;butaquicksearchshowsthattheaveragespeedofafiretruckinacityis55kph(35mph).

So,addanExpressionEvaluatortransformertotheworkspace,betweentheAutoCADRoadsfeaturetypesandtheNetworkCostCalculatortransformerLineport.ThequickestwaytodothisistoclickonthegreenLinearrowandtypethenameoftheExpressionEvaluator.Thatwaythetransformerwillbeinsertedinalloftheconnectionssimultaneously:

Noticehowthecaches"downstream"fromthenewtransformerchangetoyellow.Theyarestalebecausetheynolongerrepresenttheresultstheworkspacewouldgenerate.

9)SetUpExpressionEvaluatorInspecttheparametersfortheExpressionEvaluator.SetupanexpressiontocalculateanattributecalledTravelTime.Theexpressionshouldbe:

@Length()/916.666666

@Length()isanFMEfunctionthatmeasuresthelengthofeachroadlink.

916.66666comesfrom(speed(metres-per-km/seconds-per-minute))whichis(55(1000/60))

10)AdjustNetworkCostCalculatorNowinspecttheparametersfortheNetworkCostCalculatoragain.

ChangetheWeightTypeparameterfrom"ByLength"to"ByTwoAttributes".SelectTravelTimeastheattributeforboththeforwardandreverseweight:

11)AdjustContourGeneratorNowinspecttheparametersfortheContourGeneratoragain.

ChangetheSurfaceToleranceto1andtheOutputContourIntervalto1(i.e.eachcontourwillrepresent1minuteoftraveltime).

Nowwemustre-runtheworkspace.Youcanchoosetoeither:

RunFromThisontheExpressionEvaluator

RunToThisontheContourGeneratorRunFromThisontheContourGenerator

Thesamesectionsofworkspacewillhighlightasyouhoveroverthefirsttwooptions.ThethirdoptionwillonlyhighlighttheContourGeneratoritself,andwhenchosenwillwarnthatpriortransformersneedtobeincludedastheydon'tyethavevalidcaches.

InspectthenewContourGeneratoroutputcache:

Theresultwillbeaseriesofisochronesrepresentingtraveltimesforemergencyvehiclesfromfirehallnumber4.

12)SaveasTemplateWe'llcontinuethisexerciseattheendofthenextchapter.Incaseyoucloseandre-opentheworkspaceinthemeantime,we'llsaveitasatemplate.Thatwaywecankeepthecaches.

SoselectSave>SaveAsTemplatefromthemenubar.Whenpromptedenteratemplatename.BesuretochecktheoptiontoIncludeFeatureCaches:

Nowwhenwere-opentheworkspaceusingthistemplate,thedatawillalreadybecachedandreadytouse.

CONGRATULATIONSBycompletingthisexerciseyouhavelearnedhowto:

TurnonFeatureCachingInspectallcacheddatainaworkspaceCarryoutapartialrunusingRunFromHereandRunToHereDecidewhenRunToHereorRunFromHereismostappropriateSaveatemplatefilewithFeatureCachesincluded

CollapsibleBookmarks

Bookmarks

Asweknow,abookmark,likeitsreal-worldnamesake,isameansofputtingamarkerdownforeasyaccess.

WithFMEthebookmarkcoversanareaofworkspacethatisusuallycarryingoutaspecifictask,soausercanpickitoutofalargersetoftransformersandmovetoitwithrelativeease.

Asoneuserhasputit,bookmarksarelikeparagraphsforyourworkspace!

Theaboveworkspaceillustratesnicelyhowtomarkupdifferentsectionsofaworkspaceusingbookmarks.Asyoucansee,it'spermittedtosubdividebookmarksfurtherbynestingonebookmarkinsideanother.

ButinFME2018thereisfunctionalitytotakethisonestepfurther...

CollapsingBookmarks

InFME2018,eachbookmarkhasasmalliconinthetop-leftcornerthatallowsittobecollapsed:

Collapsingabookmarkmeansitiscompresseddowntothesizeofasingletransformer,displayingnoneofthecontentsexceptforwheredataentersorexitsthebookmark:

CollapsingBookmarks

Clickingtheiconasecondtimere-opensthebookmarktoitsprevioussize.

Thisfunctionalityallowslargesectionsofworkspacetoberenderedinamuchsmallerarea,andonlyopenedupwheneditingisrequired.

Forexamplethesectionofworkspacedisplayedabovemightbereducedtothis:

Thisisobviouslyahugesavingofspaceonalargecanvas.

CollapsingBookmarksvsCustomTransformers

It'seasytoseecollapsingbookmarksasareplacementforcustomtransformers,especiallywherethecustomtransformerwasintendedonlyasameanstosavespaceoncanvas:

However,collapsiblebookmarksarenotsuchagoodreplacementwherethecontentisintendedtobereusedinmultiplelocations.That'sbecauseeachcustomtransformerinheritsitsbehaviourfromamastercustomtransformerdefinition.Editingthedefinitionupdatesallthecustomtransformers.

Thesameisnottrueofacollapsedbookmark.Thereisnomasterdefinitionandeditingthecontentsofabookmarkdoesnotautomaticallyupdatethecontentsinanyotherbookmarks,evenifitwasadirectcopy.

Additionally,acollapsedbookmarkcannotbesharedwithotherusersanddoesn'thavetheabilitytosetpublishedparameters,inthesamewaythatacustomtransformerwould.

Sowhileacollapsedbookmarkcanreplacesomeaspectsofacustomtransformer,itcan'tfullyreplacethem.

CollapsingBookmarks

CollapsedBookmarkStyle

Withnewcollapsiblebookmarks,thereareaspectsofBestPracticeandstylethatneedtobeconsidered.

RenamingPorts

Theinputandoutputportsoncollapsedbookmarkscanberenamedtohelpclarifythedataenteringandexiting:

It'snotobviousthatthiscapabilityexists,butinfactitcanbeaccessedinthreedifferentways!

Double-clickingtheobjectSelectingtheobjectandpressingF2UsingtheRenameoptiononthecontextmenu(aspicturedabove)

Theobviousbenefithereisoneofclarity.Byknowingwhateachinputandoutputportisfor,extraconnectionscanbemadetoitwithouthavingtoexpandthebookmarkcontents.

RearrangingBookmarks

Althoughyoucancollapseonebookmarkatatime,anoptionontheNavigatorwindowallowsyoutocollapseallbookmarksatonce:

Doingsoillustrateshowmuchspacecanbesavedinasinglecanvas:

CollapsibleBookmarkBestPractice

Ofcourseithelpstothenmovethebookmarksclosertogethertoreusethatspace:

Butbesidesreusingthespace,thereisanotherreasonforthisreorganization.

Whenbookmarksarepositionedclosetoeachother,re-openingacollapsedbookmarkadjuststhelayoutoftheworkspace,movingothertransformersorbookmarksoutofthewaysothatitscontentsareshownwithoutoverlap.Re-closingthebookmarkcausestheoppositetooccur.

Forexample,intheabovescreenshot:

...ifanybookmarkisexpanded,thenthebookmarksarounditaremovedtoonesidetoaccommodatethenewspacerequirements.Whenthebookmarkiscollapsedagain,thereversetakesplace,togivethesamecompactlayoutasbefore.

CollapsedBookmarksandCaches

Collapsiblebookmarkshavebeendesignedverycarefullysothattheyinteractwithcachesandpartialruns.

RunningaBookmark

Thepop-upoptionsforapartialrun,aswellasappearingontransformers,arealsopresentonbookmarks:

TheoptionsarecalledRunToContainedandRunFromContained,andappearwhetherthebookmarkiscollapsedorexpanded.

Thisobviouslymakesiteasiertorunasectionofworkspace,particularlywherethebookmarkiscollapsed.

ImprovingPerformance

Whenanauthorcreatesaverylargeworkspace,asisusuallythecasewhentherearemanybookmarks,runningwithfeaturecachingturnedoncouldcreatealargeamountofunwantedcaches:

NoticeintheabovescreenshotthateverytransformerinthePrepareDataforMatchingbookmarkisbeingcached.

Toavoidthis,featurecachinghasbeendesignedtoavoidcachingdataforeverytransformerinsideacollapsedbookmark.Whenaworkspaceisrunalltransformersinthecollapsedbookmarkoperateasusual,buttheonlydatacachedisfortransformersrepresentedbyanoutputportonthebookmark:

Wecanseethatbyexpandingthebookmarkaftertheworkspacehasfinishedrunning:

CollapsibleBookmarksandCaches

Thissavesconsiderabletimeandresourceswhenworkingwithalargeworkspace.

Dr.Workbenchsays...Obviouslyyoudon'twanttoputaworkspaceintoproductionwhencachingisturnedon,regardlessofwhetheryourbookmarksarecollapsed.Thistechniqueisonlyrecommendedforuseinthedesign,authoring,andtestingphasesofworkspacecreation.

CollapsibleBookmarksandCaches

Exercise CachingandPartialRuns

Data Roads(AutoCADDWG)FirehallsandZones(GML)

OverallGoal CreatetimeanddistanceisolineswiththeNetworkCostCalculator

Demonstrates CollapsibleBookmarks

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Begin.fmwt

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Complete.fmwt

Herewe'llcontinuetheexercisefromtheCachingchapter.Ourgoalistoassistinplanningemergencyservicesbycreatingadatasetofisochrones-contourlinesdepictingtraveltimefromeachfirehall-formedicalemergencies.

1)StartWorkbenchStartFMEWorkbenchand,ifnecessary,openthetemplatefileforthepreviousexercise.Youcanfindacopyofitat:C:\FMEData2018\Workspaces\UpgradingTo2018\Bookmarks-Ex1-Begin.fmwt

Theworkspaceworksfine,butitneedsacoupleofadjustments,plusitisindireneedofbestpracticestyle.

2)AddBookmarkThefirstthingwecandoismaketheworkspacecanvastidierbybookmarkingalltheroadfeaturetypesandcollapsingitintoasmallerspace.

So,selectallofthefeaturetypesfortheAutoCADroadsreader.PressCtrl+B(orselectInsertBookmarkfromthetoolbar).NamethebookmarkAutoCADRoadsReader:

Exercise:EmergencyReponseProject

Ifwecollapsethebookmarknow(tryitandsee)we'llstillgetmultipleoutputsandconnections.Wedon'treallyneedthis.SoaddasingleJunctiontransformerbetweenalltheconnectionsandtheExpressionEvaluatorandresizethebookmark(expandingitfirstifnecessary)tofittheJunction:

Collapsethebookmarkandnowtherewillbejustasingleoutputport.Double-clicktheporttorenameit.NameitRoads:

Thishasobviouslysavedquitealotofspaceinthecanvas.

3)AddClipperToreturntothetaskathand,thecurrentworkspaceoutputcoversallofthecityofVancouver,whereasweonlyneedittoshowcontoursfortheareaofresponsibilityforfirehallnumber4.

ToclipthedataaddaClippertransformertotheworkspace.ConnectthecontourstotheClippeeinputportandtheMedicZonefeaturetypetotheClipperport:

Rearrangetheobjects/portstoensuretherearenooverlappingconnections.

4)DuplicateTesterTheMedicZonesdatacoversallofthecitytoo,andweonlyneedfirehallnumber4.WecanfilterthiswithaTester;infactwiththesametestastheexistingTester.

So,selecttheexistingTesterandpressCtrl+D(RightClick>Duplicate)toduplicateit.DroptheduplicateTesterintotheMedicZones-Clipperconnection:

Butdon'truntheworkspaceyet...

5)AddBookmarkIfweruntheworkspacenowthereareanumberofyellowcachesthatwillbeupdatedandsaved.However,wedon'treallyneedthese.Cachesareusefulfortestingasectionofworkspace,butwealreadyknowthemainpartofthisworkspaceiscorrect.

So,addabookmarkaroundthefourmainprocessingtransformers(ExpressionEvaluator,Tester,NetworkCostCalculator,ContourGenerator):

Nowcollapsethebookmark,renameitsports,andrearrangecontentaroundit:

Noticethatthereisnowonlyasinglecacheforthewholebookmark.

6)RunWorkspaceRe-runtheworkspacebyselectingtheClippertransformerandchoosingRunToThis.

Wearenowsavingcachespacebecauseonlyonecacheiscreatedforthebookmarkedparts.Expand(andre-collapse)theIsochronebookmarktoprovethis.TheContourGeneratorhascachesbecauseitisthelasttransformer.TheTesterhascaches,butonlybecauseitalreadyhadthem:

Noticethatasyouexpand/collapsethebookmark,otherobjectsmoveinresponsetokeepfromoverlapping.

NowinspecttheClipper:Insidecache.Youshouldseethattheisochronesareclippedtotheareaofresponsibilityforfirehall4.

7)SaveWorkspaceRe-savetheworkspaceasatemplate-includingcaches-soitisreadyforfurthereditingifrequired.

TIPToseesomeadvancedtasksforthisproject,checkouttheGISTutorialontheFMEKnowledgeCentre.

AddandcollapsebookmarksAddaJunctiontoreducethenumberofbookmarkoutputportsRenamebookmarkoutputportsRearrangeobjectsaroundabookmarksothattheymovewhenthebookmarkiscollapsed/expandedCollapseabookmarktosavecachingexcessdata

PrecisionandTolerance

ToleranceandPrecision

ToleranceParameters

FME2018introducesatoleranceparametertomanyexistingtransformers,andonenewtransformer:

AreaGapAndOverlapCleanerAreaOnAreaOverlayerClipperDissolverFeatureMergerGeneralizerIntersectorLineOnAreaOverlayerLineOnLineOverlayerSnapperTilerTopologyBuilder

TheAreaGapAndOverlapCleaneris,ofcourse,thenewtransformer.It'sadirectreplacementfortheSliverRemover.TheSnapperalreadyhadatoleranceparameter,butthathasbeenupdatedtousethisnewfunctionality.

Sowhythesetransformers?Becauseit'sasetoftransformersthatuseacommonpieceofcode.Thatcodewasupdatedtoallowatolerancetobeappliedandwasexposedasanewparameter:

Thetoleranceparametershavethreedifferentoptions:

NoneAutomaticCustomValue

NonemeansthetransformeroperatesasitalwaysdidinFME;withoutanytoleranceapplied.Theoperationwillbecarriedoutusingstrictmatchesofcoordinates.Thisisthedefaultvalue(for2018.0atleast).

AutomaticmeansFMEwilldecideuponatolerance.HereFMEistryingtoresolveminorissueswithcoordinates,suchascoordinateprecisionandrounding.

CustomValuemeanstheuserwillenteratolerance.Theuseristryingtoresolveanyissuestheybelievecanbefixedthroughtolerance,suchasbadgeometry.

CollapsedPorts

SometransformerswiththenewtolerancesettinghaveaCollapsedoutputport.Thisisforfeatureswhosesizeissmallerthanthetolerancevalue,andwhichthereforecollapsetoasinglepoint.

TIPBecausetheAutomaticmodeisnewertechnology,itsperformanceinmanycasesisactuallyfasterthanhavingnotoleranceatall!

ToleranceParameters

ExampleGeometryProblems

Ingeneral,datacannotjoin,meet,orintersectbecauseofoneoftwoissues.

Firstlythegeometryofthedatacanhavesimplybeencreatedbadly:

Inthatexamplethetwolinesaremeanttoconnect,butinsteadhaveagapofaround10metres.

Thesecondsourceofproblemsisgeometrythathasverytinyimperfectionsduetoissuessuchascoordinaterounding:

Inthiscasethetwolinesaremeanttoconnect,butinsteadthereisaverysmallgapof0.00000001metres.Thissortoferrorisverydifficulttolocatevisually,becauseofitssmallsize.

Suchanissuemightarisebecauseofcoordinaterounding(coordinatesareroundedtoaprecisionthatleadstogaps),becauseofdifferentdataformats(whereeachformatstorescoordinatesinslightlydifferentways),orperhapsbecauseofsomeotherautomatedprocessthathasbeencarriedout.

NoTolerance

Whentoleranceissettonone,thentheabovefeaturesarenotconsideredamatch.

Forexample,inaTopologyBuilder,thegapsineachcasewouldleadFMEtoassumethateachgeometryisaroadthatendswithoutaconnection.Therewouldbetwonodeswithagapbetweenthem:

Dependingonthedatathismayormaynotbethecorrectresult.Withagapofonly0.00000001metreswecansafelyassumeitisnotcorrectinthisexample.

ToleranceExamples

AutomaticTolerance

Whentoleranceissettoautomatic,thenFMEwillassessthedataandcomeupwithareasonabletolerancetosolveminorproblems.Forexample,inaTopologyBuilderFMEwouldcorrectlydeterminethatthesmallergapisanerrorthatshouldbecorrected:

However,itwillnotfixthegrossererrorthatcausedthelargertolerancegap:

That'sbecausethereisnowayFMEcouldreasonablydefinethatasaproblem,orfixitwithoutpossiblyalteringothersimilargaps.

CustomTolerance

Whenthetoleranceissettoacustomvaluethentheauthoristryingtoresolveissuesatascaleoftheirownchoosing.Forexample,intheTopologyBuildertheycouldsetatoleranceof10.Thatwouldsolveboththesmallissuesandthelargerone:

Butitmightalsomistakenlyfixissueswheretwopointsarelegitimatelywithin10metresofeachother,likeatNode2.

Soingeneralcasesit'sbesttouseautomatictolerance,unlessyoucanbesurethatacustomtolerancewillworkasyouwantitto.

ToleranceExamples

Exercise ToleranceandDataCleaningData PropertyBoundaries(MicroStationDGN)

OverallGoal Cleanlinedatainordertobuildpropertypolygons

Demonstrates Intersection,DataCleaning,andTolerance

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Tolerance-Ex1-Begin.fmwt

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Tolerance-Ex1-Complete.fmwt

Forasmallexercise,you'vebeenaskedtoturnasetoflinework(representingpropertyboundaries)intopolygonfeatures.ThisiseasilydonewiththeAreaBuilder.However...theremaybesomedatacleaningrequirementsthataren'tassimple.

1)StartWorkbenchStartFMEWorkbenchandopenthestartingworkspacetemplate.

TheworkspaceincludesareadertoreadasetofMicroStationDGNfeatures.ItalsoincludesabookmarkedsetoftransformersthatwerepreviouslyusedtocleandatainFME2017:

It'ssooldthattheyevenincludeInspectortransformers!Howquaint!

2)AddAreaBuilderFirstlylet'signorethatbookmarkedsectionandsimplyaddanAreaBuildertransformerconnectedtothereaderfeaturetype(ParcelLines):

Turnonfeaturecachingandrunthatpartoftheworkspace.InspectbothAreaBuildercaches:

Exercise:ToleranceandDataCleaning

Noticethatthereareseveral"unused"features.Whereafeatureisunuseditdidnotcreatetheproperboundaryandtheareacoverstwo(ormore)properties.

3)MoveAreaBuilderTransformerWenowknowthedatawon'tbuildproperpolygonswithoutsomecleaning.SodisconnecttheAreaBuildertransformerandmoveittotheendoftheworkspace,connectedinplaceoftheCleanNetworkInspectorfromthebookmark:

DeleteanyotherInspectortransformerstoo,connectthesourcedatatotheSnappertransformerinthebookmark,andruntheworkspace.

Inspecttheoutput.TherewillbeperfectlyformedpolygonsandnoIncompletefeatures.

Sowe'vefinishedtheproject.Except...surelythereisabetterwayin2018?

4)DeleteBookmarkandTransformersLet'sstartoutbydeletingthe2017bookmarkandallitscontents,plusthenewAreaBuildertransformer,sothatallwe'releftwithisthereaderfeaturetype.That'sjusttocleartheworkspaceofclutter.

5)AddIntersectorandAreaBuildertransformersPlaceanIntersectortransformer(ithasthenewToleranceparameterandsomayhelpus)plusanewAreaBuildertransformer:

InspecttheIntersector'sparameters.MakesurethetoleranceparameterissettoAutomaticandre-runtheworkspace.Inspecttheoutput:

TherearesomeIncompletefeaturesbutnotasmanybadareas.TheIntersectorhasobviouslydonesomegood.However,thetolerancerequiredisjusttoomuchforFMEtohandleautomatically.

6)SetManualToleranceInspecttheIntersectorparametersagainandthistimesettheToleranceparametertoacustomvalueof0.2.Re-runtheworkspace.Theresultsarebetter,butnotquitecomplete.

Setthetoleranceparameterto0.3andtryagain.

Thistimetheareafeaturesbuildcorrectly.AnyIncompletefeaturesaresmalloverlapsandcanbeignored.

Sowehavereplicatedtheresultsofthe2017workspacebyreplacingallofthebookmarkcontentswithasimpleIntersectortransformer!Infacttheresultsarebetterbecausethe2017workspaceproducedsomeodditiesthatshouldn'texistinthedata(that'swhyithadmoreareasoutputthanin2018).

ApplyanAutomatictoleranceontheIntersectortransformerApplyaCustomtoleranceontheIntersectortransformer

MicrosoftFormatsandTransformers

MicrosoftFormats

MicrosoftFormatsandTransformers

FME2018introducesanewsetoftransformersandformatsforMicrosoft-relatedformats.Theseare:

Reader Writer Transformer

Excel Existing Existing New:ExcelStyler

Word None New New:MSWordStyler

PowerPoint None New New:PowerPointStyler

Inshort,FME2018hasnewwritersforWordandPowerPoint,andnewstylingtransformersforallthreeformats.AnExcelreaderandwriterhasexistedinFMEforalongtime,butthewriterhasnewformattingoptionsthatcanbefulfilledusingtheExcelStyler.

FormatsandTransformers

MicrosoftWord

TheMicrosoftWordwriterinFMEallowswritingfeaturestoWorddocuments.Thewriteronlyhasasingleparameter,thattodefineaWordtemplatetouse.

ThedatasenttothewritermustalsofirstbestyledusingtheMSWordStylertransformer.

MSWordStyler

TheMSWordStylertransformerallowsthecreationofvariouscontenttypes:

Thiswayadocumentcanbebuiltupfromseveralcomponents,withheadings,images,lists,tables,andplaintext(paragraphs).Asadifferentcontenttypeisselected,theavailableparameterschangeinthedialog.Forexample,forTablecontenttheparameterslooklikethis:

...withtheoutputofthatlookinglikethis:

MicrosoftWord

MapfeaturescannotbewrittendirectlytotheMicrosoftWordwriter,butcanbewrittenasanimagebyfirstrasterizingthedatawithanImageRasterizerorMapnikRasterizertransformer.

WriteOrder

Asyoumightexpect,featuresarewrittentotheWorddocumentintheordertheyarrive,soaworkspaceauthorcancontrolthedocumentbycontrollingtheorderoffeaturesusingaSortertransformer.

MicrosoftWord

MicrosoftPowerPoint

TheMicrosoftPowerPointwriterinFMEallowswritingfeaturestoPowerPointpresentations.LiketheWordformat,thePowerPointwriteronlyhasasingleparameter,thattodefineatemplatetouse.

Aseparateslideinthepresentationiscreatedforeachfeature.

ThedatasenttothewritermustalsofirstbestyledusingthePowerPointStylertransformer.

PowerPointStyler

ThePowerPointStylertransformerallowsthecreationofvariousslidetypes:

Thiswayapresentationcanbebuiltupfromseveralcomponents,withcharts,maps,images,tables,andotherplaintextcontent.Asadifferentcontenttypeisselected,theavailableparameterschangeinthedialog.Forexample,forTablecontenttheparameterslooklikethis:

...withtheoutputofthatlookinglikethis:

UnliketheWordwriter,MapfeaturescanbewrittendirectlytotheMicrosoftPowerPointwriter.Featurescanbegroupedtogetherinoneslideandeachfeaturebecomesavectorobjectinthatslide.

ThePowerPointStylertransformeralsohasaparametertocontroltheorderoffeaturesbeingwritten.IfthisisleftasAutomatic,thenallthefeaturesgoingtotheMicrosoftPowerPointWriterwillbewrittentoapresentationinthesameordertheyarrive,meaningtheslideswillbeinthatorder.

MicrosoftPowerPoint

SettingthisparameteronlymakessensewherethereismorethanonePowerPointStylertransformer.Ifanumberisenteredintothisfield,thenthefeaturesaresortedbythatnumberonwriting.Forexample,atitleslidecouldbegiventhenumber1toallowittobefirstinthepresentation.

Ifmorethanonefeaturehasthesameslideorder,thentheyarewritteninthesameordertheyarrive.

TIPThetemplatefileforaPowerPointcanbedesignedintheMasterslideviewinPowerPoint.Forexample,settingabackgroundimageinthetemplatecouldleadFMEtoproduceamapslidelikethis:

ParkImagebyOCParks,CCBY-ND2.0

Noticehowamapslideallowsthecreationoflabelsoneachfeature(alsotosetthefeaturecolor)andthecreationofa"caption"textboxintheupper-left.

MicrosoftPowerPoint

MicrosoftExcel

FMEhashadaMicrosoftExcelreaderandwriterforanumberofyears.However,newin2018thewriterhasgainedanumberofnewformattingoptions,andtheseoptionscanbecontrolledwiththenewExcelStylertransformer.

ExcelStyler

TheExcelStylertransformerallowsstylingtobeappliedtoeitherindividualcellsortoentirerows:

Itallowssettingoffont,fill,border,andtextalignmentstyles:

Asshownabove,parameterscanbesetbyattributes;sofeaturescanbecoloredonthebasisofanattributeandthatattributeusedtosetthecolorintheExcelStyler,givingsomethinglikethis:

MicrosoftExcel

...wherecolorisappliedonthebasisofastreetnameattribute.

TheExcelStyleralsoallowsforsettingcellformat(fornumbers)andcellprotection(whetheritishiddenorlocked).

MicrosoftExcel

Exercise ThePowerPointWriter

Data Libraries(EsriGeodatabase)LandBoundary(EsriShapefile)

OverallGoal CreateaPowerPointpresentationshowingvariousitemsofinterestinthecityofVancouver

Demonstrates Microsoft-relatedtransformersandformats

StartWorkspace None

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\Microsoft-Ex1-Complete.fmw

Aspartofaconferenceoncommunitymapping,youhavebeenaskedtoprovideapresentationondifferentaspectsofcommunityinthecityofVancouver.YourealizethisisagreatchancetobothcarryoutthetaskanddemonstrateFME'spotentialforsuchprojects.

We'llbeginthetaskbycreatingaslideaboutlibraries.Sincewe'venotusedthisformatortransformerbefore,thiswillbeajointvoyageofexploration!

NB:Ifyoudon'thaveaPowerPointvieweronyourcomputer,thenyoucanviewfilesonlineonsuchsitesasonlinedocumentviewer.com

1)StartWorkbenchStartFMEWorkbench.Generateaworkspaceasfollows:

ReaderFormat EsriGeodatabase(FileGeodbOpenAPI)

ReaderDataset C:\FMEData2018\Data\CommunityMapping\CommunityMap.gdb

ReaderParameters TableList:Libraries

WriterFormat MicrosoftPowerPoint

WriterDataset C:\FMEData2018\Output\Training\VancouverCommunity.pptx

WriterParameters Template:C:\FMEData2018\Resources\DesktopBasic\VancouverTemplate.pptx

2)AddPowerPointStylerTransformerIfweranthetranslationnow,wewouldn'tgetanyslidesintheoutput.That'sbecauseallthefeaturesneedtobestylizedforthePowerPointwriterfirst.So,addaPowerPointStylertransformerbetweenthereaderandwriterfeaturetype.

Inspectthetransformerparameters.Set:

SlideType:MapSlideSlideSubtype:MapWithCaptionTitle:VancouverLibrariesCaption:AMapofLibrariesinVancouver

Nextsetthelabelparameterstolabeleachobjectwiththelibraryname.UseafontandcolorofyourchoosingandmakesureWrapLabelTextissettoNo:

Exercise:VancouverCommunityMappingProject

ClickOKtoclosethedialogandacceptthechanges.

TIPThePowerPointStyleralsohasaparameterforHyperlink.Ifyoulike,settheHyperlinkparametertopointtotheattributeLibraryURL.ThatwayalltheoutputfeatureswillhavehyperlinksinthePowerPointfile.

3)Add2DEllipseReplacerTransformerApointfeaturewon'tshowupverywellinaPowerPointslide,solet'sturnthelibrariesintosomethinglarger.

Adda2DEllipseReplacertransformerbeforethePowerPointStyler.Setituptocreateanellipseofsize200x200:

Acceptthechanges,turnonfeaturecaching(ifnotalready)andruntheworkspace.Theresultwillbeaseriesofslides,eachwithonelibrary:

We'dpreferasingleslidewithalllibraries,sowe'llhavetochangethat.

4)AddAttributeCreatorTransformerThePowerPointStylertransformerhasanoptiontogroupobjects,butweneedanattributetogroupthemby.SoplaceanAttributeCreatortransformerbetweenthe2DEllipseCreatorandthePowerPointStyler(it'simportantforittogohereforalaterstep).

Usethetransformertocreateanattribute(callitGroup)withafixedvalue(say,1).

NowinspectthePowerPointStylerparametersagain.Nearthebottomofthedialog,underFeatureHandling,setCreateSlideFromtoMultipleFeatures,andsetSlideGroupIDtotheGroupattribute:

Nowre-runtheworkspace.Theoutputshouldbeasingleslidewithallthelibrariesonit:

5)AddShapefileReaderThemapiscorrect,butitreallyneedsabackgroundtothelibraryfeatures.We'llcreatethatwithadatasetoftheVancouverlandboundary.SoselectReader>AddReaderfromthemenubarandaddthefollowing:

ReaderFormat EsriShapefile

ReaderDataset C:\FMEData2018\Data\Boundaries\LandBoundary\VancouverLandBoundary.shp

ConnectthenewlyaddedfeaturetypetotheAttributeCreatorinputport:

WeconnectittotheAttributeCreatorportsothatitgetsthesameattributevalueandappearsonthesameslideasthelibraryfeatures.

TIPIf(instep4)youweretemptedtouseanexistingattributeratherthancreateanewone,thisiswhywedidn't.Wewantsomethingthatwillworkformultipledatasets.It'salsowhyitwasimportanttoputtheAttributeCreatorafterthe2DEllipseReplacer,becausewedon'twanttoturnotherdataintoellipsestoo,justthelibraries.

6)AddSorterIfweruntheworkspacenowyouwillsee,however,thatallisnotwell.Thelandboundaryfeatureoverlaysthelibraries.Tohandlethiswemustsortthedatainsuchasorderthatthelibrariesarrivefirst.ThiswecandowithaSortertransformer.

Wecouldalsocreateattributesforeachdatatype,butwe'llavoidthatbyreusingtheformatattributefme_feature_type.

Soopentheparametersdialogforafeaturetype(itdoesn'tmatterwhich),clicktheFormatAttributestab,andputacheck-markagainstfme_feature_type:

ClickOKtoclosethisandputaSorterintotheworkspacejustbeforethePowerPointStyler.Setituptosortbyfme_feature_type,alphabetic,descending:

Nowwhentheworkspaceisrun,weshouldgetlibraryfeaturesabovethecityboundary.

7)AddFeatureColorSetterThemapisnowvisible,butitwouldbebetterifwecouldgetastandardcolorforeachlibraryobject.WecansetcolorinsidethePowerPointStyler,butthatwouldcauseallfeaturestobethesamecolor,includingthecityboundary.

So,placeaFeatureColorSettertransformerbeforetheSortertransformer.Openitsparametersandsetituptocreaterandomcoloring,butbasedontheattributefme_feature_type:

Thismeansthateachdifferentfeaturetypewillgetitsowncolor.InthePowerPointStylertransformerlocatetheForegroundFillColorparameterandsetittofme_fill_color:

TIPThistechniqueislikecreatingagroup-bywithcolor.It'salsoagreattechniquetousefortheExcelStylertransformer,whereyouwanttocreatedifferentcolorsperrow,butbasedonanattributevalue.

8)AddTitleS lideOnelastthing:wereallyneedatitleslideforthepresentation.Thisiseasilyachieved.PlaceaCreatorontothecanvas,connectittoasecondPowerPointStyler,andconnectthattotheoutput:

OpentheparametersdialogforthesecondPowerPointStyler.Setitasfollows:

SlideType:TitleSlideTitle:TheVancouverCommunitySubtitle:AseriesofFME-generatedmapsandchartsSlideOrder:1

TheSlideOrderparameterisimportantbecausewewantthistobethefirstslideinthepresentation.Yes,wecouldrelyonthisfeaturebeingwrittenfirst,butthat'shardertocontrol.AlsovisitthefirstPowerPointStylertransformerandsetitsSlideOrderto2,toensureitcomessecondinthepresentation.

Re-runtheworkspaceandchecktheoutput.Youshouldnowhavetwoslides(onetitleslide,oneforlibraries),withallslidesandfeaturesinthecorrectorderandwithuniquecolorsper"layer":

ThisformsagreatbasisforyourpresentationonthecommunitiesofVancouver!YoucouldevenstartexperimentingwithusingtheBookCountandCirculationattributesasthesizeoffeatureinthe2DEllipseReplacer,tocreateathematicmap.

CreatemapswiththePowerPointStylerReplacepointfeatureswithellipsesusingthe2DEllipseReplacerCreateslidegroupsformultiplefeaturesperPowerPointslideSortfeaturesintothecorrectorderforvisualizingonaslideSetfeaturecolorsonarandomgroup-bybasisSortslidesintothecorrectorderinapresentation

FeatureJoinerTransformer

FeatureJoiner

WhatistheFeatureJoiner?

TheFeatureJoinertransformerisnewforFME2018.It'saSQL-basedalternativetotheFeatureMergertransformer,thatjoinsdatatogetheronthebasisofcommonkeysandusingSQLterminologytodefinetheresults.

Onthecanvasitlookslikethis:

Andtheparametersdialoglookslikethis:

JoinModes

JoinedfeaturesexitthetransformerthroughtheJoinedport.WhathappenstotheremainingfeaturesiscontrolledbytheJoinModeparameter.

Therearethreejoinmodes:Inner,Left,andFull.

InInnermodefeatureswithoutajoinexitthrougheithertheUnjoinedLeftorUnjoinedRightport.

InLeftmode,LeftfeatureswithoutamatchstillexitthroughtheJoinedport.RightfeatureswithoutamatchexittheUnjoinedRightport.

InFullmode,bothLeftandRightfeatureswithoutamatchstillexitthroughtheJoinedport.

Wecanvisualizethisbehaviourwithadiagram:

Mode Description Depiction JoinedOutput UnjoinedLeft UnjoinedRight

Leftfeatureslookforamatchandareoutputwhethertheyfindamatchor

AllmatchesplusunmatchedLeftfeatures

None UnusedRightfeatures

FeatureJoinerModes

Leftfeatureslookforamatchandareoutputiftheyfindone

Allmatchesonly UnmatchedLeftfeatures

UnusedRightfeatures

BothLeftandRightfeaturesoutputthroughtheJoinedoutputport,whethertheyfindajoinornot

AllmatchesplusunmatchedLeftandRightfeatures

None None

FeatureJoinerModes

MultipleJoinMatches

EverytimeafeatureontheLeftmatchesoneontheRight,anewfeatureisoutputfromtheJoinedport.ThismeansthatwheretherearemultiplematchesforasingleLeft(Requestor)feature,therewillbemultipleoutputfeatures.

ThisiswhytheFeatureJoinerhasnoequivalenttotheFeatureMerger“handleduplicatesuppliers”parameter.Itdoesnotneedthis.ItjustcreatesanewJoinedfeatureforeachduplicatesupplier.

Tounderstandthis,let’stakealookatthiswithsomefeaturecountnumbersincluded:

Thereareeight(8)facilityfeaturesand1,463,442addresses.HowmanyfeatureswillexittheJoinedport?Well,itdepends.

Ifweassumeastrict1:1matchbetweeneachFacilityandanAddress,then8featureswillexitasJoined.Why?Becausewehave8matches.Dependingonthemodethecountwillbeoneofthese:

Joined UnjoinedLeft UnjoinedRight

Left 8 0 1,463,434

Inner 8 0 1,463,434

Full 1,463,442 0 0

However,wemightnothavesuchaclean1:1matchbetweenFacilityandAddresstable.Wecouldhave1:M,M:1,orevenM:M.

Let’ssayeachFacilityhastwomatchesintheAddresstable.Thenwegetthis:

Joined UnjoinedLeft UnjoinedRight

Left 16 0 1,463,426

Inner 16 0 1,463,426

Full 1,463,442 0 0

WhyarewegettingmorefeaturesoutthanweputintotheLeftport?Becausewearegettingonefeaturepermatch,andthereare16matches!

ThisisnotsomethingtheFeatureMergerwoulddo.Itwouldeitherignorethesecondmatchorcreatealist.ButthisisdesignedtobeaSQLequivalent,andsoitworksdifferently.Infact–inanextremecase–ifweassumedthateveryFacilityhasAddressID=1,andeveryAddresshasAddressID=1,theneveryfacilitywouldmatchtoeveryaddress!

Thatwouldgiveus8x1,463,442=11,707,536matches

IfyoudidwantthesameastheFeatureMergerwouldgive-onematchedrecordwithalistofmatches-thenyousimplyneedtocreatealistwithaListBuildergroupingbyyourjoinkeys.

MultipleJoinMatches

Exercise TheFeatureJoiner

Data Addresses(EsriGeodatabase)CrimeStatistics(MicrosoftExcel)

OverallGoal Joincrimestatisticstoeachaddressusingstreetnameandblocknumber

Demonstrates TheFeatureJoinertransformer

StartWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\FeatureJoiner-Ex1-Begin.fmw

EndWorkspace C:\FMEData2018\Workspaces\UpgradingTo2018\FeatureJoiner-Ex1-Complete.fmw

Yourtaskfortodayistomakeajoinbetweenaddresspointsandadatasetofcrimestatistics.Anexistingworkspacehasalreadybeensetuptopreparethedatabystandardizingtheattributesrequiredtomakeajoinkey.

1)OpenWorkspaceOpenthebeginningworkspacefortheexercise,FeatureJoiner-Ex1-Begin.fmw

Inspecttheworkspacecontents:

NoticethattheStringReplacerandtheStringCaseChangerinparticulararesetuptoadjustthePostalAddressattributes(200HighStreet)insuchawayastomatchthoseinthecrimedata(2XXHIGHSTREET).

AlsonoticethatthecrimedataisbeingreaddirectlyfromtheCityofVancouverwebsite.Weprobablywon'twanttoreadthatmoretimesthanisnecessary,sowillmakeuseofcaches.

TIPIf,forwhateverreason,youcan'taccessthedataontheCityofVancouverwebsite,changethereaderparameterontheExcelreadertopointto:C:\FMEData2018\Data\Emergency\crime_xlsx_all_years.zip

2)RunWorkspaceBeforeputtingdownanytransformers,turnonfeaturecachingandruntheworkspace.Wecan'tbesurehowlongitwilltaketofillthecaches,andwecancarryonbuildingtheworkspacewhileitruns.

3)PlaceAttributeManagerAFeatureJoinertransformerdoesnotallowuseofatexteditortoconstructajoinkey,thereforeanyjoinkeywewishtomakemustbedoneinanAttributeManager.

So,placeanAttributeManagertransformer,insidethePrepareAddressesbookmark,connectedtotheStringCaseChanger:

Exercise:CrimeAnalysisProject

OpentheAttributeManagerparametersdialog.CreateanewattributecalledJoinKey.Fortheattributevalueclickthedrop-downlistandchoosetoopentheTextEditor.Intheeditorenter:

@Trim(@Value(Number)@Value(Street))

...withaspacecharacterbetweentheNumberandStreetattributes:

4)PlaceFeatureJoinerAddaFeatureJoinertransformer,withthenewAttributeManagerconnectedtotheLeftinputportandtheCrimeDatafeaturetypeconnectedtotheRight:

Opentheparametersdialog.SettheJoinModetoInner,becausewewanttoonlykeepaddresseswherethereisamatchtocrimeonthesameblock.LeavetheothertwoJoinModeparameterstotheirdefault(UseLeft):

NowselectthenewlycreatedJoinKeyattributeastheLeftJoinOnkey.ChoosetheexistingcrimedataattributeHUNDRED_BLOCKastheRightJoinOnkey:

Closethedialog.

5)RunWorkspaceRuntheworkspacebyselectingtheFeatureJoinerandchoosingRunToThis.Besidesnoticinghowfastthetranslationruns,noticethattherewillbe67,896matchedfeatures.That'sbecausetherearemultiplecrimesperblockandsomultiplematchesperaddress.

Inspectthecacheddataandyouwillnoticethatthereisonejoinperfeature,ratherthanalistsuchastheFeatureMergermightcreate.

6)AddAttributeValueMapperThereasonwewanttocarryoutthisjoinistoassesscrimeforeachaddress.We'lldothisbygivingavaluetoeachcrimethatiscommittedandcreatingasumforeachaddress.

So,placeanAttributeValueMappertransformerconnectedtotheFeatureJoiner:Joinedoutputport.OpentheparametersandsettheSourceAttributetoTYPEandtheDestinationAttributetoCrimeValue:

NowclicktheImportbutton.We'llusethistoimportalistofdifferenttypesofcrimefromthecrimedataset.Enter:

ReaderFormat MicrosoftExcel

ReaderDataset C:\FMEData2018\Data\Emergency\crime_xlsx_all_years.zip

ClickNext.Whenprompted(itwilltakeawhiletoscanthedataset)pick2017astheFeatureTypeandchangeImportFromtobeAttributeValues:

NextselectTYPEastheSourceValue(noDestinationValueneedbeselected)andclickImport.FMEwillimportalistofdifferentcrimes.Giveeachcrimeavalue(sayfrom1-25)basedonwhatyouconsidertheseverityofthecrimetobe:

Don'tworry,it'sjustahypotheticalexercise.Youwon'tbemarkedonwhatvaluesyou'veawarded!

7)AddAggregatorNowweneedtogroupalljoinedfeaturesbytheiraddress,andaddtogetherthecrimevalues.AListBuilderandListSummermightbeonesolution,butherewe'lluseanAggregatortransformerbecauseitcandobothactionsinone.

SoplaceanAggregatorconnectedtotheAttributeValueMapper.Opentheparametersdialog.Set:

GroupBy:PSTLADDRESSCountAttribute:NumberOfCrimesAttributestoSum:CrimeValue

IfwewantedtocreatealistofcrimeswecoulddothatwiththeGenerateListoption,butwewon'tdothatfornow:

NowruntheworkspaceusingRunToThisontheAggregator:

TheoutputoftheAggregator(4,271features)plustheUnJoinedLeftoutputontheFeatureJoiner(9,326)shouldequalthenumberofaddressesenteringtheFeatureJoiner(13,597).Itdoes,sowecantellwehavethecorrectnumberoffeatures.

8)AddCenterPointReplacerInspecttheAggregatorcache.Noticethateachfeatureisactuallymadeupofamultipointfeature.Thiswedon'tneed,soaddaCenterPointReplacertransformertotheendoftheworkspacetoreducethistoasinglepoint.

Nowwehaveourdesiredoutput:afeatureperaddresswithacountofthenumberofcrimesinthatblockandasumoftheirseverity.Withthisinformationwecould(ifwewanted)calculatetheaveragecrimeseverityperaddress/block,andmaybeevenmapthisdatawithperhapstheMapnikRasterizertransformer.

Savetheworkspaceasatemplate,includingthefeaturecaches,anditwillbereadyforyoutocontinuetheprojectthenexttimeyoustartFME.

CreateajoinkeyfortheFeatureJoinerUsetheFeatureJoinertransformertojointwosetsoffeaturesGroupmultiplejoinfeaturestogetherwithanAggregatorCreateasinglepointgeometryforgroupedjoins

CourseWrap-Up

AlthoughyourFMEtrainingisnowatanend,thereisagoodsupplyofexpertinformationavailableforfutureassistance.

CourseWrap-Up

ProductInformationandResources

SafeSoftwareWebSite

TheSafeSoftwarewebsiteistheofficialinformationsourceforallthingsFME.ItincludesinformationonFMEproducts,SafeSoftwareservices,FMEsolutions,FMEsupportandSafeSoftwareitself.

SafeSupportTeam

BehindFMEarepassionate,fun,andknowledgeableexperts,readytohelpyousucceed,withasupportteamphilosophybuiltontheprincipleofknowledgetransfer.

YoucanrequestproductsupportthroughaSupportCase(web/email)orusingaLiveChat.

YourLocalPartner

SafeSoftwarehaspartnersandresellersaroundtheworldtoprovideexpertiseandservicesinyourregionandyourlanguage.

YoucanfindalistofofficialpartnersontheSafeSoftwarePartnersPage.

SafeSoftwareBlog

TheSafeSoftwareblogprovidestechnicalinformationandgeneralthoughtsaboutFME,customers'usecases,andspatialdatainteroperability.Itincludesarticles,videos,andpodcasts.

FMEManualsandDocumentation

UsetheHelpfunctioninFMEWorkbenchtoaccesshelpandotherdocumentationforFMEDesktop.Alternatively,lookonourwebsiteundertheKnowledgeCentersection.

CommunityInformationandResources

SafeSoftwareactivelypromotesusersofFMEtobecomepartoftheFMECommunity.

TheFMEKnowledgeCenter

TheFMEKnowledgeCenterisourcommunitywebsite-aone-stopshopforallcommunityresources,plustoolsforbrowsingdocumentationanddownloads.

KnowledgeBase

TheFMEKnowledgeBasecontainsawealthofinformation;includingtips,tricks,examples,andFAQs.TherearesectionsonbothFMEDesktopandFMEServer,witharticlesontopicsfrominstallationandlicensingtothemostadvancedtranslationandtransformationtasks.

Q&AForum

FMEcommunitymemberspostFME-relatedmessages,askquestions,andshareinansweringotherusers’questions.Membersearn"reputation"and"badges"andthereisaleaderboardofthetop-participatingusers.ComeandseehowtheycanhelpwithyourFMEprojects!

IdeasExchange

FMEdevelopmentisverymuchuser-driven.TheIdeasExchangegivesusersthechancetoposttheirideasfornewFMEfunctionality,orimprovementstoexistingfunctionality,andallowseveryonetovoteontheproposedideas.Themorevotesanideagets,themorelikelyitistobeimplemented!

TheFMEChannel

ThisFMEYouTubechannelisforthosedemosthatcanonlybeproperlyappreciatedthroughascreencastormovie.Besidesthisthereareahostofexplanatoryandhelpfulmovies,includingrecordingsofmosttrainingandtutorials.

CommunityInformationandResources

FeedbackandCertificates

Theformatofthistrainingcourseundergoesregularchangespromptedbycommentsandfeedbackfrompreviouscourses.

CourseFeedback

MissVectorsays...There'sonefinalsetofquestions–andthistimeyou'llbetellingmeiftheanswersarecorrectornot!

SafeSoftwaregreatlyvaluesfeedbackfromtrainingcourseattendeesandourfeedbackformisyourchancetotelluswhatyoureallythinkabouthowwellwe’remeetingyourtraininggoals.

Youcanfillinthefeedbackformnow,butyou'llalsoberemindedbyemailshortlyafteryourcourse.SafeSoftware'spartnerswhocarryouttrainingmayaskthatyoufillinaseparateform,butyoucanalsousetheofficialSafeSoftwareformifyouwish.

Certificates

Mr.E.Dict,(AttorneyofFMELaw)says...Inordertoproveyouhavetakenthistrainingcourse,acertificatewillbeemailedautomaticallytoanyonewhowasloggedonforthedurationofSafeSoftwarehostedcourses.

FeedbackandCertificates

ThankYou

ThankyouforattendingthisFMEtrainingcourse.

ThankYou

Table of Contents - Amazon S3...Course Overview The training will introduce new concepts and...

Documents

Review typesetting terminology. Terminology Loose line abcdefghijklmnopqrstuvwxyz

Classroom Research & Research Question. Goals Introduce education research terminology Problem Question Taxonomy of questions Draft a research question

(Terminology Summer School) TSS. Term Net Inforterm TSS Terminology Management Terminology

The Key to Introducing Vocab! An engaging way to introduce key terminology as a starter or plenary!

Terminology standardization, terminology management and ... · Terminology standardization, terminology management and best practices Or... what's going on with terminology? Kara

Chapter 14 Injuries to the Tissues. Objectives Introduce the terminology associated with injury classification Introduce biomechanical forces that cause

TEACH COMMON CORE STANDARDS WITH THE EEI … settlement and early civilizations. ... Introduce a topic clearly, ... changed in California, so did the

RDMSS Project Initiation Document...Gareth Knight First version 1.1 Caroline Lloyd 03/08/2012 Standard of terminology (Management Board changed to Steering Group, Research Office changed

Hyperbolic Pascal triangles - arXiv · PDF filethat in this paper the terminology Pascal’s triangle is used only for Blaise Pascal’s original ... In order to introduce hyperbolic

Classroom Research & Research Question. Goals Introduce terminology Problem Question Taxonomy of questions Draft a research question Determine your research

Bridge Engineering - Yavuz YARDIM · Bridge Engineering -Fall Semester 2015- 2016 Bridge ... 1 To introduce the Terminology and types of bridges. 2 To introduce classical conceptual

Cascading Behavior in Large Blog Graphsjure/pubs/blogs-sdm07.pdfnetworks [21]. 3 Preliminaries In this section we introduce terminology and basic concepts regarding the blogosphere

Evaluation of Foreign Qualifications at SAQA · Evaluation = Verification + Comparison New SAQA Certificate of Evaluation Aligned terminology Changed understanding of recognition

Ethical Hacking Terminology - USALearning · Ethical Hacking Terminology . Table of Contents . Terminology ..... 2

Lecture 1: Course Introduction & Descriptive Statisticsfaculty.nps.edu/rdfricke/Business_Stats/lecture1.pdf · 2 •Introduce professor & course •Define some basic statistics terminology

MULTINATIONAL CORPORATIONS IN INTERNATIONAL LAW · terminology changed into ‘transnational corporation’9 to emphasise the cross-border operation of the respective company and

From differential cryptanalysis to ciphertext-only attacks · Let us introduce some terminology related to differential cryptanalysis. The difference between two bit-strings X and

Combinatorics of determinantal identitieskonvalinka/thesis.pdf · In this section, we describe the way we want to think about determinantal identities, and introduce some basic terminology

Faults, Folds, and Landscapes. Objectives Introduce basic terminology used in describing rock structure Distinguish between types of fault movements and

The - McGill Universitycgm.cs.mcgill.ca/~godfried/teaching/dm-reading-assignments... · We introduce some of the terminology of atonal music theory (pitch classesandpitchclasssets),deﬁnethecommonlyusedmathematicaltrans-formationsthatactonpitchclasssets,discusstheterm“setclass,”andpose