ASQRD Webinar Telematics Data Analysis Dennis L Craggs ... · 11/8/18 ASQRD Telematics Webinar 6 • This mission profile defines launch, rendezvous and docking, docked, decent, landing

ASQRDWebinarTelematicsDataAnalysis

DennisLCraggs,ConsultantQuality,ReliabilityandAnalyticsServices

Author’sBackground

•  FiftyyearsworkexperienceatNASA,TeledyneCAE,FordMotorCompany,andFCAUSA(formerlyChrysler).

•  BachelorofMechanicalEngineering,MastersofEngineeringMechanics,andMastersofOperationsResearch.

•  PresentedandPublishedatAEC,ISSAT,SAE,andASQAutomotiveExcellenceJournal.

•  Written~30articlesonDataAnalysisincludingTelematics,ManufacturingProcessCapability,andStatistics.TheycanbefoundonAccendoreliability.com,ASQAutomotiveExcellence,andLinkedIn.

•  Currentlyconsulting,volunteeringasSAEorganizer,anddevelopingaclassfortheSAE.

11/8/18 ASQRDTelematicsWebinar 2

ProductDevelopment

•  Requirementsareembeddedin–  ProductSpecifications–  RequestsforQuotes–  VerificationTestPlans

•  Commonrequirementscover–  Businessfactorsliketiming,cost,…–  Productfactors

•  Function•  Life•  QualityandReliability•  Otherfactors


ProductEngineersNeedValidatedRequirements

•  Functions

•  Theproductlifeinyears,miles,cycles,…

•  Theambientandin-vehicleoperatingenvironments

•  QualityandReliabilityTargets

•  Determinecustomerlight,typical,heavyusageaspercentiles.

•  Provideinformationtosupportthedesignanddevelopmentprocess


Howarerequirementsdeveloped?

•  Considerthemissionprofile

•  Takeasystemsview

•  Manyinformationsourcesforrequirements

•  Datadrivenapproachfrommonitoredvehicleusage(telematics).


MissionProfileNASAExample


•  Thismissionprofiledefineslaunch,rendezvousanddocking,docked,decent,landingphasesofamission.

•  Thedesignthecomponents,subsystems,andtotalsystemtowithstandthestressesandfunctiontosupportthemission.

•  Amissionprofiledescribeshowthesystemisexpectedtobeused.

•  Brokenintophases•  Veryhighlevelview

MissionProfileAnAutomotiveDutyCycleExample

Vehi

cle

Spe

ed

Time

Drivetowork

Drivetolunch

Drivebacktowork

Drivehome

Off OffOffOff OffEngineState On On On On

Foranautomobile,vehicleoperationdefinesthemissionprofileparameters.VehicleUsageData•  Countsofstops,trips,miles,days•  Runandstoptimedurations•  Vehiclespeeddata

AnalysisApproach•  Developstandardizationmetrics,i.e.

miles/day,trips/day,miles/trip,…•  Determineifdistributionsdescribethe

usagemetrics.•  Determinevehicleaverageusageand

percentiles,e.g.,5th,50th,95th.•  Projectpercentilestodesigntargets

(years,miles,cycles)11/8/18 ASQRDTelematicsWebinar 7

ASystemDesignProcessAnautomotiveexampleofaP-Diagram

DriverInputs•  Pedal/brake•  Steeringwheel•  Keyon/off•  Passenger/cargoloads

Outputs•  OperatingTime/Distance•  Trips•  Vehiclespeeds•  Acceleration•  Turns•  Driver/PassengerComfort•  Enginestarts/stops•  Hiddenparameters

Noise•  Environment(temperature,

humidity,rain,vibration)•  RoadandtrafficConditions

DesignControls•  DesignTargetsandSpecifications•  VerificationTestsandSystem

Modeling


Theoutputsarecountableevents,states,andmeasurements.Engineersfocusontheiritemswhichmaybeasystem,subsystem,orcomponent.

MeetingRequirement•  Requirementsareembeddedinstandardsandcontract

agreementstoverifyparts,systemsandvehicles.

•  Requirementsaredevelopedusingsurveys,priorsimilarproductspecifications,and“Expert”opinion–  Requirementsmaybeincompleteorilldefined,especiallyfornewproductortechnologies.

–  Managementdirectiontocutcost/shortenproductdevelopmenttimemaycompromisetheprocess.

•  Developrequirementsfromfielddata.–  Fieldusagechangeswiththeenvironment,customerdemographics,andmarketsegments.

–  Example:Canadahasmoreextremecold,theMiddleEastGulfcountrieshavemoreextremeheat.

–  Becomecustomeroriented


ANewDataSourceTheVehicleCommunicationsBus


EngineController

TransmissionController

StabilityController

CustomerInformation

WirelessModule

BodyControlModule AirBagModule

EntertainmentSystem

•  ModernAutomobileshavemanyelectronicmodules,basicallycomputers,controllingdifferentfunctions,transmittingandreceivingdata.

•  ACAN(ControllerAreaNetwork)allowselectronicmodulestoshareinformationasmessages:o  Switchpositions(doorswitchopen/closed)o  MultipleStates(transmissiongear;ignitionswitchmodes,PRNDLselector)o  Measurements(vehiclespeed,enginespeed,temperatures,pressures,GPS…)o  Calculatedparameters(enginetorque)

•  CANstandardsmakeiteasytoaddnewmoduleswithnewfunctionality.•  Allshareddataareassigneduniquecodes.

Telematics


DictionaryDefinition:thebranchofinformationtechnologythatdealswiththelong-distancetransmissionofcomputerizedinformation.

Wikipedia:Telematicsisaninterdisciplinaryfieldthatencompassestelecommunications,vehiculartechnologies,roadtransportation,roadsafety,electricalengineering(sensors,instrumentation,wirelesscommunications,etc.),andcomputerscience(multimedia,Internet,etc.).

VehicleTelematics

•  About1000channelreadingsareavailablewhentheCANbusisactive–ignitionkeyoperation,remotestart,vehicleshutdown…

•  Includestate,parametric,andlocationvariables.•  Thedataarereadevery0.02second(50Hz).•  Specialmodulesstorerawtimestampeddatarecords.

–  Shortterm:~1minuterollingstackof50Hzrecordsforfaultdiagnostics.

–  Longterm:1second(1Hz)datastoredasmemoryallows.•  Somemodulescounteventsandmaintainparameterhistograms.•  UsingWi-Fiorcellphones,dataistransmittedandstoredinadatabasefor

analysis.

DataStorageWi-Fior

CellPhone


Usage

•  Currently,aprimaryusageismonitoringvehiclesduringproductdevelopment–  TesttrackvehiclesandRoadtestvehicles–  Whenfaultsaredetectedonthebus,about±30secondsofthebus

recordsareuploadedtoservers.–  Engineersareautomaticallynotifiedandtaskedtodiagnosethefault.

•  Ausageistoanalyzeandprojectvehicleusageoverdesignlife.–  10years,15years…–  100,000;150,000;220,000miles…

•  Canbeusedforcomparisons–  Specialcommercialfleets(police,taxi,delivery,offroad,…)–  RetailCustomers(cars,trucks,Minivans,SUVs)–  Differentdemographics(US,Canada,China…)


DataIssues

•  CANdataproblems–  Intermittentrecords(1record/secpatternisbroken)–  Incompleterecords(missingdataelement)–  Dataisunreasonable,outofrange.–  Inconsistentchannelcodingacrossvehicletypes(model,engine,

transmission,modelyear)requireatranslationtable.•  Files

–  Millionsofrecords/vehicle–  Gigabytesizevehiclesdatafiles.

•  Analysis–  Can’tperformtimeseriesanalysis.–  Don’twanttointerpolatebetweendatatocreatedata.–  Chosetofilteroutofrangedata.–  Workwithonlycompleterecordsforthechannelsbeinganalyzed.–  Treatedthedataasarandomsampletoapplystatisticalmethods.


DataBias

•  Oldervehicleswithlargeamountsofdatadominatetheanalysisvs.newvehiclewithsmallamountsofdata.–  Asolutionistoweightvehiclesequallybystandardizethemetrics,i.e.,

miles/trip,trips/day,miles/day,percentoftotalusage,…

•  Unequalnumbersofvehicles,likeretailsedansvs.policevehicles,canbiasresultbylosingsmallvehiclepopulationsinaverylargesetofvehicles.–  Asolutionistofocustheanalysistoparticulartypesofvehicles,i.e.,

policevs.retailvs.taxiforthesamevehiclemodel,engine,etc.


TypesofAnalysis

•  Thedesiredanalyticsarespecifictotheengineerormanager.

•  Theanalyticschangewithtypesofdata–  Countingdata(liketripcounts,daysinservice,enginestarts,gear

shifts)–  StateData(likethegearbeingused)–  Continuousvariables(likevehiclespeed,enginespeed,enginetorque)

•  Examples(slidestofollow)

–  Distributionanalysisoftotaleventsonavehicle–  Singleparametervs.miles,time,cycles…distributionanalysis–  ContourPlotsofenginespeedxenginetorque–  Pedalpositionstrokeanalysis–  MarkovAnalysisofstatechanges


Example:TripCounts

•  Afleetofvehicleswasmonitoredusingtelematicsdata

•  Thetotaltripsforeachvehiclewasdetermined

•  Thetotaloperatingdaysforeachvehiclewasknown.

•  Therawdatawasplotted.


TripCountsvs.DaysInService


TripsAnalysis

•  Standardizedtotrips/dayforeachvehicle,i.e.,analyzetheslopeofthelinebetweentheoriginandthedata.

•  Ratesliketrips/daycanbeprojectedto10years,15years,…

•  UsingMinitab,thetrips/daywasanalyzedasNormal,Exponential,Weibull,andLognormaldistributions.TheLognormalprovidedthebestfit.

•  Thedistributionwasusedtodeterminethepopulationpercentiles.

•  The95thpercentileusagewasprojectedfromtrips/daytotrips/10years.


Note:WherethereisaP-thpercentile,P%ofthesampledataisbelowtheP-value.Forexample,50%ofthedataisbelowthe50thpercentilevalue.

Trips/Day–DistributionIDPlot


Thedataarethereddots.

Thebluelineisthebestfitforthedistributionusingthedata.

Correlationstatisticsareshownintheupperright.

Inthiscase,thebestfitswereprovidedbyWeibullandLognormal.Lognormalwasselectedbecauseit’scorrelationwashigher.

Trips/Day–LognormalDistribution

Thebestestimateofa95thpercentileis9.441trips/day.Thiswasroundedupto10trips/dayandcascadesfromthevehicletosystems,subsystems,andcomponents.

Thelognormalanalysisofwasrepeated.

Samplepercentileswereadded

TripsPerDayConclusions

•  Thecorporatevalidationtargetwas95thpercentileusage.–  9.441trips/daywasroundedupto10.–  At10trips/dayx365days/yearx10years=36,500tripsasa

verificationtestforignitionsystemcomponents.

•  The10trips/dayisextendedtoothercomponents.–  Driverdoorcycles(assume2open/closecyclespertrip)yielding

20x10x365=73,000doorcycles/10years.–  Driverseatload/unloadfor36,500cycles.

•  Additionaluses–  Sometimesproblemsoccurredat5thpercentilecustomerusage.–  Themedianvalue(50thpercentile)insteadofaverageforatypical

usage.


AnalysisofContinuousVariables

•  Continuousvariablesmayhaveallpossiblefractionalvaluesbetweenreasonableoperatinglimits.Forexample,thevehiclespeedvariesfrom0mphtoperhaps120mph.

•  Astandardapproachistodisplaythedatawithafrequencyhistogram(countsonthey-axis)andbinsonthex-axis.

•  Considerenginespeed.


HistogramsforAnalytics

0

500

1,000

1,500

2,000

2,500

3,000

0250

550

850

1300

1900

2500

3100

3700

4300

4900

5500

6250

Freq

uency

EngineSpeed(rpm)

Histogram

0

500

1,000

1,500

2,000

2,500

3,000

0 30

0 60

0 90

0 14

00

2000

26

00

3200

38

00

4400

50

00

5600

65

00

Freq

uenc

y

Engine Speed (rpm)

Multi-Vehicle Histogram

VIN 1 VIN 2

Goodvisualizationforasingledataset,butitisdifficulttodeterminestatisticsorusagepercentiles.

Itisdifficulttomakecomparisonsbetweenmultipledatasets.


CumulativeFrequencyPlot

0

500

1,000

1,500

2,000

2,500

3,000

0 30

0 60

0 90

0 14

00

2000

26

00

3200

38

00

4400

50

00

5600

65

00

Freq

uenc

y

Engine Speed (rpm)

Multi-Vehicle Histogram

VIN 1 VIN 2

0

5,000

10,000

15,000

20,000

25,000

0 1000 2000 3000 4000 5000 6000Cu

mulativeFreq

uency

EngineSpeed(rpm)

CumulativeFrequencyHistogram

VIN1 VIN2


Thecumulativefrequencyhistogramshowsthesetwovehicleshaddifferentamountsofdata.Somestandardizationfortheamountofdataisneeded.

CumulativeHistograms

•  Frequencyhistogramissues.–  Analysisresultscanbebiasedtoafewhighusagevehicles.–  Percentilesarehardtodetermine.

•  Solution–  Standardizethey-axisfromfrequencytopercentileusageforeach

plot.–  Useapercentilehistogram.


Standardization

0%

25%

50%

75%

100%

0 1000 2000 3000 4000 5000 6000

Cum

ulat

ive

Perc

ent

Engine Speeed (rpm)

Cumulative Percent Histogram

VIN 1 VIN 2

Afterstandardizingthedatatopercentoftotalusage,itisclearthatthetwovehicleshadverysimilarenginespeedusagepatterns.

0%

25%

50%

75%

100%

0 100020003000400050006000

CumulativePe

rcen

t

EngineSpeed(rpm)

VehicleEngineSpeeds

Thecumulativeusageplotallowscomparisonsofmanyfleetvehicles.


Cumulative(Usage)PercentileHistogram

•  Issues–  Fleetswillshowover-lapofenginespeedtraces.–  Manyfleetvehicleswillproduceunreadableoverlappingtraces.–  Difficulttodefinevehiclepopulationpercentileswithheavyorlight

usagepatterns.

•  Solution:Foreachvehiclespeed–  Determinetheintersectionofthetraceswiththevehiclespeed.–  Calculateintersectionpercentiles,i.e.,5th,50th,and95thby

•  Ifthesamplesizeissmalllessthan20,useanormaldistributionapproximationtodeterminepercentiles.

•  Ifthesamplesizeislarge,thencalculatethemedianranksforeachsampleandinterpolatetodeterminethepercentiles


StatisticalComparison

0%

25%

50%

75%

100%

0 1000 2000 3000 4000 5000 6000

CumulativePe

rcen

t

EngineSpeed(rpm)

EngineSpeeds


Determinethepercentileswherevehiclespeedtracesintersectthevehiclespeed.

Selectavehiclespeed

Generallythe5th,50th,and95thpopulationpercentilesareused.

Thecumulativeusagepercentilesformapopulation.

FleetPopulationPercentiles

0%

25%

50%

75%

100%

0 1000 2000 3000 4000 5000 6000

CumulativePe

cent

EngineSpeed(rpm)

EngineSpeedNormalDistributionAnalysis

P5 Average P95


The5th,95th,andaveragewerejoinedtoformP5,P95,andaveragespeedtraces.

TheP95traceshowstheengineRPMisbelow1800RPMabout92%ofthetime.

TheP5traceshowstheengineRPMisbelow1800RPMabout73%ofthetime.

TheP5tracerepresentshigherenginerpmusage.

ComparisonExample

•  Themedianvehiclespeedtracewasusedtocompareapolicewitharetailvehicles.

•  Themedianpolicevehiclespeedtracewasextremelydifferentfromaretailcustomervehiclespeedtrace.–  66%idletimevs.16%idletime.

•  Acoolingengineerwouldneedthisinformationtodevelopvalidationtests.


FleetComparison

0%

25%

50%

75%

100%

0 10 20 30 40 50 60 70 80 90 100

CumulativePe

rcen

tile

VehicleSpeed(mph)

VehicleSpeedComparison-Policevs.SedanRank

Police Sedan


Policevehicleusageisverydifferentthanretailsedanusage.

About16%idletimeforretailvehicles.

About66%idletimeforpolicevehicles.

MultipleParameterAnalysis

•  Dataformultipleparametersareavailableinthetimestampeddatarecords.

•  Example:–  EngineSpeedandEngineTorque

•  Powertrainengineersliketoviewacontourplot•  Process:

–  Determinethenumberofrecordsineachspeedxtorqueinterval.–  Calculatethepercentoftotalrecordsineachinterval.–  Averageacrossthefleet.–  Displayasacontourplot.


Multi-ParameterAnalysis–ContourPlots


•  Enginefieldusageismostlylowtorqueandlowspeed.•  Dynamometertestingishighlyacceleratedasitis

conductedathighspeedandhightorque.•  Datacountsineachtorque–rpmintervalcanbeused

tocalculatecumulativedamageonparts.•  Damageforfieldanddynamometertestingcanbe

compared.

DamageAnalysis

•  Thisisanopportunitytoperformmanycomponentcumulativedamageanalyses.

•  Thecountsoftimetorquexrpmproducesisproducedforeachgear.Thetorqueandrpmarescaledbythegearratiotodeterminewhateachgearexperiences.Whenthegeardesigninformationisadded,thecumulativedamagecanbecalculatedforeachgear.

•  Thiscancascadedownthedrivelinethroughthetransmissionandaxle.

•  Thisanalysiscanberepeatedforeachvehicle.Theanalysiswouldthenfocusonthedistributionofcumulativedamage.

•  Thefieldusagedamagecanbecomparedtodynamometerdamageduringvehicledevelopment.


Othertypesofparameteranalysis

•  Acceleratorpedalpositionanalysis–  MonitorthepedalpositionusingthepedalvoltageontheCANbus.–  Thepedalpositiontracecanbebrokenintostrokesections.

•  Pressingtheacceleratorpedalprovidesacontinuouslyincreasingvoltagesignal.Thishasalowstartingvoltageandahighendvoltage.Conceptuallythevoltageisrisingfromalocalvalleytoalocalpeak.

•  Releasingtheacceleratorpedalprovidesacontinuouslydecreasingvoltagesignal.Now,thereisahighstartingvoltageandalowendvoltage.Conceptually,thevoltageisdescendingfromalocalpeaktoavalley.

–  Eachpedalupordownstrokeisusedtoincrementacounterinacelldeterminedbythestartandendvoltage.

•  Theresultswereexactlytwicethecyclecountsprovidedbyrain-flowstresscycleanalysis.


AcceleratorPedalStrokesTransitionsAnalysis

0 10 20 30 40

0

20

40

0

0.01

0.02

0.03

0.04

Finish Voltage

WL Accelerator Pedal Strokes

Start Voltage

Pro

babi

lity


TemperatureCycles

•  Asimilarapproachwasusedtoanalyzethermalcyclecounts.

•  Thisisimportantforelectroniccomponentsthatexperiencefatigueduringthermalcycles.–  Electronicsolderjoints,boards,chips

•  Importantinformationtodevelopelectronicvalidationtests–  Fatiguemodelsfordifferenttypesofdevices.


ThermalCycleAnalysis

JamesMcLeish,“AcceleratedLifeTest–SimulationIntegration”,SAEAutomotiveExcellence,Spring,2017.


StateChanges

•  Somecomponentsarestressedwhenstatechangesoccur.

•  Anexampleisatransmissionwhenagearisbeingengaged.

•  Thetotalcountsmaybeused,butamorerefinedmethodwouldbetocountgeartransitions.–  Adownshiftfromthecruisinggear,saythe8thgeardownto7thisless

stressfulthanthe8thdownto6thor5th,…–  TracktheshiftswithaMarkovTable


StateTransitionsAnalysis-MarkovStartwithcountsofstatemeasurementsfromstateXtostateY.XandYcanbethesamestate.

ApplicationExample:Transmissiongearusage


Dividebythenumberofmeasurementsinarowtodetermineprobabilitiesofchangingorremaininginastate.

Issues

•  Gigabytesizevehiclefilescan’tbeheldinmemoryforanalysis.•  Datafromtoomanyvehiclestoprocess.•  Timeconsumingtoreadandanalyzeafile.

–  About1Gigabyte/minuteonalaptop.


TheToolMix

•  Matlab–  Readfilesandextractdata–  Performstatisticalanalysis,creategraphicsandsummarytables..

•  Minitab–  Forsomestatisticalanalysisandgraphics.–  OthertoolslikeSAS,JMP,andStatgraphicscanbeused.

•  Excel,Word,andPowerPointforsummaryreports.–  Providelongtermstorageofanalysisresults


Software

•  Matlabscriptswerewrittenforalaptoptoperformtheanalysisascommercialsoftwarewasnotavailabletoperformtheanalysis.–  Currentanalysiscanbeextendedfurther.–  Currentlydevelopingagraphicalinterface.

•  Otherprogramminglanguagesandplatformscanduplicatetheanalysis.Thecurrentresultsareaproofofconcept.


UserFriendlyTools

•  Needtodevelopagraphicalinterface.–  Makeiteasierfortheanalysttouse

•  Scriptssupportautomaticreportgeneration

•  Needdistributedprocessingandcloudstorage.–  Fasterprocessingofindividualvehiclefileswithmanyparallel

processors.–  Redundancyincloudstorageprotectsthedata


Conclusions

•  Telematicsdataanalysiscanprovidedatadrivendesigntargetsconsistentwiththeorganizationmissionprofile.–  Examples:10years/100,000miles;10years/150,000miles…

•  Targetingaggressivecustomerusagesispossiblebyselectingthehighstressusers,i.e.,1stor99thpercentiles,or5thand95thpercentiles.

•  Worktodateisaproofofconcept.Furtherdevelopmentisrequiredtodevelopsoftware,databases,hardware,…

•  Monitoringfocusesonthepopulation,nottheindividual.Privacymustbemaintained.


ContactInfo

DennisCraggsQuality,Reliability,andAnalyticsServices

[email protected]

12028TownlineRd.,GrandBlanc,MI48439


Documents

ASQRD Webinar Telematics Data Analysis Dennis L Craggs ... · 11/8/18 ASQRD Telematics Webinar 6 • This mission profile defines launch, rendezvous and docking, docked, decent, landing