Building aCondition Monitoring Application with SensorTile Wireless Industrial Node

Ernesto Manuel CANTONE

Product Marketing Manager

MEMS and Sensors, Americas

Agenda

1Condition based monitoring typical applications

2ST's portfolio for condition monitoring applications

3Sensor Tile Wireless Industrial Node

4 STWIN Function Packs

5 Focus on HS Data Logging

6How to move from condition monitoring to predictive maintenance

2

Condition based monitoring typical applications

3

ST's portfolio for condition monitoring applications

4

Steps to a predictive maintenance system

5

Predictive

maintenance

▪ Acquisition sensor setup

▪ Retrieve data over wired/

wireless connectivity

▪ Label data

▪ Store data

▪ Data cleaning / denoising

▪ Data visualization

▪ Preprocessing and

Feature Extraction

▪ Feature Engineering

▪ Machine learning of the

system behavior

▪ Semi-supervised

learning at the edge for

anomaly detection

▪ Supervised learning to

classify anomalies

▪ Model deployment

▪ Remaining Life

prediction models

▪ Overall efficiency

optimization

▪ Operational systems

integration

Data

Acquisition

Condition

Monitoring

Anomaly

detection &

classification

Edge - Factory Level (processed sensors data) Company Level (ERP, etc.)

ST MEMS and sensors focus products

6IndustrialConsumer Automotive

AEC-Q100

MP34DT05-A IMP34DT05

MP23ABS1 IMP23ABSU

IIS3DWB

LIS2DE12

IIS2ICLX

LIS2DH12

LIS2DW12 / LIS2DTW12

IIS3DHHC

IIS2DLPC

Magnetometers / e-Compass

AIS2DW12

Accelerometers / Inclinometers

LIS2MDL

LSM303AGR / LSM303AH

IIS2MDC

ISM303DAC

6-axis IMUsLSM6DSO / LSM6DSOX

LSM6DSO32/LSM6DSO32X

LSM6DSR / LSM6DSRX

ISM330DHCX

ASM330LHH

STLM20 / STTS751

STTS22H

HTS221

LPS22HH

LPS27HH(T)W

MP34DT06J

MP23DB01HP

Environmental

Microphones

H3LIS331DL

VL53L0X

VL53L1CB

VL53L1CX

VL53L3CX

VL6180V1

VL6180X

Proximity and Ranging

Sensors

AIS2IH

LIS2DU12

VD6283TX

VL53L5CX

Ambient Light Sensor

Sensor Tile Wireless Industrial Node STEVAL-STWINKT1B

www.st.com/stwin

SensorTile Wireless Industrial Node

Available Software

FP-IND-PREDMNT1

FP-CLD-AZURE1

FP-SNS-HSDATALOG1

Mobile App

ST BLE Sensor

Cloud App

DSH-PREDMNT

IMP23ABSU

BlueNRG-M2

STEVAL-STWINWFV1

7

SPI3

ADC1

USART2

DFSDM1

I2C2

IMP34DT05Digital Microphone

LPS22HHPressure Sensor

HTS221Humidity and

Temp. Sensor

STTS751Temperature

Sensor

IIS2MDC3D Magnetometer

STM32L4R9ZIJ6Microcontroller

Ultra Low Power Cortex-

M4F@120MHz

32 kHzCrystal

16 MHzCrystal

BlueNRG-M2SABLE Application

Processor Module

STR485LVRS485 Interface

SPI2

IMP23ABSUAnalog Microphone

TS922EIJTLow noise, low

distortion Op Amp

20-pin STMOD+

connector

12-pin female sensors

connector

12-pin male comm.

connector

40-pin Flex connector

STSAFE-A110Secure Element

I2C2

SPI2, I2C4

USART3

IIS2DH3D Accelerometer

IIS3DWBVibrometer

ISM330DHCX6-Axis IMU

USBLC6-2P6USB ESD protection

EMIF06-MSD02N16EMI filter and ESD protection

STBC02Li-Ion linerar battery

charger

ST1PS01EJRstep-down switching

regulator

LDK130Low Noise LDO

DSI, CAN,

..

SPI1,

I2C3,

USART3

SAI1,

I2C2,

DFSDM2

ESDALC6V1-1U2Single Line ESD protection

• Best-in-class Industrial Grade

Sensors

• Multiple algorithms running on the

STM32L4+

• Secure Connection and

Authentication with STSAFE-110

• Out-of-the-box BLE Connectivity

• Connectivity and sensor expansions

support

• Smart Power to increase battery life

(Li-Po battery, USB or ext. 5V)

• FP-IND-PREDMNT1 IoT sensor

node for condition monitoring

• FP-CLD-AZURE1 connect an IoT

sensor node to Microsoft Azure

• FP-SNS-DATALOG1 High speed

Datalog

STEVAL-STWINKT1Bdiagram, ICs and STM32CUBE Function Packs

8

FP-SNS-DATALOG1

9

• High Speed Datalog application for STEVAL-

STWINKT1B.

• Comprehensive solution to save data from

any combination of sensors and microphones

configured up to the maximum sampling

rate

• Compatible with Unico-GUI which enables

configuration of ISM330DHCX Machine

Learning Core unit

• Inertial 3 axis MEMS: 6 Bytes per sample (2 Bytes x 3 axis)

• IIS3DWB ODR@26.7 kHz: 1.3 Mbps

• ISM330DHCX ODR@6.667 kHz (axl + gyro): 640 kbps

• Audio data rates

• Audio PCM @ 48 kHz sampling rate: 768 kbps

• Ultrasound PCM @ 192 kHz sampling rate: 3.1 Mbps

Vibration sensing: why high speed?

10Mechanical Vibration / Sound analysis (1 ÷ 10kHz)

IIS3DWBInertial

Ultrasound analysis

ISM330DHCX

IMP23ABSUAcoustic

Bearings

Gear boxes

Lubrication

Fan bearings

Venting occlusion

Unbalance

Looseness

Misalignment

Roller Bearings

Gearing

Cavitation

kHz2 806 103

Introducing High Speed Data logging

Host / Gateway

ST-Unico

SDK

GUI

11

BLE

USB

SD

Card

Control

Data High throughput

Data Low throughput

• STWIN connects to a smartphone app allowing to:

• configure sensors

• implement datalogging and labeling

• interfacing MLC configuration

ST BLE Sensor

• Sensor data reception and command transmission

over Bluetooth® Low Energy (BLE)

• Support for Condition Monitoring and High-Speed

data logging configurationEnvironmental data – FFT Analysis of Vibration – Configuration for data logging

Configuration of MLC– data acquisition and labeling on SD Card

An app to get the most out of your device

12

Python SDK

13

STEVAL-STWINKT1/B

HSD_PythonSDK

HSDatalogAcquisition

Folder

HighSpeedDatalog Python SDKv1.1.0 (requires Python 3.7):• Ready-to-use scripts for “casual” users (data extraction, validity check, conversion, visualization, acquisition control..)

• Includes three python modules:

• HSD: [Core module] Virtual Device Model management, Data extraction & Visualization

• HSD_utils: Independent module which contains and categorizes not-core features

• HSD_link: Communication with physical devices

• Jupyter notebooks with clear code explanation cells for developers

• Designed to be extensible and easily integrable

CSV xSVTSV UNICO NANOEDGE.AI

DataFrame extraction or validity check

Data format conversion

DataVisualization

Microphones data to Wav conversion

Device Configuration &Acquisition Control

AI Model train and evaluation

under definition

DEMO: High Speed Data Logging

14

• STEVAL-STWINKT1B running FP-SNS-DATALOG1

• Connected directly to PC via USB using CLI (1)

• Connected to ST BLE Sensor app for configuration (2)

and SD Card recording

• VIDEO HERE

15

How to move from condition monitoring to predictive maintenance

Steps to a predictive maintenance system

17

Predictive

maintenance

▪ Acquisition sensor setup

▪ Retrieve data over wired/

wireless connectivity

▪ Label data

▪ Store data

▪ Data cleaning / denoising

▪ Data visualization

▪ Preprocessing and

Feature Extraction

▪ Feature Engineering

▪ Machine learning of the

system behavior

▪ Semi-supervised

learning at the edge for

anomaly detection

▪ Supervised learning to

classify anomalies

▪ Model deployment

▪ Remaining Life

prediction models

▪ Overall efficiency

optimization

▪ Operational systems

integration

Data

Acquisition

Condition

Monitoring

Anomaly

detection &

classification

Edge - Factory Level (processed sensors data) Company Level (ERP, etc.)

• Advanced MEMS sensors contain a Machine

Learning Core (MLC), a Finite State Machine

(FSM), and advanced digital functions. They

run custom algorithms on the IMU and share

the workload from the main processor enabling

system functionality while significantly saving

power.

Artificial intelligence at STMicroelectronics

• Thanks to STM32Cube.AI and NanoEdge™ AI Studio and you can run pre-trained Artificial Neural Networks (ANN) or efficient machine learning algorithms with on-chip self-training on STM32 microcontrollers.

18

Validate code directly on target

Get accuracy and inference time

Optimize memory usage

Select most appropriate MCU

Review computation and memory

consumption per layer

Train Neural Network using

any major AI frameworks

run-time

Run on optimized runtime

STM32Cube.AI: Easily implement Neural Networks on STM32

And more

Convert NN into optimized

code

19

Model is self trained at the Edge

• Dynamic learning on device

• 100% accuracy

• Any embedded developer can do it

• Any STM32 Cortex M0 - M7

• Super small RAM footprint

NanoEdge™ AI Studio: Your way to Machine Learning at the edge

Create and embed a self learning engine

Standalone PC (Win/Linux) application

• For embedded developers

• Create the best ML library for each project

• No data science skills required

• Search across millions of possible algorithms

• Output is not a static model but a self

learning engine

CREATE the library ONCE USE the library MANY TIMES 1 2

Smart System Challenges:Sensors and AI on the Edge

SENSOR CLOUD

01101000110100

0110100011010011011000111010011100

10010110101001

10010110101001

1000111011011110001110010110101001

10111100011100101

101111000111001

1000111011011110001110010110101001

01101000110100

GATEWAY

Time-sensitive applications should be locally processed

Opportunity: move computation down to Sensors

Cloud Processing limitations:

• Latency

• Network bandwidth and/or

connectivity loss

• Data privacy

• Autonomy of battery-operated

devices

21

Machine Learning@ Sensor

• Higher computation power at

sensor level

• Lower power consumption at

system level

• Cost optimized solution

From Low Power Sensor to Low Power System

Machine Learning Core (MLC) for real edge computing enables

high system flexibility

Sensor with

Machine

Learning

Core

High-level

processing

Power optimization

at system level

+

FSM MLC

This is added value!

ML Data

Machine Learning@ Main Processing Unit

Sensor

Full activity

monitoring

processing

Raw data

22

ST toolbox for machine learning core (MLC)

23

Capture data Label data Build decision tree Embed decision tree Process new data

• Accelerometer

• Gyroscope

• External sensors

• Filters

• Features

• Real time test• Decision Tree

implementation

• Classification

• Results

HO

WW

HA

T

Unicleo-GUI

ST BLE Sensor

Unico-GUI

AlgoBuilder

Unicleo-GUI

ST BLE Sensor

Unico-GUI

AlgoBuilder

Unico-GUI *

* External tools for Build Decision Tree step:

Weka, RapidMiner, MATLAB, Python

STEVAL-

STWINKT1B

Support to MLC by ST Tools and GUI

Takeaways

24

• ST provides MEMS and Sensors dedicated to Industrial market, with

10 Year Longevity

• ST has all the building blocks to enable Condition Monitoring in

Industrial IoT Applications

• ST provides development kits to help building Proof of Concept with

existing industrial machines retrofit

• ST provides firmware packages aimed at logging sensor data

directly from a sensor node or through a cloud application

• Broad choice of ST and Partner’s tools to enable the use of

machine learning techniques with ST dev kits and products

© STMicroelectronics - All rights reserved.

ST logo is a trademark or a registered trademark of STMicroelectronics International NV or its affiliates in the EU and/or other countries.

For additional information about ST trademarks, please refer to www.st.com/trademarks.

All other product or service names are the property of their respective owners.

Thank you

Find out more at www.st.com/sensors