Budapest, 26-28 October 2016
TESTING IOT / 5G TEST NETWORKTeemu Kanstrén, Jussi Liikka, Jukka Mäkelä, Pekka RuuskaVTT Technical Research Center of Finland
© All rights reserved
5GTN Overview
2 © All rights reserved
Internet of Things, Big Data, Machine Learning, …
• Testing IoT? • Simple devices, complex combinations
• Using data analytics & machine learning for test analysis?• IoT, distributed systems, etc. produce ”big data”. How do we evaluateit?
• Using test generation to generate data for analysis?• Getting the required data to develop algorithms, tools, features can bea challenge
3 © All rights reserved
Basic sensor profiling
4 © All rights reserved
Basic sensor profiling
5 © All rights reserved
Long term(big) data
e.g., few weeks = some millions of samples
Basic sensor profiling
6 © All rights reserved
Long term(big) data
e.g., few weeks = some millions of samples
Statistical profiles per time intervals,Traffic patterns,Measurement patterns,…
Basic sensor profiling
7 © All rights reserved
Long term(big) data
e.g., few weeks = some millions of samples
Statistical profiles per time intervals,Traffic patterns,Measurement patterns,…
18,5
19
19,5
20
10:33:36 11:02:24 11:31:12 12:00:00 12:28:48 12:57:36 13:26:24
temperature
temperature
Basic sensor profiling
8 © All rights reserved
Long term(big) data
e.g., few weeks = some millions of samples
Statistical profiles per time intervals,Traffic patterns,Measurement patterns,…
18,5
19
19,5
20
10:33:36 11:02:24 11:31:12 12:00:00 12:28:48 12:57:36 13:26:24
temperature
temperature
Similarly, e.g., network data profiles (packet sizes, frequencies, …)
More Complex Sensor Networks
• For example, indoor location via beacons• Several beacons evaluated at any time• Location = value function of sensors
• Need to consider sensor relations• Signal strength vs others• Environmental factors
• Or relation to sensor itself• Smoothing curve vs hop• Moving: allowed, expected, service functions, …• …
9 © All rights reserved
Analysing the Data (Test Oracle?)
• With thousands of parameters and up to millions orbillions of data values from the tests…
• What happened?
• What matters?
10 © All rights reserved
Problem area identification
• Automated analysis of large test data sets• E.g.,
• performance spikes, drops, • Important data elements,• causes..
11 © All rights reserved
Problem area identification
• Automated analysis of large test data sets• E.g.,
• performance spikes, drops, • Important data elements,• causes..
• A simple case: statistical profiling and automated outlier detection
12 © All rights reserved
Problem area identification
• Automated analysis of large test data sets• E.g.,
• performance spikes, drops, • Important data elements,• causes..
• A simple case: statistical profiling and automated outlier detection
13 © All rights reserved
Problem area identification
• Automated analysis of large test data sets• E.g.,
• performance spikes, drops, • Important data elements,• causes..
• A simple case: statistical profiling and automated outlier detection
14 © All rights reserved
Profiling via test scenariosIoT (L1) Video (L2) Idle (L3)
55659 30025 64
55985 28009 1
55604 28021 5
55303 35789 0
55744 28950 0
… … …
15 © All rights reserved
• Labeled datasets• IoT = L1• Video = L2• Idle = L3
• What are the impacts?• Inputs to learning algorithms, • To find labeled targets (categories)
Example: Visualizations with PCA, TSNE, …
16 © All rights reserved
Finding the relevant features
17 © All rights reserved
Just a small subset ->Manually exploring all the data is hard
Finding important features: Automation
18 © All rights reserved
e.g., Recursive feature elimination(RFE) algorithm
What for?
• Test devices, networks, services against new types of input profiles
• Provide new inputs and datasets for development• Analyze and evaluate results, identify interestingproperties and characteristics• Profiles for users, devices, sensors, environmental factors, …• Performance statistics
19 © All rights reserved
Conclusions
• IoT properties & challenges• Simple sensors form complex combinations• Combining with big data, data analytics & machine learning
• Testing here• Exploring the impact of parameters, profiles, configurations, performance results, functionality
• Providing inputs to features, algorithms, optimizations, …
20 © All rights reserved
www.5gtn.fi, www.5gtnf.fi
TECHNOLOGY FOR BUSINESS