Activity Recognition in Adaptive Assistive Systems Using Artificial Neural Networks

Stefan Oniga, József Sütő

University of Debrecen, Faculty of Informatics, Hungary

Goals

� To develop technologies for independent daily lifeassistance of elderly or sick persons.

� Design a complex assistive system that can learn andadapt due to the uses of artificial neural networks (ANN).

� Design and test of several Matlab ANN models in orderto find the best performing architecture.

� Finding necessary preprocessing of raw data aiming tohave a better recognition rate.

� Optimization of the number of sensors and theirplacement in order to obtain the best trade-offbetween recognition rate and the complexity of therecognition system.

Human activity and health parameters monitoring system

� The system was developed for human activity and healthparameters monitoring

�temperature, heart rate, acceleration

� Focuses on studies and results obtained on

�arm posture recognition,

�body posture recognition

�usual activities recognition:

�lying on various sides, sitting, standing, walking, running,descending or climbing stairs etc..

Activity/health monitoring system (version 1)

• Chronos watch from TI as acceleration data source

• Chest belt from BM Innovations as heart rate data source.

• The receiver were built-up from a ChipKit Max32, a Wi-Fi shield and a

communication shield that holds the BM receiver and the TI access point.

Monitoring system block diagram

Wearable watch sized monitoring system

• ADXL350 acceleration sensor from Analog Devices,

• CC2541 low power SoC for Bluetooth low energy (BLE), from TI

• TPS61220 Step-Up (Boost) converter.

• The tag is powered by a single coin cell battery (CR2032).

Human activity and health status recognition

� Research directions

�development of a Matlab model of activity recognition system that use artificial neural network (ANN)

�development of hardware implemented RT recognition system

� Finding ANN model with good performance which is also easy to implement in hardware is not exactly an easy task

� We concluded that good results could be obtained with

�two-layer FF-BP network, with sigmoid activation function

�We have chosen Levenberg-Marquardt training method

�For performance evaluation we used the MSE function.

Arm posture recognition

� 6 arm postures

� Acceleration data are supplied by TI Chronos smart watch

� The ANN model is presented

� The recognition rate was 100 % on the data used for training

Body posture recognition

� 5 body postures (sitting, prone, supine, left lateral recumbent and right lateral recumbent.)

� Chronos watch fixed on chest

� ANN with 10 neurons on hidden and 5 neurons on the output layer.

� The recognition rate was 99.96%, MSE = 3.6747e-004

0 100 200 300 400 500 600 700 800 900 1000-1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000-1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000-1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000-1

0

1

2

0 100 200 300 400 500 600 700 800 900 1000-1

0

1

2

Samples

Neural network outputs

Activity recognition� activities to be recognized

� Using 27 samples/second rate we acquired 600 samples for each activity, from three acceleration sensor

1. Standing, 10. Left bending

2. Sitting 11. Right bending

3. Supine 12. Squats

4. Prone 13. Standing up/Sitting down

5. Left lateral recumbent 14 .Falls

6. Right lateral recumbent 15. Turns left and right

7. Walking 16. Climbing stairs

8. Running 17. Descending stairs

9. Bending forward Transitions

Optimizing activity recognition

� Research directions:

�test of several Matlab ANN models for activity recognition in order to find the best performing architecture

�finding the necessary preprocessing of raw data aiming to have a better recognition rate (as Mean value, Variance, Energy, Correlation coefficients, Frequency-Domain Entropy, Log FFT Frequency Bands, etc. )

�finding the number of sensors and their optimal placement

Preprocessing of raw data

� The standard deviation (SD)could be used with very good results as a supplementary input data for the neurons.

ANN input data

1 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc 95.44%2 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc, Std_w600(X+Y+Z-Acc) 96.28%

3 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc, Std_w50(X+Y+Z-Acc)1 98.06%

4 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc, Std_w50(X+Y+Z-Acc)2 98.07%

5 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc, Std_w50(X+Y+Z-Acc)3 97.81%

6 X-Acc, Y-Acc, Z-Acc, X+Y+Z-Acc, Std_w50(X+Y+Z-Acc)4 96.28%

Recognition rates as function of sensors arrangements

� We acquired 600 samples for each activity, from three acceleration sensors placed on: the right hand (Acc1), above the right knee (Acc2) one on the chest (Acc3)

� Recognition rates for static activities

Recognition rates for dynamic activities

Comparison between recognition rates for static and selected dynamic activities

� For static activities the recognition rates are between 0.5% limits for all possible combinations.

� For all dynamic activities the best results could be obtained using the two accelerometers setup and an ANN with 2 hidden layers.

� For the selected dynamic activities we obtained good results even for the one accelerometer setup (Acc2) => we can use a simpler ANN with one hidden layer with only 20 neurons.

Conclusions

� The use of ANN was found to be very effective even for architectures with one hidden layer with 20 neurons.

� Even using a single 3-axis acceleration tag combined with proper signal preprocessing e.g., mean, standard deviation, etc. very high recognition rates can be obtained.

� As expected the recognition rate for the static activities was better than for dynamic activities.

� We also implemented and tested a real time recognition system using Raspberry Pi mini-computer.

� Further research will be made regarding the best performing, hardware implementation friendly, ANN.

Thank you for your attention!