1

EE2 PROJECT GROUP 3 – AUTOMATED PEOPLE COUNTER Interim report – February 8th

Supervisor: Dr Tom Clarke

Name CID

Omar Wolfe Hussein 01046652

George Padley 01088571

Jacob Kay 01052280

Zheng Lee 01042500

Laura Tuckey 01049760

Karmanya Sareen 01067937

Alex Karet 01078146

Samuel Yang 01067937

Imperial College

2

Contents Page

Section Title Page

1 Abstract 3

2 Introduction 3

3 Design Specification 4

4 Concept Designs 5

4.1 Detection Methods 5

4.2 Laser 6

4.2.1 Implementation 6

4.2.2 Laser Safety 6

4.3 Computer Vision 7

4.3.1 Implementation 7

4.4 Microcontroller 8

4.5 Database and Web System 9

4.5.1 Stacks 9

5 Discussion 9

6 Conclusion and Future Work 11

7 References 12

8 Appendix 14

Table 1: Design Specification Rationale 4

Table 2: Detection Concept Evaluation 5

Table 3: Laser Device Evaluation 6

Table 4: Computer Vision Evaluation 7

Table 5: Microcontroller Decision Matrix 8

Table 6: Future Potential Problems and Mitigations 10

Table 7: Decision Matrix Justification 18

Table 8: Meeting Minutes 22

Table 9: Project Plan 29

3

1 - Abstract

Since the start of the project good progress has been made and a design concept has been chosen. Technical work has begun in order to develop the subsystems and get them working together. A set of design specification points was created and used to pick the detection method, which is the core subsystem. It was decided that the device would use two detection methods in order to minimise errors and increase accuracy, and that these would be laser ‘tripwires’ as described in the project proposal and the use of computer vision. Since the decision has been made, a choice of server platform and processing system was chosen and work on interfacing with the sensors has been started. So far; the computer vision team has created an algorithm that analyses pre-recorded video and determined the net movement of people; the hardware team has started developing on the Raspberry Pi and is working with the laser diodes; the web and server team have created a protocol for the data transmission and have started work with the servers.

2 - Introduction

Large buildings, typically office blocks, can contain thousands of people; for example the soon to be completed ‘Apple Campus 2’ will house 13,000 staff (Apple Campus 2, 2017). In the event of an evacuation, emergency responders are faced with determining how many people are remaining inside the building and where they are. Currently the fire department depend on site managers and documented building plans to direct them to the fire, however if these are not available, then the fire service start a methodical search of the building (Jerray-Silver, 2017). For buildings of the size in question, this problem is made much more difficult purely because of the number of people present. In 2015/16 firefighters in England responded to 15,984 fires in non-residential buildings, in which 1,110 people were injured or died (Office, 2016). This number can be reduced if emergency responders can be provided with live, accurate readouts of the people count of each room. This project sets out to solve this problem. Current people counting devices, although easy to use and accurate, are not designed with this usage case in mind and as such output data in a way that it not suitable for rescue teams, and hence is not used in this way at the moment (Jerray-Silver, 2017).

Making this change from the existing solutions for counting people in buildings offers a range of markets where this sort of device would be useful. One of the most obvious is for security teams to easily determine if there are people remaining in the building after hours. In this case the requirements are very similar to emergency responders; as they require the same information provided in the same way. Hence, the same device can be designed for application in both scenarios.

The concept for this device breaks down into the three distinct subsystems: the detection system, the processing and communication system, and the data interface system.

4

3 - Design Specification

The Design Specification was written to provide a framework to which the concepts and final project can be evaluated against. It is based on extensive research of the features required in order for the project to function as well as possible and can be found in the Appendix G. The rationale for some of the specification points are provided below.

Specification Point Rationale

PWR 1.2 CAT5 networking cables are commonly installed in buildings. It is trivial to use these to transmit both power and data using the Power Over Ethernet (PoE) method as all that is required is a PoE networking switch. This makes it a suitable method of powering the device.

PWR 2.2 The average length of a power in OECD1 countries was less than 60 minutes in 2009 (Hodge, 2015). Specifying a minimum time of 60 minutes of power should allow the device to operate through all but the worst of power cuts. In addition, the average response time by firefighters to an incident in England in 2014/15 was 8 minutes and 28 seconds (Government, 2015). If the power were to be lost because of a fire then staying online for 60 minutes would allow plenty of time for fire crews to find and act on the data.

DEV 4.3 The shortest person alive is 56.4cm tall (List of Shortest People, 2017) This marks a clear minimum height for which the device must work.

DEV 5.3 In a survey of our year group, 84% of responders said (Hussein, 2017) they would forget to interact with people counting system more than 70% of the time, if it required their interaction in order to count them. Therefore, the system should count people without needing their input.

DEV 8.1 Research has shown that lethal injury due to high temperatures occurs in less than 3 minutes at temperatures of 350°F (Chiefs, 2012). It is noted that flashover happens at temperatures of 1200°F and can take around 5 minutes to take place, in which case the chances of surviving longer than 5 minutes in a building fire are slim.

DEV 8.3 The device will most likely be positioned in or on a doorway. This exposes it to the risk of being kicked or hit with items such as trolleys. If this were to happen the device must be able to continue operating as expected or else it introduces the danger that it miscounts the number of people during an emergency situation.

IFC 12 It is very likely that the interface will be used both on a desktop PC or laptop and on a mobile device. Research with the Acton Fire Station (Jerray-Silver, 2017) showed that they would appreciate a tablet be left in an accessible location with the interface running. Therefore the interface must run on all such devices.

Table 1: Design Specification Rationale

1 OECD: Organisation for Economic Co-operation and Development. www.oecd.org

5

4 - Concept Designs

To manage the project effectively the group has been split into smaller teams: Software/Hardware, Web and Computer Vision. This has been done to use the skills within the group as effectively as possible and to avoid the issue of too many people focussing on one task. This also allows for a more modular design. The advantage of the modular design is that it allows for work to be done in parallel and for each module to be more flexible. Because each module is designed with an interface, each module can be made specifically for its purpose. All interfaces between teams are well specified which allows for the teams to work in parallel. The reasoning behind the teams chosen is entirely based on the current skillsets within the group. Initially the development of Computer Vision was planned to be done after the lasers however due to the decision for a modular design approach it was possible to start development of Computer Vision in parallel with the lasers. This enables the 2 concepts of how the counter detects people, and thus the dual authentication system, to be developed in parallel. The merging of the 2 detection systems has been put in the project plan to do once they are both working properly.

The idea behind this project has mainly social advantages. The project will give firemen and other emergency responders the ability to locate people in a building quicker and easier than their current methodical search approach. This will mean that people trapped in the building will be found much faster, hence reducing the chances of injury. Furthermore if the firemen arrive at the building, and the manager can say with confidence that there are no people in the building, then it saves the firemen time as they can start extinguishing the fire straight away. The project also benefits the building manager as they know that they are giving the firemen all the information relevant the building through the project’s data.

4.1 - Detection Methods

There are many different systems that could be used to detect the passage of a person through a doorway. Four different methods were identified; laser ‘tripwire’ beams, pressure pads, RFID card scanning and cameras. Below is a summary of the different methods.

Cost is defined as the cost of the hardware required to implement the system, excluding any processing required.

Laser Beams

Pressure Pads

RFID Cards Cameras Wi-Fi Tracking

Bluetooth Tracking

GPS Tracking

Cost £9 x 2 £9 x 2 £50 £15 £40 £20 No Cost

Active Interaction Required?

No No Yes No Requires phone to be present

Require phone to be present

Requires phone to be present

Interface GPIO GPIO SPI / I2C USB SPI / I2C SPI / I2C Software only

Reliability 7 7.5 10 9 5 5 6

Ability to detect multiple people

4 7 0 8 10 10 10

Table 2: Detection Concept Evaluation

As DEV 5.3 states that the system should not require any active interaction by people in order to count them, RFID,

6

Wi-Fi, Bluetooth and GPS are not viable, as they require the person to either interact with the system directly or have on them a device that would need to be on or have an app installed. Therefore the only potential options are Lasers, Pressure Pads or Camera. From these, it is desired to choose two systems so that there is a failsafe and verification of the data from two sources. Pressure Pads have an additional disadvantage that Lasers and Cameras don’t share: pressure pads are mechanical and so have a more limited life cycle. They are also more intrusive to install, as they block the bottom of the doorway. For these reasons, it was decided to continue the development using the laser and computer vision - as outlined in the original proposal.

4.2 - Laser

In order to detect a person going through the door, a receiver and transmitter of an Electromagnetic signal could be used. The options available along with their advantages and disadvantages are as follows:

Sensor Advantage Disadvantages

Laser Pointer – Class III · Good Range · Easily detectable

· Safety regulations · Cost ineffective · Power ineffective

Laser Module – Class III · Good Range · Easily detectable · Easy installation

· Safety Regulations · Comparatively less

durable

Infrared Transceiver · Extremely Cost effective · Easy installation · Power Effective

· Short Range · Error Prone · Hard to detect

Table 3: Laser Device Evaluation

Upon analysing the options, the laser modules (Class III) were chosen for the first prototype of the setup. Ideally, these modules will be powered by the Raspberry Pi.

4.2.1 - Implementation:

The current plan for the laser detection system is to use at least two pairs of beams, one above the other, each pair containing two beams parallel, at the same height. Having two beams at the same height will allow easy detection of the person’s direction of travel through the door to be detected. This can be done by evaluating which beam is broken first. Having at least two pairs of beams will decrease the likelihood of accidental detection occurring, as it is difficult to pass through both sets of beams, without actually leaving the room through the door.

Test code has been developed for the Raspberry Pi that uses rising edge interrupts to detect the laser being triggered. This has not yet been connected to the laser, but uses an Arduino to simulate the pulse that would be received from the laser. This is possible because of the modular approach to the project. The code can currently detect the time difference between the pair of lasers being triggered and hence detect the direction of a person. It still needs to use this data to count people.

4.2.2 - Laser Safety:

Laser products are regulated due to the potential for injury. When selecting a laser device, it is necessary to make sure that it is safe for use in public areas, and as such make sure it satisfies safety regulations. A summary of safety criteria is below.

7

● The laser module must have a power rating of 5mW. ● The laser must not harmful to the human eye in most cases, unless purposely looked into the laser for a long

period of time. ● A warning label or caution sticker may be required.

Standard office use laser pointers fulfil all these requirements and so is the best way of easily implementing this system. (England, 2014)

4.3 - Computer Vision

Computer vision can be used as a dual authentication method to increase the accuracy of the counter. Although lasers

are used as the primary counter, it will not be able to accurately count the number of people if more than one person

enters the building at the same time. That is where computer vision comes it, as computer vision alone has an accuracy

of about 90-95%. (Haylock, 2015) (Kinsley, 2016) (Inc., 2016) (team, 2014)

4.3.1 - Implementation

Computer vision requires a live image input to the processor. As the project is using a Raspberry Pi there is the option

of using a USB webcam as the input device. This simplified the design as USB is a developed and easy-to-interface-with

on a Pi. The camera can be mounted on the ceiling near the door facing perpendicularly downwards to record

movements of people going in and out of the room. As it will see the movement of the head and shoulders across the

image, determination of direction will be relatively simple. The software required in order to enable the detection of

people is as follows

1. Capture images from the camera frame by frame

2. Pass the frames through object recognition algorithms, from OpenCV

3. Track the objects’ movements across the field of view.

4. Determine direction of motion of the object.

5. Update net movements count based on direction of movement.

As the Raspberry Pi’s main developmental tool is written in python, OpenCV (Open Computer Vision, a python library)

will be used. Using computer vision comes with its own set of advantages and disadvantages, these are summarised

in the table below.

Advantages Disadvantages

Highly accurate. More expensive than sensors.

Tireless, able to operate 24-7. May require quite tedious coding in order to

program the camera to track movements

accurately.

Image capturing devices are easy to mount and

remove.

Illumination variation such as shadows

complicates the design of the programming

algorithm.

Table 4: Computer Vision Evaluation

8

The Computer Vision code is working for pre-recorded video (due to lack of available webcam) and the code is in

appendix E.

A programme has been written by the team using Python to count the number individuals walking in and out of a

room. The program was tested with a pre-recorded video. It was able to:

● Count individuals walking from both directions one at a time.

● Identify number of people walking through from the same direction.

However for the time being, it was unable to count accurately when there are multiple individuals walking from

opposite directions. The code can be found in the Appendix. Look at the front cover

4.4 - Microcontroller The solution requires a microcontroller of some variety to take input from the sensors, process it and send to the database system. Ideally a microcontroller is low power, cheap, has Wi-Fi built in, and is easy to program. Another major criteria is the microcontroller being powerful enough to support computer vision, the laser beam system, and sending data to the database simultaneously.

There were three main microcontrollers considered for the design, the Raspberry Pi (Foundation, 2016), the ARM Cortex MCU (Corporation, 2016), and the Arduino Yun (AG, 2017). The decision matrix below summarises the decision to use a Raspberry Pi. See appendix C for an explanation of the criteria.

Raspberry Pi ARM Cortex MCU Arduino Yun

Main programming language 9 6 8

GPIO 7 8 5

Price 8 10 4

Networking 9 6 7

Processor speed 10 5 7

RAM 10 4 6

Community support 8 4 4

Power consumption 4 7 8

Flexibility 9 4 5

Total 74 54 54

Table 5: Microcontroller Decision Matrix

The microcontroller needs to support interrupts on the pins used for the lasers. This is to ensure that no trigger of the laser is missed while the microcontroller is doing other tasks like sending data to the database. Interrupts stop the microcontroller executing its current code and make it run code to deal with the interrupt. All three microcontrollers support interrupts, however the Raspberry Pi has a library that allows threaded handling of interrupts. Hence, the Raspberry Pi, while sending the data can simultaneously prioritise the execution of the laser code in another thread ensuring no person is missed. The other two microcontrollers are not powerful enough to be able to do this.

9

Another requirement is processing power. Computer vision is computationally intensive, and as it is being developed in parallel to the laser tripwire, the microcontroller chosen needs to be powerful enough to support computer vision. This requirement is satisfied only by the Raspberry Pi.

The software on the microcontroller will be split into four main components: Laser, computer vision, data compilation, and database transmission. Python was chosen for this after the microcontroller was chosen. The Raspberry Pi supports python. Python is also a very flexible language with a lot of open source libraries that are really useful for this project. For example computer vision is made easier through the existing OpenCV (OpenCV, 2015) library.

4.5 - Database and Web System The solution will use two separate databases to store information for each building. One will be the local database. This will store information about the individual building and will process all the data sent by each unique sensor. The second database will be stored offsite. This will store information on all the buildings and their numbers as well as the history of all the buildings. It is the main server which will be accessed by the emergency services.

4.5.1 - Stacks The web side of the solution will need to be built on a software stack. This is a collection of software which have been

written to work well together. There are quite a few stacks currently in use. The main ones are LAMP (or some

variation) and MEAN.

The LAMP stack (called as such due to its Linux, Apache, MySQL and PHP software base) is one of the most popular

(Ltd, 2017). It is a very flexible stacks with a large number of variations and as such is highly customizable (see appendix

F) (Wodehouse, 2015). The stack is also really easy to install with a large number of servers having a pre-built

configuration for it. There is also a really large support community both for the stack and its individual components

due to the fact it is open source.

The MEAN stack (called as such due to its MongoDB, Express.js, AngularJS and Node.js software base) is a more modern

stack than the LAMP. It is entirely powered by Javascript. The MongoDB is not based on SQL which allows for more

flexibility in its design. However, SQL databases are really powerful when organising massive amounts of structured

data. The MEAN stack is much more equipped for building web applications. However, since this project relies

predominantly on the database side of the stack the LAMP implementation would be more beneficial. Not only is the

SQL based database more suitable for our large amount of data which will all be structured but the large community

behind it will make creating the solution much easier.

5 - Discussion

The chosen concept, of using laser modules and Computer Vision as a dual authentication system is an extremely accurate method of people counting. This will enable the best social benefits. If the building manager can give the firemen information that is close to 100% accurate regarding where people are located in the building, more lives will be saved. As the email from the fire department emphasised, time is crucial when it comes to saving lives. Significantly this project gives precise locations in the building, therefore alleviating the time lost searching rooms where no one is. They arrive at the scene and directly find someone trapped if the situation is ideal. Additionally the concept chosen will give the most accurate empty building data, thus saving more time from alleviating the need to search the building before targeting the fire. If other concepts were chosen, such as using weighing sensors over computer vision, then the information provided to the firemen would never be as accurate as the sensors would pick up trolleys etc. This would have a direct impact on the benefits of the project as searching the building would be as crucial as before the sensors and data were available.

10

Below please see a table explaining any problems that are still to come and the plans to mitigate them.

Development Stage

Potential Problems Mitigation

Sensor Setup

and Data

Processing

Raspberry pi isn’t real time and doesn’t guarantee code will run

Use interrupts to increase reliability of laser detection

Computer

vision

Visibility, changes in lighting affecting reliability.

Thorough testing to ensure reliability over maximum range of visibility

Database Needs test data for development and needs to protect against malicious attacks.

Create a testing system. Everything will be encrypted and the database will sanitise the inputs

Connect the

lasers and CV

together

They add the numbers together and data is no longer accurate. They generate differing data and there is no way to sort which is more reliable.

There will be protocols written before development of the systems.

Sending from

pi to the

database

Running different programming languages. Not being able to send data over the network.

There is a protocol in place so the way the solution should connect is well defined. The system will have backup communication channels and at least one of these should work

Real life

testing

The system needs to work in a building with multiple streams of data all being sent to the database at once. The system needs to also work when there is a large amount of people traffic moving through the door at the same time. The system will also be deployed when an emergency could be occurring within a building e.g. a fire. These will create a harsh environment and the system needs to be able to survive this.

The database protocol will be set up in order to handle all these streams of data and also identify where they are being sent from effectively. It will use two sets of authentication on all data to make sure that it will keep the counts as accurate as possible. The use of multiple sensors and sensor methods means that the system should be able to handle people walking through a door simultaneously. In a fire the system should be able to handle the temperatures up to the point where the fire encompasses the door. At this point communication with a given sensor would be lost to the server since it will be on fire. As such this can be used as extra information to help inform emergency services of the state of the building.

Table 6: Future Potential Problems and Mitigations

11

6 - Conclusion and Future work

The sub teams will be integrated once they have achieved their deliverables under the project plan. It needs to be ensured that the data sent / received different modules of the project are coherent and synchronous, and that there is no possibility of error when these modules interact. This includes the comparison of data for dual verification from the Laser beam and Computer vision, in the Database to keep count of people. This also includes the storage and representation of this data on the website and database.

Once the integration of the modules take place, a prototype door frame needs to be built for testing purposes along with the core parts of the module. The website is under development right now. Once the integration, testing and finalisation of the prototype and the website is done, a presentation needs to be delivered on the current working and future scope of the project along with the prototype.

This design above is only part of what can be achieved in this idea. The use of dual verification is quite accurate in terms of counting the number of people but still has scope for improvement with higher investment in the technology and hardware used; such as the camera quality, laser accuracy, sensitivity of the photoresistors etc.

The representation of the data to the Building Managers and the Emergency has a lot of scope as well. Instead of being a tabulated representation of data, it can be graphically represented on the blueprints of the building plans, making it easier to understand.

12

7 - References AG, A. (2017). Arduino Yún LininoOS. Retrieved from Arduino: https://www.arduino.cc/en/Main/ArduinoBoardYun

Apple Campus 2. (2017, Feb 5). Retrieved from Wikipedia: https://en.wikipedia.org/wiki/Apple_Campus_2

Chiefs, A. o. (2012, Feb). Safety Health and Survival Section. Retrieved from Safety and Health week.org:

http://www.safetyandhealthweek.org/wp-content/uploads/2012/05/Safety_ROE_Lesson_Plans.pdf

Corporation, A. (2016). ATSAMW25. Retrieved from Atmel: http://www.atmel.com/devices/ATSAMW25.aspx

England, P. H. (2014, Aug 8). Laser radiation: safety advice. Retrieved from Gov.uk:

https://www.gov.uk/government/publications/laser-radiation-safety-advice/laser-radiation-safety-advice

Foundation, R. P. (2016, Feb). Pi 3 Model B. Retrieved from Raspberry Pi:

https://www.raspberrypi.org/products/raspberry-pi-3-model-b/

Government, D. f. (2015, Nov 15). Fire and Rescue Response times. Retrieved from Gov.uk:

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/477790/Fire_and_Rescue

_Response_Times_2014-15_Statistical_Release.pdf

Haylock, D. (2015, Apr 21). Footfall: A Camera Based People Counting System for under £60. Retrieved from Wired

Watershed: https://blogs.wcode.org/2015/04/footfall-a-camera-based-people-counting-system-for-under-

60/

Hodge, N. (2015). Allianz assets. Retrieved from Allianz:

http://www.agcs.allianz.com/assets/PDFs/GRD/GRD%20individual%20articles/Power_blackout_risks_article.

pdf

Hussein, O. (2017, Jan). People counting for the Emergency Services.

https://l.facebook.com/l.php?u=https%3A%2F%2Fdocs.google.com%2Fforms%2Fd%2F1tp0KT2YKTvtTMk_V

Wzq_pUfcPaqV7bF05j27hVSlSbg%2Fedit%3Fusp%3Dsharing&h=ATN0YTdIW2U7VkWng_zaMsjcR-

rynZl9bbgh1te9pcpktBl8P8sejMpw56I-pv9GeIsH2F2cOq6zT2xT73tjJTV-8dxy7fNhTwIyOzbautesBx.

Inc., V. T. (2016). What is Computer Vision. Retrieved from Vismatic: http://www.vizmatic.com/expertise

Jerray-Silver, K. (2017, Jan 26). Watch Manager. (L. Tuckey, Interviewer)

Kinsley, H. (2016). MOG Background Reduction OpenCV Python Tutorial. Retrieved from Python Programming:

https://pythonprogramming.net/mog-background-reduction-python-opencv-tutorial/

List of Shortest People. (2017, Jan 14). Retrieved from Wikipedia:

https://en.wikipedia.org/wiki/List_of_shortest_people

Ltd, 1. I. (2017, Oct 16). Web stack—the software framework for web development. Retrieved from 1&1:

https://www.1and1.co.uk/digitalguide/server/know-how/web-stacks-the-basics-and-examples/

Office, H. (2016, Jun 29). Fire Statistics Data Table. Retrieved from Gov.uk:

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/548452/fire-statistics-

data-tables-fire0301.xlsx

OpenCV. (2015, Dec 18). Python Tutorials. Retrieved from Open CV:

http://docs.opencv.org/3.1.0/d6/d00/tutorial_py_root.html

13

team, O. d. (2014, Nov 10). OpenCV-Python Tutorials. Retrieved from OpenCV: http://docs.opencv.org/3.0-

beta/doc/py_tutorials/py_tutorials.html

Wodehouse, C. (2015). Choosing the Right Software Stack. Retrieved from Upwork:

https://www.upwork.com/hiring/development/choosing-the-right-software-stack-for-your-website/

14

8 - Appendix

A) Email conversation with the fire service - Market Research

Hello,

Further to our discussion on the phone on Thursday, I have put together a list of questions that would be really useful for our project if we got your opinion on. It would be beneficial even if the answers are that you don’t think it would be used.

The project we are working on is building a system to reliably count people= inside a building and where they are in the building. This would be displayed on an easy to access website.

1) Would the fire service have time to look at a system like this to see where people are if the building was on fire?

2) What would be the best way to access this information? On a tablet/mobile device app or through a website for example

3) How would you want the information displayed? In a map or list format for example?

Thank you very much for your help,

Kindest regards,

Laura Tuckey

KIM.JERRAY-SILVER@london-fire.gov.uk

26 Jan (12 days ago)

Hey Laura,

Many thanks for putting your questions into email form, I will do my best to answer them :o)

Question 1:

Would the fire service have time to look at a system like this to see where people are if the building was on fire?

Answer:

15

As you can appreciate time is paramount when it comes to saving lives.

When we turn up to a fire in a building many thoughts and questions during our information gathering stage need answering?

Normally when? Where? Why? Who? and how?

The faster +can get these questions answered the faster we can commit fire crews to the effected area.

What works extremely well for us is having an on site manager, duty officer or responsible person meeting us on our arrival with building plans showing us the exact location of the fire, risks that may harm us and the location of any people involved / in need of rescue.

If we have no plans we have a systematic approach starting to search the effected area first and making our way throughout the building from there.

Your idea of letting us know exactly where those people are will of course speed up our search process and avoid searching areas unnecessarily.

So in answer to question 1, yes we would have time and would allocate resources to gather this great information that you have provided.

Question 2: What would be the best way to access this information? On a tablet/mobile device app or through a website for example.

Answer:

As we carry no online technology this would be the hardest part of your great idea.

What the Brigade work with are Premises information boxes where plans, keys, building facility instructions such as mechanical ventilation systems are stored and can only be opened with a Girda key which all frontline fire engines carry. They look the attached document. We do not carry mobile phones or Ipads as yet.

Our 1st port of call when entering a building is to the automatic alarm panel or control centre where we can see which area/zone has been actuated by reading the LED display. These are almost always located at the main entrance to the building.

Any information you would like to provide for us is best stored near to this panel either on the wall or within a folder.

If it was something like an Ipad or something of value the safest place for this would be within the Premises information box as I explained above.

We can make a note of any pre planned information on our basic information monitor called a mobile data terminal which is on the fire engine but this is purely to say that further information for this incident/address is stored in your premise information box.

We can also make a date to visit the site and familiarise ourselves with your particular premises information box or other systems that you have in place.

Unfortunately our on board information monitor does not have the technology to store your idea on.

Question 3:

How would you want the information displayed? In a map or list format for example?

Answer:

The best format for us would be in map form as we are very used to drawing plans/maps and reading them.

I hope that this answers your questions Laura, If you need any further information or for me to explain in more

16

detail any of the areas listed above please feel free to get back to me.

Many thanks and good luck with what sounds like a great project,

Kim. :o)

Watch Manager

Red Watch

Acton Fire Station (G26)

0208 555 1200 Ext 84726

Email disclaimer

The information in this email may contain confidential or privileged materials.

Please read the full email disclaimer notice at london-fire.gov.uk/EmailDisclaimer <http://www.london-fire.gov.uk/EmailDisclaimer.asp>

For fire safety advice please go to london-fire.gov.uk/YourSafety <http://www.london-fire.gov.uk/YourSafety.asp>

17

B) Google form survey results - market research

Figure 1: Market Research Results

C) Criteria for Microcontroller Decision Matrix

Ranked on a scale of 0 to 10, with 10 meaning the device fits the criteria as well as possible.

Main programming language - The ease of use and flexibility (for more sensors etc) of the programming language used and how proficient the team is with it. Pi - Python, ARM - C, Yun - Arduino + Python

GPIO - Number of input/output pins, and how flexible they are (i.e. can they all be used as digital inputs etc.).

Price - The cheaper, the better as each door/room would require one of these devices.

Networking - is there Wi-Fi and ethernet built in, and the ease of transmitting data using the network interface to the database.

Processor speed - Clock speed of the processor, faster is better.

RAM - Ammount of RAM built into the microcontroller, more is better.

Community support - How big is the online community for the device and how much online documentation exists in case of issues that arise during development and testing.

18

Power consumption - How much power is the device going to use in normal operation, can it be put into a low power state at night for example. The less power used the better.

Flexibility - How flexible is the device for testing and further development. Can it support computer vision and laser tripwires simultaneously?

Below is a table justifying the numbers in the decision matrix

Raspberry Pi ARM Cortex MCU Arduino Yun

Main programming language

Python C Arduino + Python

GPIO 40 38 20

Price £32.99 £30 £58.58

Networking Built in Built in Wi-Fi, via SPI Built in

Processor Speed 1.2 GHz 48 MHz 16 MHz + 400 MHz

RAM 1GB 32KB 2.5KB + 64MB

Community Support

Great Small Small

Power Consumption

Biggest Small Small

Flexibility Powerful enough for CV + laser + data sending

Can only do laser + data sending

Can only do laser + data sending

Table 7: Decision Matrix Justification

D) Meeting minutes

Date Minutes

05/12/16 Project concept discussed and various ideas put forward.

Leader and other roles given out, including secretary

Skills within the team discussed.

12/12/16 First meeting with Dr Clarke.

Discussed deliverables and the importance of project planning.

THIS WEEK project plan including assigning tasks and risk management needs to be completed.

19

He also stressed the importance of keeping him involved - weekly updates wanted.

11/01/17

Database Communication

● Send data as a SQL Query

● Send how many people / leaving as a number along an ID.

● Each device to push to central local server, which updates to global server

● Could set up own internal network (Wi-Fi / Subnet Mask)

○ Could we find a way to use their own wireless networking

● Database may want to be probabilistic rather than absolute

● Local server runs SQL database - which is then pushed to web

Processing

● ARM Cortex would be nice to use, but would be too expensive and too difficult to implement in the time we have.

● Maybe best option for commercial production but beyond our limits for this project.

Arduino

● Yun would be good, is built for IoT. Costs £58. Has a Linux engine

● Can run python scripts which can do more advanced processing.

● Has Wi-Fi on board and supports PoE.

● 20 GPIO

Pi

● Raspberry Pi's have more than sufficient processing power and have a large developer community to help provide support.

● Can be used to host a server if we want to.

● B+, lacks wireless so needs USB Wi-Fi.

● 3B, has Wi-Fi. £33. 40 GPIO

Conclusion: Use a Pi 3B

Sensing

May need to use different sensors for different sized doors. Higher traffic doors won't function very well for tripwires. Computer Vision will suit them better.

Computer Vision

● OpenCV runs with Python and will be fine on a Pi

● Webcams are cheap

● Probably the most accurate way to do

20

Concept Plan

● For Proof of Concept, built it for a single door, one person at a time using tripwires.

● When done then make it work for higher traffic doors using a vision system.

Action Points

● Laura to email Tom Clarke about meeting

● Omar to make a slack

● Karmanya to investigate tripwires and sensors

Future Plans

● After meeting with Tom Clarke finalise sub system selection and split into sub teams

13/1/17 SQL for backbone of system, Alex looking into how to process signals sent through the SQL and only accepting predefined data to be accepted to reject external/unauthorised request.

Teams: Web - Database, Web - Front End, Device, Testing

Member Team Role

Omar Oversight Encryption

George Device Implementation

Alex Web - Database Manager

Laura Web - Front End Design

Jacob Laser Implementation

Karmanya Laser Implementation

Zheng Camera Implementation

Samuel Camera Implementation

27/01/16

Laser Laser team need a waveform with edges - their code is edge triggered Need to buy laser! What about smoke? If you have enough smoke will it trigger the laser? Need a laser to go more than 76cm Can only find 80cm lasers maximum Laser pointer is going

21

to the best option ACTION POINT - buy laser pointer

#Computer vision Waiting for webcam for more testing Need to work on exceptions - multiple people at once etc. Would be easier with a white floor

#Web Alex to write a protocol of how data is to be sent to him

#Market research Don't aim to give info to the firemen, just give it to the building manager who has a tablet They want a map of people We can't deliver this in the time Continue

Report ACTION POINT - put mark scheme on google drop People go onto report this weekend and write stuff Use Surface to annotate report mark scheme

LAN/Wireless Don't have a router, so using LAN and 5 way switch

Could request a budget increase for testing purposes

03/02/17

Report

● Need to say how design meets spec points - 5 marks

● Need to show how this is innovative compared to already existing things - 20 marks

○ Bring in firefighter research as it doesn't exist.

● 15 marks are for difficulties we might encounter...

○ Interconnection of CV and lasers, however protocols exist

○ Pi is not real time

○ Implementation section

● 20 marks social economic and environmental

○ Firefighter research

● Cut down most of the microcontroller stuff

○ Show how the numbers are actually coming from

○ Remapping of the numbers

06/02/17 Discussion Regarding meeting with Mrs Perea

● We are further ahead of other groups

● Dr. Clarke might mark us different from the marking scheme due to the that reason

● Need to explain decisions taken against the alternatives that were available

● Need to give reasons for concept designs

● Need to write Conclusions and Further Work

22

● Alex needs to cut down on the Web stuff

● Omar - Abstract

● Laura - References

● Put comments in for References for Laura to deal with during formatting.

● Karmanya - Future Work

● Need more concept text - Omar (fix the part)

● Refer to the first and second meetings

● Detail edit of the Project Plan - everyone

● When is the report actually due on 8th February 2017? Update:

1. Tom Clarke will be marking the report.

2. Page limit is maximum of 9 pages

3. Deadline: 08 February 2017 (Wednesday) - unconfirmed time (something like 2259 hrs)

All content to be put into the Report by 2000 hrs - 7 February 2017 (Tuesday) - STRICT

Tasks have been allotted on Slack

Table 8: Meeting Minutes

E) Computer Vision Code

import numpy as np import cv2 count = 0 im = cv2.VideoCapture('counttest.mp4') fgbg = cv2.createBackgroundSubtractorMOG2(varThreshold=325) # Create some random colors color = np.random.randint(0,255,(100,3)) # Take first frame ret, old_frame = im.read() old_frame = cv2.resize(old_frame,(1920, 1080), interpolation = cv2.INTER_CUBIC) old_fgmask = fgbg.apply(old_frame) old_retval, old_threshold = cv2.threshold(old_fgmask, 10, 255, cv2.THRESH_BINARY) old_median = cv2.medianBlur(old_threshold,15) old_median = cv2.erode(old_median, None, iterations = 1) old_median = cv2.dilate(old_median, None, iterations = 10) # find first contours of moving object image, old_contours, hierarchy = cv2.findContours(old_median,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) if len(old_contours) > 0: try: hierarchy = hierarchy[0]

23

except: hierarchy = [] # computes the bounding box for the first contour, and draws it on the frame, for contour, hier in zip(old_contours, hierarchy): (x,y,w,h) = cv2.boundingRect(contour) if (w > 100) and (h > 200): #To find first centroid of the Car old_x1 = w/2 old_y1 = h/2 old_cx = x+old_x1 old_cy = y+old_y1 old_centroid = (old_cx,old_cy) while(1): # Take second frame ret, frame = im.read() ret, frame = im.read() ret, frame = im.read() frame = cv2.resize(frame,(1920, 1080), interpolation = cv2.INTER_CUBIC) fgmask = fgbg.apply(frame) retval, threshold = cv2.threshold(fgmask, 10, 255, cv2.THRESH_BINARY) median = cv2.medianBlur(threshold,15) befmedian = median median = cv2.erode(median, None, iterations = 2) median = cv2.dilate(median, None, iterations = 10) # find second contours of moving object image, contours, hierarchy = cv2.findContours(median,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) line = 940 cv2.line(frame, (line, 0), (line, 1080), (0,255,0), 4) #draw line if len(contours) > 0: try: hierarchy = hierarchy[0] except: hierarchy = [] # computes the bounding box for the second contour, and draws it on the frame, for contour, hier in zip(contours, hierarchy): (x,y,w,h) = cv2.boundingRect(contour) if (w > 100) and (h > 200): #cv2.rectangle(frame, (x,y), (x+w,y+h), (255, 0, 0), 3) #To find second centroid of the Car x1 = w/2 y1 = h/2 cx = x+x1 cy = y+y1 centroid = (cx,cy) #cv2.circle(frame,(int(old_cx),int(old_cy)),2,(255,0,0),-1) #show coordinates of previous centroid #cv2.circle(frame,(int(cx),int(cy)),2,(0,0,255),-1) #show coordinates of current centroid

24

if abs(cx - old_cx) < 300: if cx > line: if old_cx < line: if h > 900: count += 2 else: count += 1 if cx < line: if old_cx > line: if h > 900: count -= 2 else: count -= 1 #update centroid old_cy = cy old_cx = cx # write number of count font = cv2.FONT_HERSHEY_SIMPLEX cv2.putText(frame, str(count) ,(960,950), font, 3, (255,0,0), 2, cv2.LINE_AA) #cv2.imshow('frame', frame) cv2.imshow('frame', frame) #cv2.imshow('median', median) k = cv2.waitKey(1) & 0Xff if k ==27: break cap.release() cv2.destroyAllWindows()

F) Flexible Stacks

WAMP (Windows/Apache/MySQL/PHP): A Microsoft Windows OS equivalent, it’s all-inclusive and easy to get started with. The WIMP stack is similar but has the IIS server.

LAPP (Linux/Apache/PostgreSQL/PHP): a PostgreSQL database variation that’s optimized for enterprise-level projects.

MAMP (Mac OS X/Apache/MySQL/PHP): A MacOS X operating system variation, it’s available for both Windows and Mac.

XAMPP (Linux, Mac OS X, Windows/Apache/MySQL/PHP, Perl): A more complete bundle, it includes an FTP server, which is cross-platform, able to run on Linux, Windows, and Mac operating systems.

(Wodehouse, 2015)

G) Design Specification

Definitions (DEF)

25

1. LifeCounter is made up of three sub systems,

i. The detection and processing unit, referred to as the device,

ii. The server which collects data from all the devices in a building, referred to as the local server,

iii. The server which collects data from all the local servers and collates prepares them for viewing, referred to as the global server,

iv. The online front end for data display, referred to as the interface

2. Nominal Operation is defined as

i. Detecting the movement of people past the device with the same rate of success as in ideal conditions,

ii. Communicating data from the device to the local server with the same rate of success as in ideal conditions,

iii. Communicating data from the device to the local server within 150% of the time taken to do so as in ideal conditions.

3. Where Ideal Conditions are defined as

i. Room temperature (approx. 20 degrees Celsius),

ii. The device is powered from its primary power supply. See PWR-1.

iii. The building in which the device is located is experiencing low people traffic.

iv. The building in which the device is located is not on fire.

Design Specification

Power (PWR)

1. The device must be able to be powered from standard power means within an office building. This is further referred to as the Primary Power Supply (PPS). This includes but is not limited to

i. 230V mains electrical supply,

ii. Power over Ethernet (PoE) by using installed CAT5 Ethernet cable runs.

2. The device must have some self power system.

i. This system will allow for nominal operation in the event of power failure to the device from the PPS. This is further referred to as the Secondary Power Supply (SPS).

ii. The SPS must be able to maintain nominal operation of the device for at least 60 minutes.

iii. The SPS must be able to recharge from the PPS in the event of depletion of SPS followed by reconnection to the PPS.

iv. The SPS must be replaceable without significant difficulty if the SPS fails.

v. The device should notify the user if the SPS is close to running out of charge.

3. The device must not draw more than 60W from the PPS.

Device (DEV)

4. The device must be able to detect the passing of a person through the doorway.

i. The device must be able to detect which direction the person is passing in.

ii. The device should have an accuracy rate of greater than 95%.

iii. The device must be able to detect the motion of a person of a height greater than 56.4cm.

iv. The device must be able to detect the motion of a person of a shoulder width greater than 35cm.

v. The device should be able to detect multiple people using the same entrance / exit to a room simultaneously.

vi. The device should be able to uniquely count the number of people using a doorway for rates of up to

26

2 people a second.

5. The device must used a detection method that

i. Is completely safe for use through all reasonable interactions with people,

ii. Does not cause damage to the building in which it is installed,

iii. Does not require active interaction from people in order to register their movements.

6. The device should be able to maintain normal detection accuracy even if an object is left in the doorway.

7. The device must communicate data to the the local server.

i. This should happen within 30 seconds of the detection of a person passing through the door.

ii. Data communication to the local server should occur wirelessly.

iii. The device should use existing wireless infrastructure to transmit the data to the local server.

8. The device must be robust.

i. The device should be able to withstand temperatures of 176 degrees Celsius for 4 minutes.

ii. The device should be able to be easily attached to existing infrastructure (door ways etc.) with minimal time and effort.

iii. The device should be able to withstand mechanical interactions with people, for example being kicked or walked in to, and continue to function correctly.

Local Server (LCL)

9. The local server must reject data not sent from a device registered with the system.

10. In the event of the local server loose connection to the global server, the local server should log all data sends from the devices on its network until connectivity has been restored to the global server.

11. Specification points PWR-1, PWR-2 & PWR-3 and their sub points all apply to the local server.

Global Server (GLB)

12. The global server must reject data not sent from a local server registered with the system.

Interface (IFC)

13. The interface must facilitate access to the global server in an easy to use way.

14. The interface should not allow for data in the global server to be changed through the interface.

15. The interface must clearly show the number of people in each room in the building.

16. The interface must only display data to authorised users.

17. The interface should allow for critical data about the device(s) on the system to be shown to the user, including but not limited to

i. The battery level of the SPS,

ii. Connection strength of the wireless network,

iii. If the device is powered from it's PPS or SPS,

iv. Any critical internal errors or faults found on by the device.

18. The interface must work the same on a mobile device (smartphone, tablet etc.) as it does from desktop.

27

H) Project Plan

Task Start Date End Date Duration

Delegated to

Level of Risk Risks

Write Project Plan

12/12/2016

16/12/2016 4 Laura Medium

Too little detail, not able to follow plan; too much detail, too much time spent updating and managing project plan

Research on Raspberry Pi/Arduino/ARM

16/12/2016

09/01/2017 24

George/Omar/Jacob High

If sufficient research is not completed, the wrong interface will be chosen, affecting the entire project

Research on wireless interface

16/12/2016

09/01/2017 24 Samuel Medium

If sufficient research is not completed, the wrong interface will be chosen, affecting the entire project

Research on Servers

16/12/2016

09/01/2017 24 Alex Medium

If sufficient research is not completed, the wrong server will be chosen, affecting the entire project

Research optical sensors

16/12/2016

09/01/2017 24 Laura Medium

If sufficient research is not completed, the wrong sensor will be chosen, affecting the entire project

Research on dual authentication

16/12/2016

09/01/2017 24 Zheng Yang Medium

If sufficient research is not completed, the wrong sensor will be chosen, affecting the entire project

Choose the right options for use

09/01/2017

13/01/2017 4 Everyone High

Not enough evidence is submitted, leading to ill-informed decisions. Omitted by thorough research

Research on Lasers

11/01/2017

13/01/2017 2 Karmanya Medium

Safety research on lasers was required along with a cost approximation and to see compatibility with Arduino

Developing the laser code

14/01/2017

21/02/2017 38

Jacob/George Medium Could be very time consuming

Developing the CV code

14/01/2017

21/02/2017 38

Samuel/Zheng Yang Medium Could be very time consuming

Start cost analysis of the project

14/01/2017

18/01/2017 4 Alex Medium Unable to give an accurate estimate at this point in the project

Start sensor testing(CV)

14/01/2017

20/01/2017 6

Samuel/Zheng Yang Medium

If tests aren't thorough, the wrong sensor layout wont be chosen, impacting future functionality of the project

Database designing

18/01/2017

10/02/2017 23 Alex Medium

Creating the wrong type of database would mean the rest of the project is directly affected

Design interim report structure

18/01/2017

22/01/2017 4 Laura Medium

Mark scheme is not out for report, therefore what is needed is not known

Start drafting interim report

18/01/2017

25/01/2017 7 Everyone Low

If report is not drafted then group input is limited and time pressure is more apparent

Start sensor testing(Laser)

20/01/2017

28/01/2017 8

George/Jacob/Karmanya Medium

If tests aren't thorough, the wrong sensor layout won’t be chosen, impacting future functionality of the project

Website design

20/01/2017

24/01/2017 4 Laura Medium

People getting too involved in the website that it engulfs the entire project

Make interim deadline plans for individuals

20/01/2017

23/01/2017 3 Laura Medium

Checking people are on track is tasking, if the checking system is not usable then the project may lose its timings

Conduct market research

23/01/2017

27/01/2017 4 Laura Medium

Market research with firefighters could give us data to show we are approaching this incorrectly

Purchase Lasers

29/01/2017

02/02/2017 4 Karmanya Low

Late arrival of the lasers could delay the testing of the system and the project has hard time constraints

28

Group review of interim report

31/01/2017

31/01/2017 1 Everyone Medium Conflicting views and writing styles affect the reports fluidity

Final interim report writing

01/02/2017

08/02/2017 7 Everyone Medium Report is not written quickly enough to emit deadline stress levels

Creating private network and servers for the databases to run on

07/02/2017

13/02/2017 6 Alex Medium

Connecting devices together is never as easy as it seems, there is a lot of hardware components and things which work on a test development wont necessary work in a real deployment

Submit Interim report

08/02/2017

08/02/2017 1 Laura Low

Report is not submitted on time, omitted by reminders and calendar entries

Run tests and simulations on the code with sensors

12/02/2017

13/02/2017 1

George/Jacob/Omar Medium Tests are not thorough enough to give conclusions

Bring together the sensor data and code

12/02/2017

15/02/2017 3

George/Jacob Low Code cannot adapt to the sensor data

Create a way to simulate data for large scale testing of the database and test

13/02/2017

18/02/2017 5 Alex Medium

The database needs to work and a way to test it without having to manually input data will be really useful. However a flawed test system will make the database look like it is wring when it isn’t

Start developing dual authentication system

15/02/2017

20/02/2017 5

Omar/Samuel/Zheng Medium

Too many resources going into the second system, so that the first system doesn't work as well

Run tests on accuracy of the sensor triggering

15/02/2017

21/02/2017 6

George/Jacob Medium Tests are not thorough enough to give conclusions

Refine database and increase functionality to include using the large amount of stored data

18/02/2017

18/03/2017 28 Alex High

This although not necessary for implementation is likely to be very hard to do. There is a large amount of data which each sensor will have. Some data will be used in multiple ways. The database will need to do statistical analysis on this data in real time and will also need to display all data in an easy to read format via a web interface. This is the most challenging part although will not stop the project from working

Developing the Pi-Server interface code

18/02/2017

21/02/2017 3

Jacob/George/Alex Medium Could be very time consuming

Start testing transmitting data in extreme conditions (fire)

20/02/2017

28/02/2017 8 Omar High Could contradict the entire concept of the project

Run tests on dual authentication system

21/02/2017

24/02/2017 3

Omar/Samuel/Zheng Medium Tests are not thorough enough to give conclusions

Speed and accessibility tests (from emergency service point of view)

24/02/2017

28/02/2017 4 Omar Medium Tests are not thorough enough to give conclusions

Analyse and evaluate the effectiveness of the project

28/02/2017

02/03/2017 2 Everyone High

Conflicting views on what classes as effectiveness mean analysis not all relevant.

29

Further develop the costing analysis of the project

28/02/2017

02/03/2017 2 Everyone Medium

Conflicting views on expenses for the high end components delay the analysis

Start drafting final report

01/03/2017

08/03/2017 7 Everyone Low

If report is not drafted then group input is limited and time pressure is more apparent

Set up all items for demo

05/03/2017

10/03/2017 5 Everyone Medium Project is not working at this point, delaying preparation for final demo

Group review of final report

05/03/2017

08/03/2017 3 Everyone Medium Conflicting views and writing styles affect the reports fluidity

Rework draft report

08/03/2017

13/03/2017 5 Everyone Medium Report is not written quickly enough to emit deadline stress levels

Website launch

13/03/2017

13/03/2017 1 Alex Medium

Website launch is not effective or on time, affecting the project mark proportionally

Submit Final Report

13/03/2017

13/03/2017 1 Laura Low

Report is not submitted on time, omitted by reminders and calendar entries

Preparation for presentation and demo

13/03/2017

21/03/2017 8 Everyone Medium

There are conflicting views in the team, meaning the preparation is delayed

Presentation and demo

21/03/2017

21/03/2017 1 Everyone Medium

Presentation and demo do not go to plan and project mark is proportionally affected

Table 9: Project Plan

30