Potential for MRC volunteers

to use drone camerasto monitor kelp areas

Overview for the Northwest Straits Commission

John Githensdronespeak1@gmail.com

29 January 2016

1

Presenter background

● 10 years in electronic product design and manufacturing engineering● 1 year promoting Systems Engineering Methods, Boeing Commercial Airplanes ● 27 years in Boeing Information Technology (Sr. Business Systems Architect)● 2012 moved to South Whidbey; Sea kayaker for 25+ years● 2012-15 posts on DIYdrones.com; Custom flying robot systems● 2013 founded DroneSpeakTM.com, a public service website● 2015 Media Partner for InterDrone 2015 in Las Vegas, NV

2Images by John Githens

Surveying kelp in Washington State

● Surveys of kelp species in Washington State started in the late 1800s.○ There has been variation in survey methods, timing, consistency and areas covered.○ The longest continuing aerial survey began in 1989 for the outer coast and Straits, not

including Puget Sound areas East, South, North of Port Townsend (Van Wagenen 20151).

● Survey methods and techniques have included:○ Boats, airplanes, SCUBA divers and sea kayaks.○ Aerial film (visible and infrared) and digital images;

GPS devices; in-water collection devices.○ Professionals (many organizations) and volunteers.

● To fill the gaps for Puget Sound, the Northwest Straits Commission sponsored development of A boat-based survey protocol for Bull Kelp in Puget Sound for use by volunteer kayakers starting Summer 2015 (Bishop 20142).

Photo credit: Northwest Straits Initiative

3

Identifying kelp beds and counting bulbs (2015)

Photo from airplane with outlined kelp beds (G.Ridder)

1. Determining kelp bed perimeter○ Initial kelp bed perimeter identified using aerial

photos taken by an MRC volunteer with a private airplane.

○ Volunteer kayaker records many GPS points to specify the current bed perimeter.

○ Subsequent bed perimeters likely to vary from current GPS points (measured area; position).

2. Determining kelp bulb density○ Kayaker positions their kayak near a GPS

point at one end of the bed.○ Kayaker tries to paddle in a straight line (a

transect) across the kelp plants.○ Kayaker counts kelp bulbs within an estimated

rectangle along the kayak.○ Transects were altered by water current speed

and direction, and other factors.○ Bulb counts were subject to the observer. Hastie Lake GPS kelp bed

perimeter, transects (L.Corin)

Northwest Straits Initiative

4

5

6

7

8

9

kelp bulb

tennis ball

120 feet

● Sunlight Beach, Whidbey Island● Test image (1 video frame)● 50 feet altitude● Sony NEX-5N, 16mm lens● 120’ x 120’ area (approx.)● 10 January 2016● 3 PM

120 feet

Slide by John Githens

Routine monitoring of kelp areas by volunteers

10

1. Doable by volunteer survey teams, with technical support from a consultant.

2. Volunteers operate a small, low-cost, robotic drone.

3. Volunteers process the camera images, and analyze to produce useful data.

4. Need to have a clear vision of:○ What kinds of data to produce. How often during a year. Over how many years.○ How will the data be used. Who will use the data.○ Where to survey/map. Most indicative areas in the counties.○ Measure and analyze data about biomass under the water’s surface?○ Make 3D models of terrain around or near kelp (rocks, beach, bluffs, trees, structures)?

Roles for kelp monitoring volunteers

11

BeachArea Monitor

● Alerts walkers

● Outreach● Assists with

Ground Control Points

Operator / Pilot In Charge

● Preps system● Gets airport

clearance● Conducts flights● Manages

equipment

Optional kayakArea Monitor

● Alerts boaters● Outreach

Survey Coordinator

● Schedules surveys

● Confirmsvolunteerparticipation

Image/data Analyst● Identifies kelp area

boundary, kelp bulb density, other data

● Ensures file/data transfer to SoundIQ

A way to consistently monitor a kelp area

1. Use drone mission planning software and satellite images.

2. Select a drone launch location, and enough GPS points to specify a baseline area along a length of shoreline, extending from Low Water to a depth of -35 ft. MLLW.

3. For efficiency, select a survey grid pattern for the selected camera and drone altitude. Save the flight plan.

4. Load the flight plan to the drone’s autopilot to fly a consistent pattern that records overlapping images.

5. Post-process the images and analyze to develop data about kelp presence in the selected area.

12

Image by John Githens

Potential kelp monitoring system and process1) Transport to survey location

2) Prepare Ground Control Station

3) Prepare drone and cameras at launch/landing point

5) Launch drone to survey kelp area

6) Use cellular service to upload image files to cloud-based service4

that generates one image/map file

7) Analyze5 composite image file on laptop (area data)

4) Program drone’s auto-flight pattern

First four images by 3D Robotics

8) Enter files/data in SoundIQ6

9) Manage equipment items and records

13

Suggested next step

14

● A recommendation from the Science Advisory Committee ○ How aerial images from drones would fit with existing approaches:

■ Images from airplane cameras■ Data from kayaker surveys with GPS devices■ Data availability through SoundIQ

○ Who would collaborate to update the boat-based Protocol.■ NWSC staff, MRC volunteers, WA DNR technical advisors, others

○ Where and when a revised Protocol would be used.

■ Kelp areas at-risk from human activity, natural events or global trends

Additional slides

15

Uncertainties, risks, and mitigations (1 of 3)A) Time for Federal Aviation Administration (FAA) to grant a Section 333 Exemption

- Risk of kelp survey delay. Exemption requested 10 Jan 2016 by DroneSpeak LLC.

B) Time for acceptance by Washington State, County or municipality officials

- Risk of political pressure. Emphasize compliance with FAA regulations, and the use of quick, low-altitude flights primarily over kelp (minimizes risk of harm or reduced privacy).

C) Perceptions of drone use, if negative and expressed by property owners or beach/bluff walkers

- Risk of formal complaints to authorities. Place temporary warning signs. Volunteer monitors explain the project’s environmental purpose, and emphasize its compliance with FAA safety regulations.

D) Physical access to beach or bluff for drone launches/landings

- Risk of damage to equipment while reaching or leaving a beach or bluff. Use padded backpacks.- Risk of insufficient drone communications if using bluff launch/land point. Buy longer-range equipment.

E) Cellular system service in field locations- Risk of delay in accessing a cloud service that creates a composite image from multiple camera images.

Walk or drive to a better location to initiate transfers of images stored by the drone’s camera(s).

16

Uncertainties, risks, and mitigations (2 of 3)

F) Volunteer hours per the schedule of training and survey events- Risk of insufficient number of trained volunteers to fulfill the operations roles (FAA-certified Pilot in

Command (PIC); Survey Area Monitors). Publish the training and survey event schedule 1-2 months ahead. Obtain commitments. Maintain a list of backup volunteers in case of illness.

G) Competency of the system operator and volunteers- Risk of camera preparation failure, ground control station failure, or drone crash due to failure of operator

or PIC to follow all steps and checklists for each flight plan and flight. Maintain complete, easy-to-use system and process documentation. Maintain agreements and VHF communications with local airports.

- Risk of survey area intrusion by people or animals. Place temporary warning signs. Survey area monitors wear labeled, reflective vests, while asking walkers to wait a few minutes until the drone flight is complete.

H) Weather conditions- Risk of system damage due to high wind or rain. Cancel event if conditions are excessive.

I) Project resources- Risk of inadequate funds, volunteer hours or partner hours to achieve all NTA objectives, outcomes and

outputs. Periodic reviews with an Advisory Panel and the participating Marine Resources Committees.17

Uncertainties, risks, and mitigations (3 of 3)

J) Image quality- Risk of excessive sun glare from water surfaces. Adjust the drone’s flight pattern to minimize glare.

- Risk of digital anomalies in the composite images of kelp areas. Buy camera(s) with sufficient resolution from the anticipated drone altitude. Maintain constant speed and altitude during flight to maintain focus on kelp. Ensure sufficient overlap in source images. Optionally, use RAW images.

K) Liability insurance- Risk of legal action due to an incident or accident that results in harm to people or property. Operator to

carry sufficient insurance, paid by project funds. Volunteer insurance coverage to be determined.L) Property management

- Risk of harm due to handling of components of the Kelp Monitoring System. Train volunteers in proper handling of components. After field use, return all items to designated storage locations, followed by evaluation, maintenance or repairs per documented practices. Avoid harmful weather conditions.

- Risk of loss of components during operation of the Kelp Monitoring System (example: drone crash on land or in water). Provide sufficient system training and operation opportunities for the operator and volunteers. Purchase additional drones, batteries and repair parts.

18

Reference links

1. Washington Coast Kelp Resources, Port Townsend to the Columbia River, Summer 2014 (Van Wagenen 2015) 2. A boat-based survey protocol for Bull Kelp in Puget Sound (Bishop 2014)3. Maptalker.org, personal blog of Keith Holmes4. DroneDeploy5. Pix4D Mapper Pro6. Sound IQ Data System7. MAPIR Infrared, Green, or other Camera8. MAPIR Dual Static Mount9. 3D Robotics Solo Drone and Controller Package (no gimbal or GoPro)

10. Kelp forest mapping by use of airborne hyperspectral imager (Volent 2007)11. Remote monitoring of giant kelp biomass and physiological condition: An evaluation of the potential for the

Hyperspectral Infrared Imager (HyspIRI) mission (Bell 2015)12. FAA Section 333 Exemption13. 2016 Action Agenda Update: Near Term Action Proposals (Puget Sound Partnership 2015)14. Puget Sound Partnership’s Strategic Initiatives: Stormwater, Habitat, Shellfish, Crosscutting Substrategies15. Puget Sound Vital Signs

19

Imaging system alternatives

1. An airplane-borne camera flown more often, at lower altitudesa. Volunteer pilot obtains images using personal airplane and mounted camera, fying at ~2500 feet.

b. Kelp bulb counting may be inhibited by sun glare or image resolution. Do more frequent, lower altitude flights? (noise? privacy?)

2. A consumer remote-controlled quadcopter with GoPro (DJI Phantom 3 Pro)

a. Current DJI products require a skilled pilot to guide the drone while it records waypoints for a future flight pattern.

3. A commercial fixed-wing robotic drone (component of a precision agriculture package)

a. Fixed-wing drones appear to be inappropriate for volunteer operation over some shoreline or kelp areas.

4. A commercial multirotor robotic drone (component of an aerial mapping package)a. From a supplier experienced with robotic drone technology (programmed agriculture and infrastructure surveying and mapping).

b. Relatively easy to setup, operate, maintain and repair at the selected kelp areas.

20

‘Kelp area’ images/data: Good enough for what?

1. An RGB (visible light) image that enables a volunteer to:a. count individual bulbs of floating kelp (Nereocystis luetkeana).b. draw polygons around dark areas, to estimate plant density.c. draw polygons including Nereocystis luetkeana or Macrocystis pyriferad. use software to count kelp bulbs or create polygons to estimate plant density.e. use software to create polygons of Nereocystis luetkeana or Macrocystis pyrifera.

2. An Infrared, Near-Infrared or other image that enables a volunteer to:a. draw polygons around dark areas, to estimate plant density.b. draw polygons including Nereocystis luetkeana and Macrocystis pyriferac. use software to create polygons of Nereocystis luetkeana and Macrocystis pyrifera.d. use software to estimate the density of all floating and submerged biomatter in the area.

3. A 3D model of the terrain around or near a kelp area (rocks, beach, bluffs, trees, structures)?4. What degree of “ground/water-truth” precision is enough to help with Puget Sound Recovery?

21

Are GCPs essential?Image by Trimble

Initial Kelp Monitoring System (KMS) components

● 3DR Solo Backpack (x2)● Micro SD Cards (x5)● Camera lens protectors (x2)● Batteries (Camera, Drone, GCS)● Battery chargers● Replacement propellers● Other replacement parts● Long Range GCS Antenna● Cellular dongle and service● VHF radio for airport contact● Quality GPS device

SurveyParticipants

● Coordinator● Operator/PIC● Area Monitors● Image/Data

Analyst

MAPIR‘Visible’Camera

MAPIR‘Infrared’Camera7

MAPIRDual

Mount8

3DR Solo Drone & Controller9

Pix4DSoftware5

LaptopComputer

● Perimeter warning signs● Whistles, Walkie-talkies● Tripod; lightweight stand● Ground control point materials● Ground tarps (launch/land)● Operation, maintenance,

repair documentation22