Martell Forest Northend Survey Mission Fall 2020

Martell Forest Northend Survey Mission
Fall 2020
Purdue University Forestry & Natural Resources
In association with Purdue University Unmanned Aerial Systems Department
Compiled by: John Cox, Kaleb Gould, James “Jeff” Hines

Executive Summary

For the duration of the Fall 2020 semester the senior class of the Unmanned Aerial Systems (UAS) department have been flying data collection missions for the Purdue University forestry department. These missions were conducted triweekly by four different flight crews. At the conclusion of each week, each flight crew submitted a field report reviewing their flight and highlighting any challenges or complications that arose. These reports, along with the metadata collected during the flights, allow for a comprehensive review of the flight operations.

Mission

Purdue University's forestry department requested UAS surveying flights over Martell forest, shown in figure 1.

Figure 1: Outline of Martell Forest & Study Area

These flights were to be conducted triweekly from September to November. This dataset would allow the forestry department to monitor the change in color of the foliage and provide a more accurate inventory of the property. The collected data would also allow the forestry department to identify invasive species in the study area as shown in figure 1. These flights were conducted with a DJI M600 pro equipped with a Sony A6000 camera.

The desired study area covers 115.89-acres and had to be split into two sections to allow for data collection; yellow shading represents the north-west (NW) flight area and the north-east (NE) flight area in their respective figures.

Figure 2. shows a magnified image of the NW flight area; the area covers 64.80-acres and incorporates grasslands, a variety of tree plots, and a small apiary area. The average flight duration for the NW plot was 21-minutes and 505 photos were collected on average. More information on the flight mission can be found in the flight ops section below.

Figure 3. shows a magnified image of the NE flight area at 59.09 acres; it mainly consists of trees. On average, there were 393 photos taken during each flight and the average duration of the flight was 17-minutes. More information on the flight missions can be found in the flight-operations (flight ops) section below.
Flight Operations

Flight Planning and Crew Roles

The mission area was 115.89 acres. Each mission was flown at 500ft above ground level to avoid colliding with trees reaching upwards of 100ft in the flight area. This altitude provides an excellent field of view for the imagery. The image overlap and sidelap was set at 80 percent to ensure efficient processing. When processing foliage, you need good overlap to help aid the image stitching to create strong orthomosaics of the entire area. With these settings, the plot was divided into two areas. Figure 2 below is a screen shot of the northwest flight plan. It is a 17-minute flight with 11 laps over the 51-acre area. These 11 laps ensure the 80/80 overlap/sidelap that we are looking for. 393 photos are taken with a resolution of 7.8 inches per pixel for the Zenmuse XT2-Thermal camera attached to the plat form.

Figure 2: Measure flight plan for Martell Northwest plot

Figure 3 below is a screen shot of the northeast flight plan. This mission area is 1.5 times larger than the northwest area. The flight time is 20 minutes, expending the majority of battery life. These 65 acres are covered by 16 laps to capture 505 photos with 80% overlap and 80% sidelap. Divided into these two plots provides manageable chunks that can be flown on one batter.

Figure 3: Measure flight plan for Martell Northeast plot

As with any sort of operation of this scale, teamwork and effective crew management is vital to the success of the mission. The flight crew typically consisted of 2-3 people, following the required roles outlined by the Federal Aviation Administration (FAA) for sUAS operations. These are the remote pilot in command (PIC), first officer (FO), and sensor operator (SO). The roles that each member of the flight crew occupied could vary from mission to mission depending on induvial crew members' availability and experience or confidence in their ability to fill the role on a particular flight.

Legal Limitations

Many factors went into completing these flights. While there were no legal limitations faced during the flight, the Martell forest is located near Purdue University Airport, FAA identifier LAF. Fortunately, it is outside of controlled airspace, so flights are free to operate 400 feet over the tallest object in the mission area. The image below shows the airspace limitation adjacent to the area of operation.

Figure 4: KLAF airspace & Martell flight zone

A concern that affected each flight crew was ensuring the aircraft is flight worthy. This is something that gets checked during dispatch and right before takeoff during set up. Other limitations could be the weather, temporary flight restrictions (TFR), and maintaining currency. The weather for an intended flight period is checked with NOAA certified weather services multiple times before and during the flight. The FAA says that we must have 3 statute miles of visibility, fly 500 feet below the clouds and 2,000 feet horizontally from the clouds. A TFR can pop up randomly but must be filed, so these are checked during dispatch too. Finally, a pilot’s certificate never expires, but to exercise that certificate a pilot must pass a biannual flight review every 2 years. They must have the certificate, proof of recurrent training, state photo ID, and the aircraft’s registration certificate available for inspection by any law enforcement upon request. All of these and other federal and state limitations were followed before each flight.

Operating Limitations and No Fly Days

When flights could not be performed, there was a vehicle limitation, weather limitations, safety consideration or technology limitations impleading operation. The aircraft used was a DJI Matrice 600 Pro. It is a very capable vehicle, used for a variety of research mission across the world, but all aircraft, regardless of size or operation, have operational limitations. Shortened to M600 Pro, the research vehicle we use, is able to operate in winds up to 18 mph. Past this, it is expected that the vehicle will be unable to hold its place in space or along a flight path. This has been a recurring problem with the changing of the season. This had cancelled 8 outings throughout late September continuing until the end of November. Another weather limitation is the federal minimum operating limitations as mentioned above. No done is allowed to fly if conditions are worse than 3 statute miles of visibility, or within 500 feet below or within 2,000 feet horizontally of clouds.

These operating limits were organized into our safety management system (SMS) through a series of checklist entries. Before the flight, local weather conditions would be checked using a National Oceanic and Atmospheric Association (NOAA) approved source. The PIC had the final command authority to make a go/no-go decision on the flight based on the operation conditions in the field and their personal limits.

Figure 5: Visualization of Wx Lims

Since safety is the number one factor in all aviation a series of checks happen before, during and after an operation. There is always an assumption of risk but reducing risks as much as possible is as important as the flight. Some flights were canceled because pilots were uncomfortable flying in certain weather conditions. Pilots have personal minimums, either the same as or lower than the ones published by the manufacturer or the government; these are a pilot’s personalized minimum conditions for safe flight. They include rules, criteria, and guidelines to help he pilot decide if, and under what conditions, to operate or continue operating based on the individual's level of experience and qualification.

As equipment is used they wear and scheduled maintenance cycles are a must. In between these events, the aircraft is inspected by the pilots before every flight. If they feel there is something wrong with the vehicle or the equipment they will cancel the flight and immediately inform the maintenance team. These discrepancies are taken very seriously, investigated and the problem is resolved in an efficient and professional manner. These are recorded to look for trends and better service our aircraft. As an example of this, in operation for you we had to remove and replace 2 sets of 6 batteries. After more than 400 flights they had a hard time keeping a charge long enough for us to complete the flights. They were easily pulled from service and replaced with a new set. We will use the information that we have gained from using them to better predict the replacement of batteries in the future based on how we operate.

Lastly, even after seemingly successful flights in the field, there could be underlying issues with the data being corrupted or mismatching. This is found in preprocessing and usually came down to poor ppk data. Fortunately, we have a strong working relation with the company manufacturing our ppk system. They have provided copious amounts of documentation and worked with us for personal special projects so that we can trouble shoot quickly. We can still use the imagery and would review it after every flight to ensure quality and clarity. These problems would require a re-fly of the area to replace lost or damaged data.

Overall Operation

This operation was executed by the senior class of the Unmanned Aerial Systems department. This group of 12 individuals was divided into 4 groups at random. Each of the 3 members assumed a roll and perfected it over the course of the semester. These rolls were pilot in command (PIC), first officer (FO) and sensor operator (SO). Each group flew the same aircraft and the same mission. The data was collected, organized, post-processed and stored in the same way. Since there were 3 flights that needed to be completed in full each week, 3 groups would fly and one would sit out. The intent was to be able to cycle active groups. At the conclusion of each week every flight team wrote up a report for their involvement that week and any troubles they may have faced. This report was used to gauge efficiency and continually review the operation for the best performance.

Over time the groups came together to create documents and guides to help one another and create a uniform way to handle this specific mission. Additionally, students stepped up and took on more responsibility. Roles developed like, dispatcher, fleet manager, maintenance engineer, and data processer. Each roll taking on responsibility for continued safety and support of the mission. This developed it to a strong operation when it was in motion.

Team Performance Review

At the start of the semester the team struggled initially to find its footing. As a group we struggled to learn the content for operating the drones as well as the overall orientation of the class. As a direct result flights were not being made in the quantity that they needed to be. Gradually as the teams became more familiar with the platforms the quantity of flights started to be picked up. Also, as a class there was consistent confusion regarding the level of work needed on the weekly field reports, the root of the issue seemed to stem from a lack of understanding of the mission and its goals. Once the team was able to orient itself, however, the individual groups were able to settle into something of a rhythm with rotating groups for flights so that each week met the required number of flights.

As the mission progressed the team also collaborated in creating documentation to ensure efficient and safe operations. These included a crew resource management (CRM) document, a standard operating procedures (SOP) document, and safety management system (SMS). All of these documents can be viewed in their entirety in the appendixes section below.

Goup 1 consisted of PIC Kaleb Gould, FO John Cox, and SO James “Jeff” Hines. This team averaged 1.8 successful flights a week. They went above and beyond the basic expectation to support the operation. Kaleb worked dispatch and fleet manager to organize outings and ensure the vehicle was available when it was needed, for regular and special missions. He organized other operations to ensure the AT 409 students could go out each week with charged batteries, operational equipment and a flight worthy aircraft. John worked the maintenance department. He serviced the M600 Pro used twice, along with chasing down discrepancies and recording fixes as they came. John headed equipment modifications and sensor integration. Jeff operated with the data, ensuring that it was organized in a timely manner. If there were any problems or holes in the data he would reschedule a flight or coordinate with another group to get it done. Jeff was the spokesperson for group-to-group coordination. This group was always on standby. Frequently they would fill in for other teams with as little as a 10-minute notice. This group is trusted so much that they would be dispatched for operation with a 2-man crew and on 4 occasions for solo, 1-man, operations. There was a clear, constant and reliable line of communication with all persons in relation to the flight side of the operation at all times. Crew resource management was an operational must as they supported and relied on each other for all operations.

Conclusion

Successfully competing 36 mission, flying over 4,000 acers, and collecting over 32,000 photos, this operation has met the requirements of the mission. The problems faced during this semester were not unique to this operation and were delt with in a timely manner. The organizational development within the UAS department shows a newer and stronger understanding of the things needed to complete an extensive mission like this. The things learned here will be used and applied to any future operation. Group 1 showed professionalism and operated with great efficiency. The Fall 2020 Martell Forest survey mission was an extensive 13-week operation utilizing a variety of resources and brought together a collection of professionals.

Appendix A

Forest Progression

Appendix B

Standard Operating Procedure

Vehicle Checkout/Check-in:

1. Flight crews will initially post, date and times for vehicle checkout in google calendar have flights scheduled a week in advance.

a. Flight crew working dispatch Tuesday morning will transfer this data for the week to the whiteboard in comp 101.

i. Include on the whiteboard: light crew, location, vehicle, sensor, date.

2. Check out (flight crew)

a. Digital infrastructure

i. SD card shall be formatted

1. Insert SD card into aircraft sensor

2. navigate to menu

3. Select format option

4. Hit ok

ii. Flight plan

1. Open Measure Ground control app on appropriate interface

2. Sign in using Purdue credentials

3. Select flight plans

4. Select Grid

5. Navigate to “My Saved Plans”

6. Select the flight plan you are flying

7. Ensure that the flight details match the location, altitude, and sensors you will be using for the flight

a. If settings do not match, ensure that you have selected the correct flight plan

b. If the correct flight plan is selected the sensors setting can be changed by using the sensor drop down tab on the flight details screen

8. Hit next

9. Ensure that flight area and settings are correct

a. Flight setting may be changed from this menu if required

10. Once all settings have been verified, select “save flight” to save any changes and close the measure app.

b. Physical Infrastructure

i. Batteries

1. Ensure all batteries in assigned set are charged

a. Press the power button on the batter while it is disconnected from the aircraft, if fully charged all stadia lights will flash

b. If batteries are not fully charged, it is the responsibility of the flight crew to charge them.

2. Place batteries back in their respective cases for transport

ii. Interfaces

1. Ensure all tablets, phones, and or controllers to be used in the flight are charged.

a. Interfaces may be checked out so long as their battery is charged at 70% or more.

b. Place the iPad in its appropriate slot in the foam case (Reference Appx. A Fig 2.)

2. Ensure all connection and charging cables needed for the interfaces are stored in their appropriate positions in the foam case

iii. SD card

1. Ensure that 4 SD cards are available for the flight

a. 1 is for the A6000 sensor, 2 are for the Zenmuse XT2 sensor, and one is for the ppK.

2. Place all SD cards in their protective case

3. Place the SD card case in its appropriate slot in the foam case

iv. Checklists

1. Ensure that appropriate checklists are available in sufficient numbers for the entire flight crew.

2. Checklists should be packed in the case with the aircraft.

v. Sensors

1. Sony A6000 RGB Sensor

a. The A6000 is permanently fixed to the M600 platform

b. Ensure that lenses cover is secure

2. Zenmuse XT2 Multispectral Sensor

a. Retrieve XT2 sensor case from comp 101 office

b. Ensure Sensor is present and undamaged

c. Check SD card slot, ensure that SD card is in the slot

d. Ensure that SD card is formatted

i. Plug SD card into any computer and ensure that any and all data from previous flights have been deleted.

vi. Vehicle cases

1. Open case and ensure that all components are present and undamaged (See Appx. A. Fig. 2)

3. Check out (dispatcher)

a. Digital infrastructure

b. Physical Infrastructure

4. Check in (flight crew)

a. Digital infrastructure

i. SD cards

1. Upon returning, the SO shall retrieve the SD cards and take them to NISW 145 for post-processing. - See SD Card Handling

b. Physical Infrastructure

i. Batteries

1. All used batteries shall be removed from the case and placed on their appropriate chargers

a. M600 batteries are placed on their chargers vertically, with the overhang of the battery facing toward the charger.

b. Once placed ensure that the lights on top of the battery flash sequentially, this indicated that the batteries are charging.

i. If batteries do not start charging, ensure that the batteries are seated in the charger properly

ii. If batteries are seated properly and still do not charge, remove the battery from the charger and notify dispatch.

ii. Interfaces

1. All interfaces shall be removed from the aircraft case and placed on their respective chargers

iii. Sensors

1. Sony A6000 RGB Sensor

a. Ensure that lenses cover is secure

2. Zenmuse XT2 Multispectral Sensor

a. Ensure lenses cover is secure

b. Remove sensor from the aircraft

i. Twist the sensor base until the red and white dots align, the sensor should fall free.

c. Place sensor back in its case and return case to the comp 101 office

iv. Vehicle case

1. Ensure that all equipment in the case is present and placed in its appropriate location

2. Return vehicle case to the comp 101 storage room

5. Check in (dispatcher)

a. Digital infrastructure

b. Physical Infrastructure

File Structure:

1. When returning from a data collection flight, the SO or a stand-in for the SO will take the four SD cards to NISW 145 – See SD Card Handling

2. The person will log into a computer in NISW 145 and access the Classes Drive a. If you do not have access to the Classes Drive:

i. Open the file manager

ii. Right-click on “This PC”

iii. Click “Map Network Drive”

iv. Select the “T:” Drive from the drop-down menu

v. Within the Folder box, enter the following:

\\ecn-usalab.ecn.purdue.edu\Classes

vi. Click “Finish”

3. The person will access the Data Dump

a. Classes Drive --> AT409_Fall20 --> AT409_DataDump

4. Copy and paste the “0_Template” folder back into the DataDump. There should be two template folders in the DataDump.

a. The “0_Template” has the following file structure:

│0_Template

│ Metadata.txt

├───1_Collection

│ ├───Images

│ │ ├───A6000

│ │ ├───XT2_RGB

│ │ └───XT2_Thermal

│ ├───PPK_Data

│ └───Video

├───2_Processing

└───3_Analysis

b. Rename the copied folder according to the following structure:

i. Date_Location_Altitude Flown (in meters)

ii. Ex. 061720_PWA_152m

iii. DO NOT EDIT, ADD TO, OR RENAME THE ORIGINAL TEMPLATE FOLDER

5. Repeat step 4 if multiple plots were flown (I.e., one folder for Martell NW and another for Martell NE)

6. Access the Metadata text file within the new folder and add the appropriate metadata from that day’s flight.

a. If the flight did not require LAANC clearance, put “N/A” under the “Approval #” and “Flight Number” sections

b. Save the document and close it

SD Card Handling:

1. Any SD cards will be transferred between the vehicle and/or sensor within the microSD to SD converter case

2. The 4 SD cards are divided as such:

a. One 16 GB microSD card – GeoSnap PPK

b. Two 32 GB microSD cards – Zenmuse XT2

c. One 128 GB SD card – A6000

3. Insert the PPK’s microSD card into the computer using the microSD to SD converter kept within the PPK case

a. Cut and paste the PPK data from the SD card to the PPK_Data folder

i. There should be four files transferred: A text document, A BIN file, a Chrome HTML document, and a Geotags Application

b. If there are multiple sets of PPK data, reference your metadata and ensure each set of PPK data goes to the proper folder

i. The folder for the PPK data is automatically named in a way where the last 6 digits identify which flight occurred first. A smaller number means the flight occurred earlier in the day.

c. Once the data is transferred, delete the now empty folder from the PPK. The only document that should be left is a file called “CONFIG”

d. Right click and eject the SD card

4. Insert the A6000’s SD card into the computer

a. Locate the SD card in the file manager and access the MEDIA folder

b. If the crew flew multiple flights on one day, the photos will need to be divided according to PPK data

i. Access the first flight of the day’s PPK data by opening the text document under “PPK_Data”

ii. Scroll to the bottom of the text document and note the number of sessions for that flight

iii. Access the photos stored on the SD card once again and select the same number of photos as there were sessions starting from the first photo.

iv. Cut and paste the selected photos into the A6000’s images folder (“1_Collection” → ”Images” → ”A6000”)

v. Repeat the above steps until all the photos from the SD card are properly stored in the correct file location.

1. As a secondary checking method of proper separation, check the “date modified” of the photo set and looking for any egregious inconsistencies between photos in the photo set.

a. For example: a difference of 10+ minutes between two photos might indicate the start of a new flight

vi. If there is an inconsistency between photos collected and PPK session logs, cross reference the photos and session logs again for each flight that day

1. If an error is found, contact Zach Miller a schedule a time to look over the data together

c. Once the MEDIA folder is empty and all the photos are accounted for, delete the folder

d. Right click and eject the SD card

5. Insert the Zenmuse XT2’s microSD card(s) into the microSD to SD converter and then into the computer

a. One microSD card will only have an update log within it. Do not edit this card, simply eject the card and insert the other.

b. Once the photos have been located, ensure the files are displayed line format, not thumbnail format

i. To change/check this, in file explorer look at the bottom right corner of the display and ensure the option that looks like a bulleted list is selected c. Near the top, click on the headers “Date Modified” and “Type” so that like files are together and the time progresses forward

d. Scroll through the files and identify any noteworthy leaps forward in time – these will help you to identify flights

i. The Zenmuse does not have an estimated photo count, so the best way to identify different flights is through the time stamps

e. After identifying the correlation between flights and images, separate them into the proper folders:

i. All JPEG files will go into the “XT2_RGB” folder

ii. All TIFF files will go into the “XT2_Thermal” folder

f. Ensure the microSD card is clear of all files and delete the folder:

i. The only thing left on the card should be the “Update Log”

g. Right click and eject the microSD card.

6. Return all the SD cards to their proper locations as noted in Step 2 of SD Card Handling

Vehicle Transportation:

If a student needs a vehicle to transport flight equipment from the home base - Niswonger Aviation Technology building - to the field, the student will email one of the graduate students associated with the class - William Weldon or Zachary Miller - or the course professor - Dr. Joseph Hupy - to request use of an official Purdue vehicle at least 24 hours in advance of their departure time.

Cross-Team Communication:

1. Individual groups are responsible for setting up their own means of communication outside of class

2. The class will use either outlook or the application GroupMe to communicate between groups

3. If there is a conflict in scheduling it must be disputed by the individual groups’ PICs 24-hours before flight

4. All flights should be scheduled a MINIMUM of 1 week in advance

5 On a week-by-week basis one group will be responsible for dispatch Tuesday morning before lab. This group will also be responsible for writing down the flights every week.

6. When a group finishes a flight, it is their responsibility to send a GroupMe message with a picture of batteries charging to verify they properly stowed the vehicle

7. If a flight is cancelled, they should be held responsible for communicating with the rest of the groups.