Tech, music, and more

Author: uuwdrmmy

Fall Reading Update

I finished Open Water by Caleb Azumah Nelson. A touching story about the effects of racial dynamics and disconnection from family on relationships.

I am currently reading How to Solve It, by George Pólya

and Finding the Mother Tree by Suzanne Simard, an incredible narrative on her journey to discover how trees and fungi link together forests.

Part 4: Second Mapping Test

The…interesting results from my first mapping flight indicate that I need to have even more overlap between images. For my second flight, I dialed up the percentages on the Mission Planner to 95% overlap and 80% sidelap. I also set up the Raspberry Pis to run the image capture scripts as a service (systemd) on boot.

Here is a link to the github repository with the updated python scripts I run on the pis: https://github.com/Bobcati/BLD_NDVI_Scripts

More images and overlap

After the flight, I put the photos into photoshop image merge again to see if I had enough overlap. I think the results speak for themselves:

This image has some blurriness on the top right
This photomerge came out very well

Here are the links to the full resolution images:

IR Photo: https://drive.google.com/file/d/1OZyCyW7UsAowfsIsTLQ5eUnsoYk_OpFJ/view?usp=sharing

RGB Photo: https://drive.google.com/file/d/1K13roW3xFs-y1rMmT7aaTy99zUDu_ExO/view?usp=sharing

Issues

Unfortunately, the Pi Zero W took one fewer image than the Pi 4B did (Zero took 77, 4B took 78). I dialed down the resolution of the pi zero Additionally, in order to put the images into GIS software, I have to geotag them with the GPS coordinates stored in the flash memory on the drone. Hopefully the Pi 4B took as many images as the drone triggered, otherwise I will not be able to geotag.

Next Steps

  1. I will attempt to run the NDVI script on the infrared merged image. Hopefully I don’t run out of RAM.
  2. Geotag infrared images.
  3. (longer term) Getting telemetry working over 4G using Tailscale instead of my prior plan of using port forwarding and reverse ssh.

Actually, why wait. I’m writing this post and I can run an NDVI analysis right now. The code I use is an altered version of the formula:

Well, I’ve run the analysis and it all worked out. For some reason, when I run an NDVI analysis using the blue filter, the results often make the shadows seem like they are reflecting the most IR light. The red filter method seems to produce better results. I need to do more testing with a camera that has a proper red filter (stay tuned…).

Method for blue filter NIR camera
Method for red filter NIR camera

Here are the results:

NDVI with the red channel as the NIR channel
NDVI with the blue channel as the NIR channel

Red as NIR channel full res: https://drive.google.com/file/d/1W7WYLnQ18g7xPWomkYj4CvAvjNDVCz5C/view?usp=sharing

Blue as NIR channel full res: https://drive.google.com/file/d/1IsNUrNv0EhQxVk-itpbDc_DQoc5xgdeC/view?usp=sharing

Although some of the beech trees are infected with BLD, the disease has not progressed enough to create clear gaps in the tree canopy. Additionally, many of the beech trees have put out another batch of leaves, making the canopy appear healthy again.

My Summer Reading

Every summer I like to read a lot of books across a variety of genres. I also enjoy re-reading books because I often see new insights and connections.

My Summer Reading

  • The Immortal Life of Henrietta Lacks by Rebecca Skloot

Skloot paints a multidimensional portrait of the dual nature of medical advances. She chronicles the stories of exploitation of Black Americans in medicine through the story of Henrietta Lacks and her cervical tumor cells (HeLa cells). Skloot uses her journey of forming a relationship with the Lacks family to put all the medical advances and doctors she describes into a broader context. I will always remember that honoring the person that a sample came from is as equally important as the research itself.

  • Memphis by Tara M. Stringfellow

A beautiful cross-generational story centered in the city of Memphis, Tennessee in the south of the United States that covers themes from the legacy of segregation to the intimate struggles of a young female artist as she grapples with the expectations of her mother.

  • Moonwalking With Einstein by Joshua Foer

An autobiographical account of how Foer learned to improve his memory, but from the perspective of a science journalist. I will always remember the concept of a memory palace. Instead of rote memorization, I should think of a place I am very familiar with, like my house, and store information in the form of unique images that occur at certain spaces as I walk though the house.

  • My Beloved World by Sonia Sotomayor
  • (Currently Reading) Open Water by Caleb Azumah Nelson

Part 3: First Autonomous Mapping Tests

For the past two weeks, I have been working on building an autonomous drone to run mapping missions. I connected two Raspberry Pis to the drone to take pictures using the Raspberry Pi Cam v2. I ran into several issues with my receiver not connecting and GPIO pins not triggering, but I managed to come up with workarounds for each problem. I used the Ardupilot Mission Planner software to operate the drone: https://ardupilot.org/planner/.

Parts list

Total Cost: $887

Setup

Here is a quick breakdown of the build setup.

Completed Build
NoIR cam and RGB Camera
Inside look at raspberry pi wiring

The Pololu 5v 5.5 A voltage regulator provides power to the servo rail on the flight controller and both Raspberry Pis (4b and Zero W). This voltage regulator has an XT30 plug and is powered off of a LiPo battery that is separate from the one that powers the drone. The flight controller connects to the Pi 4B’s GPIO and has a camera trigger. The Pi 4B (with LTE HAT attached on top right) provides power to the Zero W. I also connected one of the 4B’s GPIO pins to the Zero W to act as a secondary camera shutter relay. The Pi 4B is attached to the NoIR camera and the Zero W is attached to the RGB camera.

Stay tuned for another blog post with a more in-depth technical breakdown of the drone build.

Testing Highlights

Watch this video:

Map Construction

The test flights were a success and the cameras were a partial success. My RGB camera was slightly out of focus for the tree pictures. I also set the angle incorrectly on Mission Planner so the images did not have enough overlap. I still gave it a go using the auto image merge in photoshop (aka the Brenizer Method): https://digital-photography-school.com/5-steps-to-rock-the-brenizer-method/

The results are promising. The top left image is from the first test flight of a field, which looks pretty good. After that first flight, I ran another test flight above the trees. The tree images in the second flight got fractured into separate parts, which means I need to take more images during each flight.

Future Plans – Short Term

Now that I trust the drone to fly over trees on its own, I am going to set up another, longer mapping mission where I look at some diseased beech trees.

I am also going to set up the raspberry pi’s to automatically start run the python programs at boot using systemd.

Then I’ll geotag the images and put it into photogrammetry software (webODM) to construct a map from the images. I’m also going to lower the resolution of the RGB photos because the Zero W is a slower at taking pictures so it often misses picture triggers and is out of sync with the 4B.

Future Plans – Long Term

I want to get the 4G telemetry working on the raspberry pi. The raspberry pi can send telemetry data from the drone directly to my laptop so that I do not need to use the telemetry radios. However, I have to set up port forwarding and I have not had access to a router to do so. I have an idea to use a sim card router and set up dynamic DNS…stay tuned.

Tutorials I am using:

Raspberry Pi Pixhawk drone: https://www.youtube.com/watch?v=kB9YyG2V-nA

Setting up Telemetry over 4G: https://www.youtube.com/watch?v=IokyotAGbJI

Using port forwarding on a https://www.youtube.com/watch?v=2d5HfU39tyM

Part 2: Initial NDVI Tests

Field Testing

I decided to run some field tests using hardware I already had in hand.

Parts list:

  1. FPV Drone
  2. Raspberry Pi Zero W
  3. Raspberry Pi NoIR cam v2
  4. Camera Case
  5. Zip Ties
  6. Anker Power Supply

Note: I tried soldering the Pi Zero W directly to the drone flight controller, but it drew too many amps and the drone kept rebooting (this started happening after my first flight). I also tried to trigger the camera using the drone radio transmitter by connecting a GPIO pin of the Pi Zero W directly to the flight controller. I will write a separate blog post if I can get this to work.

I took my Raspberry Pi Zero W, attached a NoIR camera, put it in a camera case, and zip tied it to the bottom of my 5″ FPV drone. I wrote a python script to make the zero w automatically take pictures every couple seconds.

Code with GPIO trigger

from picamera2 import Picamera2
import RPi.GPIO as GPIO
from time import sleep
from time import strftime

picam2 = Picamera2()
camera_config = picam2.create_still_configuration(main={"size": (3280, 2464)})
picam2.configure(camera_config)
picam2.start(show_preview=False)

GPIO.setmode(GPIO.BCM)
GPIO.setup(23, GPIO.IN)

def debounce():
    sleep(0.06)

#Time stamp each image
def takePicture():
    print("picture captured")
    picam2.start_and_capture_file("Plant_Test_Pics/" + strftime("%d/%m/%y at %I:%M%p:%S:%f") + ".jpg")

while True:
    time.sleep(0.01)
    if GPIO.input(23):
        takePicture()
        debounce()

Time based image trigger

from picamera2 import Picamera2
import RPi.GPIO as GPIO
from time import sleep
from time import strftime

picam2 = Picamera2()
camera_config = picam2.create_still_configuration(main={"size": (3280, 2464)})
picam2.configure(camera_config)
picam2.start(show_preview=False)

def takePicture():
    print("picture captured")
    picam2.start_and_capture_file("Plant_Test_Pics/" + strftime("%d/%m/%y at %I:%M%p:%S:%f") + ".jpg")

while True:
    time.sleep(2)
    takePicture()

All of my code is on my github repository:

https://github.com/Bobcati/BLD_NDVI_Scripts/tree/main

Last minute Anker power supply addition

I also recorded some GoPro footage to go along with the Infrared (IR) images I took. Check out this video:

Here is a slideshow with some of the infrared photos I took with NDVI processing and unfiltered IR images comparisons.

Part 1: Starting a Research Project

Step 0 – The Background

Beech Leaf Disease (BLD) has been infecting beech trees across the Northeast of the United States. This disease impacts the leaves of beech trees and can cause tree mortality. I have a personal connection to this because the trees where I grew up are mostly beech. I want to track this disease in order to ensure the beech forests are kept safe. This summer, it has been devastating to see how unhealthy the beech tree canopy is.

Beech leaves with symptoms of BLD

Step 1 – Questions

The first step to every research project is generating questions. I like these initial questions to be open-ended inquiries. In particular, I look back at previous projects to see how I can apply the tools/skills I learned to my current research. I wanted to contribute to research on using drones to map tree canopies.

For this project, I divided my project into two areas: researching beech leaf disease itself and researching methods to analyze vegetation using aerial photography.

BLD Question

Where did BLD originate?

What spreads BLD?

What is the current state of the trees with BLD?

What path does BLD take when spreading between trees?

What are the current studies on BLD?

Drone Mapping Questions

What are the current methods for analyzing plant health with images?

What hardware will I need to use?

What software will I need to use for mapping?

What studies have been done using drone mapping to detect tree disease?

Step 2 – Initial Research

I then find resources to start answering my questions. I like to put all the hyperlinks I find into a document.

For example, here are the initial hyperlinks and resources to answer the drone-mapping questions: https://docs.google.com/document/d/1lnaTr6k-asLfqFJ-mOqyBl3UcgFEhXrW8v_NtLyYdF8/edit?usp=sharing

After doing the initial research, I take a step back and isolate the major takeaways. Some of the key points I found were:

  1. Normalized Difference Vegetation Index (NDVI) will be key.
  2. I want to take the Open-Source route in terms of software for now
  3. I need to put together a parts list for the drone I plan to build
  4. I can already start doing image processing by looking at satellite imagery

I put academic research papers into their own document. I also continue to add resources and hyperlinks to the initial research document every time I come up with a new idea or learn about a new resource.

Initial resources on BLD:

The PicoGamer Glove

I recently got a Raspberry Pi Pico W because I wanted to try out making wearable technology. The Pico is a microcontroller similar to an arduino. I like the Pico because I can program it using Python.

I recreated alien invaders using java for my final class project. I wanted to add a more immersive experience to playing the game. I did some research into microcontrollers that can emulate Human Interface Devices (HID). Joysticks, keyboards, and mice are some examples of HID devices. There are HID libraries for the Pico that enable it to emulate a keyboard, joystick, and even multimedia controls. Each finger acts as a momentary switch that is triggered when it comes into contact with my thumb.

List of Materials

To build this glove, I used the following:

  1. Raspberry Pi Pico – $4 or Pico W – $6:
  2. Headers for the Pico (alternatively, you can buy a Pico with the headers pre-soldered, but where’s the fun in that?)
  3. Inertial Measurement Unit (IMU) MPU 6050 – $13
  4. Jumper Wires
  5. Soldering iron
  6. Solder
  7. Aluminum Foil
  8. Tape
  9. Old Glove
  10. Scissors
  11. Needle and Thread

All of the code and libraries I wrote/used are on this Github repository: https://github.com/Bobcati/pico-glove-rev1

Building the Glove

Cut five pieces of aluminum foil with the dimensions of about 2 cm by 12 cm. Take 4 jumper wires of any color other than red or black and strip about 1 cm of insulation off. Do the same for 4 black jumper wires. For the black wires, tape all four of them onto one strip of aluminum foil. For the other four wires, tape each of the exposed wires to a strip of aluminum foil like so:

Wrap the tin foil strip with the black wires around the thumb. Then wrap each tinfoil piece around a finger on the glove.

Sew the MPU6050 IMU onto the back of the glove using a needle and some black thread.

On the MPU6050, attach a yellow jumper wire to the SCL pin, a white wire to the SDA pin, a red wire to the VIN pin, and a black wire to the GND pin.

Connect each finger to the corresponding GPIO pin on the Pico. The ring finger should attach to GP6, the pointer finger to GP9, the pinky finger to GP21, and the middle finger to GP2. Connect each of the black wires attached to the thumb to the GND pins closest to each GPIO pin the finger wires are plugged into.

Take the MPU6050 and connect the SCL (yellow) to GP15 and SDA (white) to GP14 on the pico.

Setting up the Pico

I had to install the CircuitPython firmware to use my pico with the Adafruit libraries. Adafruit is a great company that builds every sensor or HAT imaginable for SBC or microcontroller related projects.

Hold down the BOOTSEL button on your Pico while plugging it in to a computer.

For the regular Pico, download the CircuitPython firmware here: https://circuitpython.org/board/raspberry_pi_pico/

For the Pico W, download the CircuitPython firmware here: https://circuitpython.org/board/raspberry_pi_pico_w/

Watch this video…

or follow along with this tutorial to install the firmware (skip the first step – the firmware is already downloaded): https://learn.adafruit.com/getting-started-with-raspberry-pi-pico-circuitpython/circuitpython

In order to make the Pico function as an HID device and read sensor input from the MPU6050, I had to add several libraries to the Pico. Every Pico has a “lib” folder, where all of the libraries must be stored. The Pico acts like an external USB flash drive when it is plugged into a computer labeled “CIRCUITPY.” Download the files from my github repository and paste them into the “CIRCUITPY” drive.

Et Voila: when the Pico is connected to the computer, the arrow keys will be triggered depending on hand tilt and tapping the thumb and pointer finger together will trigger the space key.

Tapping the pinky switches to manual mode, where the middle finger becomes the right arrow key and the ring finger becomes the left arrow key.

To edit programs on the Pico, download either the Thonny IDE: https://github.com/thonny/thonny/releases/tag/v4.1.1

or the Mu code editor:

https://codewith.mu/

© 2024 sukkendi

Theme by Anders NorenUp ↑