weecology / deeplidar Goto Github PK

LIDAR and RGB Deep Learning Model for Individual Tree Segmentation

Python 93.33% Shell 6.67%

deeplidar's Introduction

This repository is archived. The model used in this research has been re-written in PyTorch and is now maintained as part of the DeepForest package. Please visit that repository to build on this work. Given the speed of development in deep learning and computer vision packages this code is now unlikely to run without the specific package versions from 2019.

Geographic Generalization in Airborne RGB Deep Learning Tree Detection

Ben. G. Weinstein, Sergio Marconi, Stephanie Bohlman, Alina Zare, Ethan White

Summary

DeepLidar is a keras retinanet implementation for predicting individual tree crowns in RGB imagery.

How can I train new data?

DeepLidar uses a semi-supervised framework for model training. For generating lidar-derived training data see (). I recommend using a conda environments to manage python dependencies.

Create conda environment and install dependencies

conda env create --name DeepForest -f=generic_environment.yml

Clone the fork of the retinanet repo and install in local environment

conda activate DeepForest
git clone https://github.com/bw4sz/keras-retinanet
cd keras-retinanet
pip install .

Update config paths

All paths are hard coded into _config.yml

Train new model with new hand annotations

python train.py --retrain

How can I use pre-built models to predict new images.

Check out a demo ipython notebook: https://github.com/weecology/DeepLidar/tree/master/demo

Where are the data?

The Neon Trees Benchmark dataset is soon to be published. All are welcome to use it. Currently under curation (in progress): https://github.com/weecology/NeonTreeEvaluation/

For a static version of the dataset that reflects annotations at the time of submission, see dropbox link here

Published articles

Our first article was published in Remote Sensing and can be found here.

This codebase is constantly evolving and improving. To access the code at the time of publication, see Releases. The results of the full model can be found on our comet page.

deeplidar's People

Contributors

Stargazers

Watchers

Forkers

nnu-gisa pclc7z2 cczls parkecogeo cuggisrs riowong azh2 fvangef abdurrasith hjkluo ergincagataycankaya okikiola-michael

deeplidar's Issues

HSI 20 or 40x40 clips may be too little for anchors to work properly

for what I understood an anchor is considered positive if the IoU>=0.5 with a hadlabelled box. This could be problematic for hiperspectral clips where the resolution is too coarse and it is too easy to drop down the 0.5 threshold. we may want to: (a) increase the resolution of the hipserspectral by fusing it with RGB (https://pdfs.semanticscholar.org/f4c9/0b4a685ea0ffd19ca7a9ef6e9dd913a327ec.pdf); (2) or reduce the anchors IoU threshold?

TypeError: Failed to convert object of type <class 'theano.tensor.var.TensorVariable'> to Tensor

Hello,
even if I change os.environ['KERAS_BACKEND']='tensorflow', each time the same error.
Further: Contents: Elemwise{add,no_inplace}.0. Consider casting elements to a supported type.

Thanks

Protect against spatial autocorrelation in the evaluation plots

Remove any 1km2 tile that contains an evaluation plot from the pretraining. This should atleast speak to Alina's concern of potential spatial autocorrelation in forest conditions leading to inflated results. a conservative choice.

How much local training data is needed?

Create training sequence to test data sizes.

Car

Move to train models into the config, makes it easier for all of this analysis framework. Make a new analysis script for Hannah and a Tatian only performance. Consider transfer learning with Hannah and a Tatian’s from other sites. Does it matter if it goes in the first training, or in the patches with the evaluation Hannah Tatian’s.

Postprocessing test

if there isn't much training value in the canopy height model, can we do away with LIDAR completely in the postprocessing?

branch a non-blended version

branch a non blended
correct on-the-fly generator
recompute samples (dask generate)
rerun prcurve
Overlay prcurve figure.

cannot convert float NaN to integer

in Generate.run()

distributed.worker - WARNING -  Compute Failed
Function:  execute_task
args:      ((<function apply at 0x2ae045fcc840>, <function run at 0x2ae0e5a9d8c8>, (<class 'tuple'>, ['/home/b.weinstein/TreeSegmentation/analysis/Results/detection_boxes/TEAK/2018/NEON_D17_TEAK_DP1_315000_4091000_classified_point_cloud_colorized.csv']), {'DeepForest_config': {'num_hipergator_workers': 10, 'h5_dir': '/orange/ewhite/b.weinstein/training_h5s/TEAK/', 'patch_size': 400, 'patch_overlap': 0.05, 'rgb_res': 0.1, 'preprocess': {'zero_area': True}, 'training_csvs': '/home/b.weinstein/TreeSegmentation/analysis/Results/detection_boxes/TEAK/2018/', 'rgb_tile_dir': '/orange/ewhite/NeonData/TEAK/DP3.30010.001/2018/FullSite/D17/2018_TEAK_3/L3/Camera/Mosaic/V01/', 'training_images': 1000, 'snapshot': 'None', 'batch_size': 6, 'epochs': 5, 'save_snapshot_path': '/orange/ewhite/b.weinstein/retinanet/snapshots/', 'backbone': 'resnet50', 'shuffle_training': False, 'single_tile': False, 'score_threshold': 0.15, 'evaluation_site': 'SJER', 'evaluation_images': 50, 'evaluation_tile_dir': '/orange/ewhite/N
kwargs:    {}
Exception: ValueError('cannot convert float NaN to integer',)

Tree Segmentation

In create lidar plots.
"/orange/ewhite/b.weinstein/NEON/SJER/NEONPlots/Lidar/SJER_056.laz"

No point found for within region of interest 1.
Error in lidR::writeLAS(clipped_las, cname) :
  Cannot write a file with 0 points

move preprocess data to /ufrc/

will need to edit config paths.

Laz regular expression

What if there are two crops from the same tile?

laz_files
['/Users/Ben/Documents/DeepLidar/data/MLBS/training/NEON_D07_MLBS_DP1_541000_4140000_classified_point_cloud_crop2.laz', '/Users/Ben/Documents/DeepLidar/data/MLBS/training/NEON_D07_MLBS_DP1_541000_4140000_classified_point_cloud_crop.laz']
row
{'tile': '2018_MLBS_3_541000_4140000_image_crop2.tif', 'window': 0, 'site': 'MLBS'}

see Lidar.find_lidar_tile

Data augmentation

During re-training can I do rotational augmentation, just the hand annotations