Code Monkey home page Code Monkey logo

camtrapml's People

Contributors

bencevans avatar jaivardhan-bhola avatar vlucet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

vlucet ainstitute

camtrapml's Issues

Tensorflow throws a pb related error

Hi! I tried to take this package for a spin and got stuck at the step

from camtrapml.detection.models.megadetector import MegaDetectorV4_1

I get the following message

TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
 3. More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
The full trace
``` Traceback (most recent call last): File "", line 1, in File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/camtrapml/detection/models/megadetector.py", line 23, in from .tensorflow import TF1ODAPIFrozenModel File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/camtrapml/detection/models/tensorflow.py", line 11, in import tensorflow as tf File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/__init__.py", line 37, in from tensorflow.python.tools import module_util as _module_util File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/python/__init__.py", line 37, in from tensorflow.python.eager import context File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/python/eager/context.py", line 29, in from tensorflow.core.framework import function_pb2 File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2 File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2 File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2 File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2 File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in _descriptor.FieldDescriptor( File "/home/vlucet/.cache/pypoetry/virtualenvs/camtrapmltest-FmJ6uYFH-py3.10/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 560, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

</details>

Feature based optional dependencies

The core functionality should be available with pip install camtrapml, e.g. image processing and filtering.

Features that require a more heavy set of dependencies, such as Exif extraction, detection and classification tooling, should require installation as options e.g.

pip install camtrapml[md4]
pip install camtrapml[md5]
pip install camtrapml[exif]

Everything with something along the lines of

pip install camtrapml[md4,md5,exif]

Exif IPTC Keywords Parsing

def parseKeywords(keywords):
    if type(keywords) == str:
        keywords = [keywords]

    def parse(keyword):
        split = keyword.split(':')
        if len(split) == 1:
            return split[0], True
        else:
            key = split[0].strip()
            val = split[1].strip()

        return key, val

    return [parse(keyword) for keyword in keywords]



parseKeywords(['contact1', 'placeID: 01', 'species1: Squirrel'])
parseKeywords('placeID: 01')

Retry Logic for Downloading Models

Fairly often tests are failing due to connection issues when downloading models. Proposal to add retry logic to the download function so that if there's a socket hang up etc. it retries up to three times.

def download(url: str, path: Path, md5_hash: str) -> None:
"""
Downloads a file from a URL to a path.
"""
if path.exists() and (md5_hash == "" or md5_hash == hash_file(path)):
return
path.parent.mkdir(parents=True, exist_ok=True)
resp = get(url, stream=True)
total = int(resp.headers.get("content-length", 0))
with path.open("wb") as file, tqdm(
desc="Downloading Model",
total=total,
unit="iB",
unit_scale=True,
unit_divisor=1024,
) as progress_bar:
for data in resp.iter_content(chunk_size=1024):
size = file.write(data)
progress_bar.update(size)
if md5_hash != "":
assert md5_hash == hash_file(path)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.