Code Monkey home page Code Monkey logo

Comments (13)

altear avatar altear commented on June 10, 2024 1

Heya, what's the status on this? It seems like it's pretty much ready to go

I made a few changes and can run the testscript in tf2.2 (minus tensorpack, which hangs in my ubuntu docker env). Changes: https://github.com/rat-emotion/DeepLabCut-core/commit/f532ebbb7b9c8d2827ad79ba2a565b6d272d52b9

p.s. I'm interested in contributing but haven't touched open source before, if it's useful I'm happy to PR those changes

from deeplabcut-core.

altear avatar altear commented on June 10, 2024 1

Awesome!
The export_model issues seems to be related to freeze_graph which a lot of people have been having troubles with in tf2

Relevant: https://leimao.github.io/blog/Save-Load-Inference-From-TF2-Frozen-Graph/

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024 1

awe, that's too bad! It's a rather core feature of DLC now ... I don't have more time today, but I'll look into it (or please feel free! :) Thanks again!)

from deeplabcut-core.

AlexEMG avatar AlexEMG commented on June 10, 2024 1

This seems to be a great resource: https://github.com/leimao/Frozen_Graph_TensorFlow/tree/master/TensorFlow_v2

@gkane26 do you want to have a look at the export?

from deeplabcut-core.

gkane26 avatar gkane26 commented on June 10, 2024 1

I took a quick stab at the model export with tf 2.3 on my macbook -- it took some minimal changes -- the freeze_graph function was throwing an error that I didn't see a way around (see tensorflow/tensorflow#24591). What did work was using the convert_variables_to_constants function (see the question here https://stackoverflow.com/questions/55299995/exporting-a-frozen-graph-pb-file-in-tensorflow-2). In addition to that change, there were some package imports that needed to be changed (tf_slim which seems to be required for mobilenets is now its own package), but otherwise things look good!

from deeplabcut-core.

altear avatar altear commented on June 10, 2024

The save_format='h5' option mentioned in the short list warnings seems to refer to Model.save and isn't part of tf.train.Saver so I removed it

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

Hey! I agree it’s very close! If you have edits that make it fly, please do make a PR :). You can comment on what you changed overall, and we will review it. Thanks for contributing! Much appreciated!!!

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

few remaining issues after PR #8: see--> #8 (comment)

(1) warnings w/default loader, but it does train w/default and with imgaug on tensorflow 2.2:

WARNING:tensorflow:From /Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From /Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1666: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/nnet/losses.py:38: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
Loading ImageNet-pretrained resnet_50
Training parameter:
{'stride': 8.0, 'weigh_part_predictions': False, 'weigh_negatives': False, 'fg_fraction': 0.25, 'weigh_only_present_joints': False, 'mean_pixel': [123.68, 116.779, 103.939], 'shuffle': True, 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut-core/Testcore-Alex-2020-09-19/dlc-models/iteration-0/TestcoreSep19-trainset80shuffle1/train/snapshot', 'log_dir': 'log', 'global_scale': 0.8, 'location_refinement': True, 'locref_stdev': 7.2801, 'locref_loss_weight': 0.05, 'locref_huber_loss': True, 'optimizer': 'sgd', 'intermediate_supervision': False, 'intermediate_supervision_layer': 12, 'regularize': False, 'weight_decay': 0.0001, 'mirror': False, 'crop_pad': 0, 'scoremap_dir': 'test', 'batch_size': 1, 'dataset_type': 'default', 'deterministic': False, 'crop': True, 'cropratio': 0.4, 'minsize': 100, 'leftwidth': 400, 'rightwidth': 400, 'topheight': 400, 'bottomheight': 400, 'all_joints': [[0], [1], [2], [3]], 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'], 'dataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TestcoreSep19/Testcore_Alex80shuffle1.mat', 'display_iters': 2, 'init_weights': '/Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt', 'max_input_size': 1500, 'metadataset': 'training-datasets/iteration-0/UnaugmentedDataSet_TestcoreSep19/Documentation_data-Testcore_80shuffle1.pickle', 'min_input_size': 64, 'multi_step': [[0.001, 5]], 'net_type': 'resnet_50', 'num_joints': 4, 'pos_dist_thresh': 17, 'project_path': '/Users/mwmathis/Documents/DeepLabCut-core/Testcore-Alex-2020-09-19', 'save_iters': 5, 'scale_jitter_lo': 0.5, 'scale_jitter_up': 1.25, 'output_stride': 16, 'deconvolutionstride': 2}
Starting training....
iteration: 2 loss: 1.1634 lr: 0.001
iteration: 4 loss: 0.5873 lr: 0.001

... 

Starting with imgaug pose-dataset loader.
Batch Size is 1
Initializing ResNet
Loading ImageNet-pretrained resnet_50
Training parameter:
{'stride': 8.0, 'weigh_part_predictions': False, 'weigh_negatives': False, 'fg_fraction': 0.25, 'weigh_only_present_joints': False, 'mean_pixel': [123.68, 116.779, 103.939], 'shuffle': True, 'snapshot_prefix': '/Users/mwmathis/Documents/DeepLabCut-core/Testcore-Alex-2020-09-19/dlc-models/iteration-1/TestcoreSep19-trainset80shuffle1/train/snapshot', 'log_dir': 'log', 'global_scale': 0.8, 'location_refinement': True, 'locref_stdev': 7.2801, 'locref_loss_weight': 0.05, 'locref_huber_loss': True, 'optimizer': 'sgd', 'intermediate_supervision': False, 'intermediate_supervision_layer': 12, 'regularize': False, 'weight_decay': 0.0001, 'mirror': False, 'crop_pad': 0, 'scoremap_dir': 'test', 'batch_size': 1, 'dataset_type': 'imgaug', 'deterministic': False, 'crop': True, 'cropratio': 0.4, 'minsize': 100, 'leftwidth': 400, 'rightwidth': 400, 'topheight': 400, 'bottomheight': 400, 'all_joints': [[0], [1], [2], [3]], 'all_joints_names': ['bodypart1', 'bodypart2', 'bodypart3', 'objectA'], 'dataset': 'training-datasets/iteration-1/UnaugmentedDataSet_TestcoreSep19/Testcore_Alex80shuffle1.mat', 'display_iters': 1, 'init_weights': '/Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/models/pretrained/resnet_v1_50.ckpt', 'max_input_size': 1500, 'metadataset': 'training-datasets/iteration-1/UnaugmentedDataSet_TestcoreSep19/Documentation_data-Testcore_80shuffle1.pickle', 'min_input_size': 64, 'multi_step': [[0.001, 5]], 'net_type': 'resnet_50', 'num_joints': 4, 'pos_dist_thresh': 17, 'project_path': '/Users/mwmathis/Documents/DeepLabCut-core/Testcore-Alex-2020-09-19', 'save_iters': 5, 'scale_jitter_lo': 0.5, 'scale_jitter_up': 1.25, 'output_stride': 16, 'deconvolutionstride': 2, 'num_outputs': 1, 'Task': None, 'scorer': None, 'date': None, 'video_sets': None, 'bodyparts': None, 'start': None, 'stop': None, 'numframes2pick': None, 'skeleton': [], 'skeleton_color': 'black', 'pcutoff': None, 'dotsize': None, 'alphavalue': None, 'colormap': None, 'TrainingFraction': None, 'iteration': None, 'resnet': None, 'snapshotindex': None, 'cropping': None, 'x1': None, 'x2': None, 'y1': None, 'y2': None, 'corner2move2': None, 'move2corner': None}
Starting training....
iteration: 1 loss: 1.5909 lr: 0.001
iteration: 2 loss: 0.7031 lr: 0.001
iteration: 3 loss: 0.5709 lr: 0.001
iteration: 4 loss: 0.5077 lr: 0.001
iteration: 5 loss: 0.3996 lr: 0.001

...

ALL DONE!!! - default cases without Tensorpack loader are functional.

(2)

  • 'analyze_time_lapse_frames' needs updated:
ANALYZING some individual frames
Traceback (most recent call last):
  File "testscript.py", line 225, in <module>
    dlc.analyze_time_lapse_frames(path_config_file,os.path.join(cfg['project_path'],'labeled-data/reachingvideo1/'))
  File "/Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/predict_videos.py", line 681, in analyze_time_lapse_frames
    TF.reset_default_graph()
AttributeError: module 'tensorflow' has no attribute 'reset_default_graph'

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

Currently default train, evaluate cases are functional with TensorFlow 2.2 and 2.3 with only the following warnings:

  • WARNING:tensorflow:From /Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer_v1) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead.

  • WARNING:tensorflow:From /Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:1666: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers.

  • WARNING:tensorflow:From /Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/nnet/losses.py:38: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.cast` instead.

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

outstanding issue that would need resolved before merging:

  • export_model:
Initializing ResNet
WARNING:tensorflow:From /Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py:127: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
  File "testscript_cli.py", line 162, in <module>
    dlc.export_model(path_config_file,shuffle=1,make_tar=False)
  File "/Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/export.py", line 327, in export_model
    tf_to_pb(sess, ckpt, output, output_dir=full_export_dir)
  File "/Users/mwmathis/Documents/DeepLabCut-core/deeplabcutcore/pose_estimation_tensorflow/export.py", line 214, in tf_to_pb
    initializer_nodes='')
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py", line 361, in freeze_graph
    checkpoint_version=checkpoint_version)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py", line 190, in freeze_graph_with_def_protos
    var_list=var_list, write_version=checkpoint_version)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 836, in __init__
    self.build()
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 848, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 886, in _build
    build_restore=build_restore)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 490, in _build_internal
    names_to_saveables)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 349, in validate_and_slice_inputs
    for converted_saveable_object in saveable_objects_for_op(op, name):
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 210, in saveable_objects_for_op
    variable, "", name)
  File "/Users/mwmathis/opt/anaconda3/envs/DLC-CPU/lib/python3.7/site-packages/tensorflow/python/training/saving/saveable_object_util.py", line 84, in __init__
    self.handle_op = var.op.inputs[0]
IndexError: tuple index out of range

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

@gkane26 - could you make the required changes here; #9

from deeplabcut-core.

altear avatar altear commented on June 10, 2024

Would there be any interest in a tf2 keras model + loader?

I had some trouble extending the current tf.compat.v1 version, so I made a tf2/keras implementation and wrote a function to load weights from pose-tensorflow/deeplabcut models

  • It's functional but I'm missing some of the deepercut cfg options like output_stride, and only use the part_pred and locref outputs

from deeplabcut-core.

MMathisLab avatar MMathisLab commented on June 10, 2024

Hi @altear is there something in PR #9 that does not work for you currently? But happy to discuss another loader.

from deeplabcut-core.

Related Issues (10)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.