Documentation for run_model_analysis is outdated about model-analysis HOT 6 CLOSED

Thanks for looking into this. Yes, the API docs are out of date, and while we work on updating them, you can instead refer to the docstrings on the functions, either by looking at the code or by using Python's help builtin.

from model-analysis.

Besides, I think I found a bug on

I'm getting:

...

/usr/local/lib/python2.7/dist-packages/tensorflow_model_analysis/api/model_eval_lib.pyc in ExtractEvaluateAndWriteResults(examples, eval_shared_model, output_path, display_only_data_location, slice_spec, desired_batch_size, extractors, evaluators, writers, write_config, num_bootstrap_samples)
    566 
    567   example_weight_metric_key = metric_keys.EXAMPLE_COUNT
--> 568   if eval_shared_model.example_weight_key:
    569     example_weight_metric_key = metric_keys.EXAMPLE_WEIGHT
    570 

AttributeError: 'str' object has no attribute 'example_weight_key'

This happened when I called

tfma.run_model_analysis(eval_shared_model='WORKING_DIR/model_analysis',
                        data_location='WORKING_DIR')

I would like to create a mockup with all the emulation of this bug but I find it too difficult to reproduce (sorry). I think that from the last version of tfma to this one you did some changes around example_weight_key and forgot to create some tests for it.

from model-analysis.

Thanks for bringing this to our attention! It looks like we neglected to rebuild the docs before releasing. We will do so shortly.

from model-analysis.

After exploring a bit more, we found that the cause seems to be just deprecated documentation. Here is the contradiction:

• The documentation says that run_model_analysis
  • has the argument example_weight_key
  • The first argument is model_location
• The actual implementation of the function
  • does not have the argument example_weight_key
  • The first argument is eval_shared_model, which in turn has an example_weight_key

from model-analysis.

Can you please fix this issue, it's been there for three months and it's in the Getting started guide, the very first thing that users read.

from model-analysis.

Fixed.

from model-analysis.