udacity / aind-recognizer Goto Github PK

Hi udacity-team

I was trying to submit my solution for review regarding this final project of Term-1 but I wasn't able to do so. It would fail with the following error message

aind-recognizer git/master  9s
(aind) ❯ udacity submit
Traceback (most recent call last):
  File "/Users/eklavya/anaconda/envs/aind/bin/udacity", line 7, in <module>
    from udacity_pa.projectassistant import main_func
  File "/Users/eklavya/anaconda/envs/aind/lib/python3.6/site-packages/udacity_pa/projectassistant.py", line 29, in <module>
    raise  NotProjectAssistantError()
udacity_pa.projectassistant.NotProjectAssistantError: Not an Udacity Project Assistant directory. No .udacity-pa folder found.

After a little digging, I found that the presence of .udacity-pa is necessary for a proper authentication and clearly this was what's lacking from this repo since all other Term-1 repos do have their own respective .udacity-pa/projects.py file.

https://github.com/udacity/AIND-Sudoku/blob/master/.udacity-pa/projects.py

https://github.com/udacity/AIND-Isolation/blob/master/.udacity-pa/projects.py

https://github.com/udacity/AIND-Planning/blob/master/.udacity-pa/projects.py

But it's not in this repo. Could you please fix this soon ?

The ModelSelector class is being referred to as the SelectorModel class in the asl_recognizer notebook. Below is how it is currently written in the notebook for reference.

"Review the SelectorModel class from the codebase found in the my_model_selectors.py module. It is designed to be a strategy pattern for choosing different model selectors. For the project submission in this section, subclass SelectorModel to implement the following model selectors."

In the beginner of Part 3 there is a jupyter instruction to autoreload modules.

It should go above (at least) the implementation of SelectorCV in Part 2.

# autoreload for automatically reloading changes made in my_model_selectors and my_recognizer
%load_ext autoreload
%autoreload 2

The function recognize() in my_recognizer.py as part of Part 3 of this project is suppose to do two things:

1. Calculate the probabilities (log likelihoods) of all the test samples given the training models
2. Make a guess based on those log likelihoods

However, one of the unittests can fail if all the training models fail to yield a score for a given test sample which means the resulting list of dicts returned for "probabilities" will not have the same length as the number of test samples and thus the student will see something like:

"Number of training word probabilities in test item dictionary incorrect.")
AssertionError: 107 != 112 : Number of training word probabilities in test item dictionary incorrect.

The above is from the following line in the unit test case TestRecognize of asl_test.py:

def test_recognize_probabilities_interface(self):
probs, _ = recognize(self.models, self.test_set)
self.assertEqual(len(probs), self.test_set.num_items, "Number of test items in probabilities list incorrect.")

I think given there is a possibility that all of the SelectorConstant models can fail to yield a score for a test sample (I found this issue somewhat illuminating since this is what I see occasionally: hmmlearn/hmmlearn#101), the unit test should be more lenient regarding comparing numbers or else the project has to guarantee that a score() can always be generated for all the test word items.

NOTE: To pass this unit test I had to set float('-inf') for certain words which again, is not obvious.

Consider giving a brief explanation of jupyter in the README. Something like "This will open Jupyter..., which is where you should directly edit and run your code." I am command line obsessed and didn't realize what Jupyter was initially which is to say I was making and running my own scripts outside of Jupyter.

Lines 77, 138, 148, 165, 248, 258 are incorrectly formatted and do not run.
Removing all items before the : in the function inputs fixed it.

Lines 9 and 45. Removing the text before the colons in the function inputs fixes it.

The delta features test assumes that the differences are calculated globally within the dataframe, but that causes difference for each video to be initialized relative to the last value in the preceding video. The differences should instead be grouped by video, so that they are re-initialized in the first frame of each sequence.

Consider scaling features before fitting HMMs to avoid precision errors.

https://datascience.stackexchange.com/questions/6144/sensitivity-to-scaling-of-features-in-a-multivariate-gaussians