Converting to RNN about realtime-action-recognition HOT 1 OPEN

It seems like the issue is related to the input dimensions of the RNN model during training and testing. In your train method, you are reshaping the input features to have the shape (batch_size, 1, num_features) before training the RNN model. However, during the prediction phase in the predict method of the ClassifierOnlineTest class, you are reshaping the features to have the shape (1, 1, -1).

To resolve this issue, make sure that the input dimensions during training and testing are consistent. In the train method, you are reshaping the input features using PCA, and in the _init_rnn_model method, you set input_shape=(None, NUM_FEATURES_FROM_PCA) for the first LSTM layer. Therefore, the correct input shape for the RNN model during training is (batch_size, timesteps, features).

Here's a suggested modification to your code:

Update the train method in the ClassifierOfflineTrain class to reshape the input features to have the shape (batch_size, window_size, num_features):

def train(self, X, Y):
    n_components = min(NUM_FEATURES_FROM_PCA, X.shape[1])
    self.pca = PCA(n_components=n_components, whiten=True)
    self.pca.fit(X)
    print("Sum eig values:", np.sum(self.pca.explained_variance_ratio_))
    X_new = self.pca.transform(X)

# Reshape input features to match the RNN input shape
X_new = X_new.reshape(X_new.shape[0], self._window_size, X_new.shape[1])

print("After PCA, X.shape = ", X_new.shape)
self.clf.fit(X_new, Y, epochs=100)

Update the predict method in the ClassifierOnlineTest class to reshape the input features consistently:

  def predict(self, skeleton):
      # ... (existing code)

if is_features_good:
    features = features.reshape(1, self._window_size, -1)
    print(f"Shape of input features: {np.array(features).shape}")

    curr_scores = np.argmax(self.model.predict(features), axis=-1)
    self.scores = self.smooth_scores(curr_scores)

    # ... (existing code)

By ensuring consistent reshaping of input features, you should be able to resolve the dimension mismatch issue during prediction. Does that help?

from realtime-action-recognition.