queries which cannot be classified should be added to database to be used for future calibrations of NLP algorithms

Currently the project uses flatfile (train.csv) for training. Instead, create dataframe from MongoDB, and use it directly.

This will ensure easy re-training directly from database, and will help in extending model specific trainings.

improve accuracy of the prediction model using LSTM?

Check feasibility of saving the model in database (as a blob)?

Add support for follow up questions - Add linkages of input and output circumstances to support data flow

predict the following (using SpaCy?) in input query -
Noun, place, Animal, date, number, email, time, colour, url.

Add an input variable automatically when system predicts any of the above in the input query.

prototype a basic chatbot which inputs a query and predicts following -

• question category
• answer category
• answers

@usriva2405 @usrivastava24 After setting up requirements when I run flask_controller.py and then I try authenticate API on postman, then I got error of credentials. I wants to what email and password I need to enter here?

I tried to enter mongo db user email and password with which I created account on MongoDB cloud but it didn't accept that email and password.
Can you please tell me what email and password we need to enter to run api authenticate ?

Thanks

Add a project layer between user and languages specific training data.

Users --(can add multiple)--> Projects(broker) --(can support multiple)--> languages --(will contain)--> training data

Add it once user layer has been added and language support has been added

Currently TF-IDF vectorizer coupled with a logistic/ SVC provides a 60% accuracy. Add preprocessing to improve the accuracy score

This system be moved behind an authentication layer, to enable custom predictors for logged in users.
Add a user layer to the database

classifier instance should not have to worry about structuring response, saving unclassified query to database. It's job is to only classify and return.

hence need to move this logic to chat service.

cachetools sometimes gives the correct email corresponding to a session token, and other times returns null. need to check why this is happening, and if we can instead use built in decorator for cachetools

• make use of same columns as mongoDB in the flatfile
• remove 3 classifiers and replace with 1 classifier per category

Currently this library gives inconsistent predictions if there are spelling mistakes in input query. Add a support for normalizer before vectorization to help with spelling mistakes.

Add options to include output variables which can be utilised at client side as placeholders

Add support for variables in input

• train a classifier(s?) to identify date, number, email, time, colour, url

check if we can use NLTK for small talk module. train a separate classifier, allow copying to user account as optional.
Similar to dialogflow, following intents could be added -

1. About Agent, Who are you?, How old are you?, You're annoying., Answer my question, You're bad, Can you get smarter?, You're beautiful., What's your birth date?, You're boring., Who is your boss?, Are you busy?, Can you help me?, You're a chatbot., You're so clever, You're crazy, You're fired., You are funny., You are good., Are you happy?...
2. Courtesy - thats bad, great!, No problem, thank you!, you're welcome!, well done.
3. Emotions - ha ha ha! Wow!
4. Hello/ Good bye - Hello, hi, hey there!, wassup? good morning, good afternoon, good evening, good night, How're you? nice to meet you. nice to see you, what's up?, nice to talk to you
5. About user - I'm very angry right now., I'm back!, I am bored., I am busy., I can't sleep., I don't want to talk., I'm so excited., I'm going to bed., I'm good., I'm happy., Today is my birthday., I am here., I'm kidding., What do I look like?, I'd like to see you again., I just want to talk.,
6. Confirmation - Yes, No, Cancel
7. Other Questions/ phrases - give me a hug, i don't care, sorry. what do you mean? you're wrong.

sample json from dialogflow is as below -

{
  "responseId": "64907be5-b773-40d1-9027-abd34fe1ab7e-f6406966",
  "queryResult": {
    "queryText": "I don't care",
    "action": "smalltalk.dialog.i_do_not_care",
    "parameters": {},
    "allRequiredParamsPresent": true,
    "fulfillmentText": "Ok, let's not talk about it then.",
    "fulfillmentMessages": [
      {
        "text": {
          "text": [
            "Ok, let's not talk about it then."
          ]
        }
      }
    ],
    "intent": {},
    "intentDetectionConfidence": 1,
    "languageCode": "en"
  }
}

Small talk and expletives should be added as separate classifiers.
Add it separately under the user "@def" (default) or "@sys" (system), same as what is used for defining default variables
Clone a copy when user requests for expletives/ small talk module to their account. Allow them to add custom trainable queries to their account.

Add multi-lingual support AFTER user layer is added

Restructure the training data to include categories, multiple responses, follow-up questions, and variables

System should have following default intents -

1. unclassifiable queries : reactions such as "I'm not sure", "Sorry didn't get you" etc.
2. small talk
3. expletives

upgrade config file to yaml
create a python class to extract variables
add environment specific structures (local, prod)

Idea would be for users to be able to configure bot auths for each broker. So as a user, if I have multiple projects (or brokers in our case) (for e.g. HR project, bank project etc), each project could individually support a bot specific to that project on multiple social media platforms.

Now I don't know about FB, Slack, Skype, Wechat, but Telegram has a auth token which binds to a bot (assuming rest also work in similar fashion.)

So every auth token for every bot should be configured at broker level.

create service + flask endpoint for retraining data. gradually move away from saving file to filesystem, instead move it to database.

Add support of multiple queries lifecycle to extend a conversation at the same time limit the number of retries/ depth

• Add multi-language data
• Add multiple responses for same category

Add flask support for exposing predictions as an API

data model should have classification to segregate between system intents and user intents

extract input variables and embed it to output conversation responses.
For e.g. -
input_query : Do you know my name is Sam?
output_response : Hello Sam! Nice to know you.

For quick prototyping, setup telegram support for the bot

use bootstrap4 via ajax API or play framework to setup GUI for

Add support for mongoDB to help store queries and responses.