EzFinbot - A bot to provide financial advise easily
(Disclaimer: We are not financial advisors, advise provided is from data from our dataset.)

FiQA-2018 dataset (https://sites.google.com/view/fiqa/home)

• Python 3.6+
• Anaconda

• Run pip install -r requirements.txt in root directory

1. Folders are split up into modules, where each module serves a function in the project process in sequence.
2. Intermediary files are outputted when running the code in each module.
3. Intermediary files are already included in this repo. However, if you wish to run the code to output intermediary files, just run the code in each folder in sequence of numbers
4. Description of what each module does can be found in README inside each Folder
5. To run telegram bot, simply open telegram and search for @EzFinBot to start using.
   4.1. Telegram bot is already deployed with AWS Lightsail.
   4.2. In the event we take down the bot, you may deploy/ run the telegram code locally
   4.3. Telegram bot code can be found in 9.Deployment
   4.4. (If bot is taken down) Simply run python ezfinbot-telegram.py in terminal and you can use the bot at @EzFinBot.
   4.5. We will update this README if bot is taken down.
6. Alternatively, you may run chatbot locally.
   5.1. Head to 9.Deployment
   5.2. Run ezfinbot-cli.py

This project's ultimate aim is to create a Information Retrieval based Question Answering platform to answer common financial questions. From a high-level perspective, the project flow involves:

1. Query expansion using SpaCy wordnet financial domain to expand nouns of user query
2. Retrieval with BERT models
   2.1. Evaluated a few BERT models from sentence-transformers library.
   2.2. Finetuned with SimCSE technique and evaluated MSMARCO-distilBERT and MSMARCO-BERTbase (both untuned and fine-tuned versions) and using cosine similarity to retrieve top 20 relevant documents to question.
   2.3. Best performing BERT model (fine-tuned MSMARCO-BERTbase) was used as retrieval model
   2.4. Results are evaluated using P@1, R@1, F1@1, and also qualitatively.
3. Reranker with cross encoder. Utilized ms-marco-miniLM-L-12-v2 as cross-encoder.
   3.1. This reranks the top 20 relevant documents retrieved by retrieval model.
   3.2. Cross-encoders are more accurate but takes longer to run. Hence, we do not implement this on entire dataset.
   3.3. Evaluation method similiar as using retrieval.
4. Deployment
   4.1. Inside folder 9.Deployment, users can find code for telegram bot (ezfinbot-telegram.py), and code to run locally (ezfinbot-cli.py).
   4.1. Telegram bot can be accessed at @EzFinBot.
   4.2. Telegram bot is currently already hosted on AWS, so no need to run ezfinbot-telegram.py to use bot.