API Used
Request is sent to the above API, which returns JSON data for the response.
The data is ingested in a database named "twitter_db" with "donations" as collection name.

{
    "_id" : ObjectId("62ab286e9d3ced498f284ccb"),
    "id" : "3",
    "datePledged" : ISODate("2020-02-07T05:30:00.000Z"),
    "dateConfirmed" : "",
    "source" : "Ireland",
    "sourceType" : "Public",
    "sourceGeographicalLocation" : "Europe",
    "target" : "WHO",
    "targetType" : "Multilateral",
    "targetGeographicalLocation" : "Global",
    "purpose" : "WHO Strategic preparedness and response plan",
    "access" : "Yes",
    "amount" : 1100000,
    "Comment" : "",
    "Loans & Guarantees" : "No"
}

API Used
Request is sent to the above API, which returns JSON data for the response.
Country name is read from a CSV file which is then passed to the Base URL as parameter.
The fetched data is transformed in dictionary format and ingested in a database named "twitter_db" with "cases_data" as collection name.
Field of date is parsed using dateutil.

{
    "_id" : ObjectId("62a1ad33188951e3b3f2be5c"),
    "Country" : "India",
    "Code" : "IND",
    "Confirmed cases" : 43197522,
    "Deaths" : 524723,
    "Last Updated" : ISODate("2022-06-09T04:20:54.000Z")
}

API Used
Request is sent to the above API using quandl, which returns JSON data for the response.
Country code is read from a CSV file which is then passed to the Base URL as parameter.
The fetched data is cleaned and transformed in dictionary format.
The data is then ingested in a database named "twitter_db" with "global_economy" as collection name.
Since the API can handle only 20 requests in 10 minutes, necessary time delay is provided between each request.

{
    "_id" : ObjectId("62a304375c5abb7917533f24"),
    "Country" : "Argentina",
    "Code" : "AR",
    "Date" : ISODate("1970-01-01T00:00:00.000Z"),
    "GDP" : 31584210365.5447
}

Data fetched from Twitter API have date field "created_at" in form of String.
This field needs to be converted into ISO Date format to perform computation.
The operation is performed using dateutil.parser and later updated in the collection.
Data stored in collection "donations" also has fields "datePledged" and "dateConfirmed" in INT format which needs to be converted in ISO Date format.
The operation is carried out using datetime and later updated in the collection.

Summary tables are created to optimize the performance of queries made by the user.
It aggregates twitter data on a daily basis.
Summary table1 consists of no. of tweets made from a location on a daily basis along with the date in consideration.

{
    "_id" : ObjectId("62a74142c093ffc891956361"),
    "location" : "India",
    "tweet_count" : 9,
    "date" : ISODate("2022-06-08T00:00:00.000Z")
}

Summary table2 consists of frequency of words occurring in tweets on a daily basis along with the date in consideration.

{
    "_id" : ObjectId("62a7418a725ea904c913c799"),
    "word" : "covid",
    "count" : 37,
    "date" : ISODate("2022-06-12T00:00:00.000Z")
}

To increase the security of the data stored in mongoDB, Client side field-level encryption is performed.
One field from each collection is encrypted using pymongo encryption (ClientEncryption and Algorithm).
Different local_master_key is generated using urandom function for every collection.
The encryption algorithm converts the field from string to binary format.
Same key must be used to decrypt the field.

Data visualization is done using matplotlib.
Various bar graphs, line graphs, histograms, scatter-plots are created to visualize the data stored in twitter_db.
GDP growth of a country over the years is visualized on a scatter-plot.
Rise of covid cases and deaths in a country over a time period is visualized on a line graph.
Amount of donations made by top n country for covid relief fund is visualized on a bar graph.

Unit test cases are written to validate the working of API endpoints.
One test case is written for each query.
Test will show passed status if the response from the server is 200 (ok response).

def test_api_query1(self):
    response = requests.get("http://127.0.0.1:8086/api/query1?startdate=&&enddate=")
    assert response.status_code == 200