Personal Project for Politicians Analysis.
The idea comes from a university project for the Social Media Management course.

The project aims to make stats about Politicians' use of Social Media.
It analyses, for each social media, politicians of any country belonging to any political group.

The structure of the project is designed below.
All the components have been created on Docker.

Every day, at 00:00 AM UTC, it starts a job with three main phases managed by Manager.

• Reading: it reads from the YAML config file all the info necessary to make stats. Precisely, it contains socials, countries and their respective political groups and accounts we want to analyse. I have decided to use a YAML file rather than a relational DB because it is easier to update and store.
• Fetching & Processing: for each social and country, it retrieves data for each account for the last 24hrs and puts them into an Account object (one for each account). Once constructed a list of accounts' objects, it instantiates a Group object. These objects make all the stats required for the last 24hrs.
• Saving: once made all accounts and political groups' objects associated with a given country on a given social, the data are saved through a Helper into a database.

Below is illustrated a simplified UML of the crawler.
It has to be noted that Account, Group, and Helper use a Factory Method Design Pattern.

The Logical Schema can be found in /docs/imgs/db.png.
Let's see a light version of the E-R schema:

UI has been implemented using HTML, CSS, Bootstrap, and JS (with JQuery).
On request, via REST APIs using AJAX, the Node.js back-end connects to the MySQL database and returns in JSON format to the front-end what it has asked for (or an error code 404).

Let's see the available APIs at this moment:

/api/countries
/api/:country/groups
/api/:country/:social/accounts

/api/:social/groups/:group/info
/api/:social/groups/:group/insights

/api/:social/accounts/:group/all
/api/:social/accounts/:handle/info
/api/:social/accounts/:handle/insights
/api/:social/accounts/:handle/hashtags/:since/:limit (:since is among 'w', 'm', or 'y').

Below is illustrated the backend UML.
It has to be noticed that Managerimplements a sort of Factory Method Design Pattern.

The charts have been created and managed using Grafana.
I have generated HTTPS certifications to allow a secure connection.
Furthermore, I have set Grafana to be accessible just by the admin (username and password are required).

In case of an update in MySQL settings (environment variables in docker-compose.yml: host, port, user, password, database name), you must update /grafana-charts/provisioning/datasources/default.yml to import the data source into Grafana.

Datasource and Dashboard are automatically imported.
Dashboard Panels are embedded in the UI through specific <iframe> tags.

• For Each Account and Political Group:
  • Total Likes (yesterday)
  • Total Retweets (yesterday)
  • Total Replies (yesterday)
  • Average Likes (yesterday)
  • Average Retweets (yesterday)
  • Average Replies (yesterday)
  • Average Length of a Tweet (in characters) (yesterday)
  • Average Sentiment Analysis (Positive, Negative, Neutral, Null) (yesterday)
    
    
  • Charts of Compare Among All Political Groups (followers, likes, etc.)
    
    
  • Chart with the History of the Number of Analysed Tweets Over Time
  • Chart with the History of Total Followers Over Time
  • Chart with the History of Avg Tweets Length Over Time
  • Chart with the History of Avg Likes Over Time
  • Chart with the History of Avg Retweets Over Time
  • Chart with the History of Avg Replies Over Time
  • Top 5 Most Used Hashtags (just for accounts) (last week, last month, last year)

So that the repository is successfully cloned and the project runs smoothly, a few steps need to be followed.
N.B. All used keys, tokens, and passwords are invalid and have to be re-generated.

• A stable internet connection.
• Use of Linux, MacOS, or Windows WSL.
• Having a Twitter Developer Account with related keys and tokens (for Twitter Analysis).
• Need to download and install Docker (Docker Desktop is recommended).
• The use of Visual Studio Code is strongly recommended for modify.

   $ git clone https://github.com/antonioscardace/Stats-Of-Politicians-On-Social-WebApp.git
   $ cd YOUR_PATH/Stats-Of-Politicians-On-Social-WebApp/
   $ bash run.sh

• Re-design Node.Js back-end to be hybrid with more socials and countries. Design Patterns.
• Add other countries in addition to Italy.
• Add other socials (e.g. Instagram).
• Re-implement front-end using Vue.js or similar frameworks.

Author: Antonio Scardace.
See LICENSE for more information.