Code Monkey home page Code Monkey logo

saquibtechlotraining / skytrax_airlines_project Goto Github PK

View Code? Open in Web Editor NEW
0.0 1.0 0.0 29.63 MB

This project scrapes the latest airline reviews from Skytrax ✈️ (as of Sep 14, 2023) to derive insights for data-driven decisions 📊. It aims to enhance customer satisfaction 😊, optimize services 🔧, and maintain competitiveness in the aviation industry 🌍 through a focus on meaningful analysis of passenger feedback 🗣️.

Jupyter Notebook 99.73% Python 0.27%
azuresql etl-pipeline nltk-python pandas powerbi python webscraping pyodbc sqlalchemy

skytrax_airlines_project's Introduction

Skytrax Airlines Project

Dashboard Link : https://www.novypro.com/project/aviation-fleet-dashboard-airlines--aircrafts-overview-power-bi

Note:- For this project, I scraped data of latest airlines reviews from the website Skytrax of 14th of September, 2023.

① Sample Data of one airline Click Here

Project Overview

Background 
Objective 

➡ Background -

In today's rapidly evolving airline industry, customer feedback plays a pivotal role in shaping business strategies. This project centers around harnessing the power of data by collecting and analyzing airline reviews from Skytrax, a leading source of passenger opinions and reviews for airlines worldwide.

➡ Objective -

The primary objective of our project is to extract meaningful insights from daily airline reviews, enabling airlines to make data-driven decisions to enhance customer satisfaction, optimize services, and stay competitive in the market.

Skytrax Airlines Project Piplines:-

➡ Each stage is meticulously designed to ensure the accuracy, integrity, and reliability of the extracted information.

Pipeline 1: Data Collection & Data Storage

① Task: Collecting airlines data from Skytrax.

② Components:

• WebScrapping (Python)
• Utilize advanced web scraping techniques to extract relevant information from the Skytrax website.
• Employ cutting-edge technologies to navigate through web pages and transform raw data into a structured format.
• Data Storage (Azure SQL Database):
• Leverage Azure SQL Database, a cloud-based relational database service, for seamless and efficient data management.
• Collected reviews are securely stored in Azure SQL Database in the form of table, ensuring data availability, scalability, and robustness.

Database Connection Information Click Here

Table Creation Code Click Here

Pipeline 2: Data Extraction and Data Preprocessing

① Task: Process and analyze the collected data to prepare it for insights generation.

② Components:

• Data Extraction with Pyspark (Databricks):
• Utilize Pyspark for large-scale data analysis and efficient extraction.
• Store the extracted data in CSV format in the local system.

Screenshots : Click Here

Raw Data : Click Here

• Preprocessing with Pandas:
• Leverage Pandas for data manipulation and analysis.
• Clean and transform the CSV-formatted data into a structured format for further analysis.

Pipeline 3: Exploratory Data Analysis (EDA) and Advanced Analytics

① Task: Explore data patterns, perform sentiment analysis, and apply advanced analytics techniques.

② Components:

• Exploratory Data Analysis (Pandas):
• Visualize and summarize the preprocessed data using EDA techniques.
• Gain insights into passenger sentiments, preferences, and experiences.

Preprocessing with Pandas Part-1 Click Here

After Part-1 Save data Click Here

Preprocessing with Pandas Part-2 Click Here

Clean Data : Clean Data

Pipeline 4: Data Visualization

① Task: Visualization of Clean Data

② Components:

• For Visualization use Power Bi :
 • After obtaining the final clean data, utilize Power BI for creating interactive and visually appealing dashboards and reports.
 • Power BI allows for seamless visualization of insights, making it easy to communicate and interpret complex information.

➡ Business Implications:

① Translate insights into tangible business strategies.

② Explore real-world implications and demonstrate how airlines can optimize operations, refine customer interactions, and devise innovative marketing campaigns based on data-driven insights.

➡ Project Impact Assessment:

Reflect on how the project contributes to the evolution of the airline industry and fosters a culture of data-driven decision-making.

➡ Lessons Learned:

Discuss challenges encountered during the project lifecycle and strategies employed to overcome them.

➡ Future Enhancements:

Outline potential avenues for future enhancements, including the integration of additional data sources, implementation of more advanced analytics, and exploration of predictive modeling.

Dashboard View

Screenshot 2023-12-01 222814

Screenshot 2023-12-01 222843

Screenshot 2023-12-01 222923

Model View

image

Tools Used:

• WebScraping - Python

• Data Storage - Azure SQL Database

• Data Extraction - Pyspark

• Data Preprocessing - Pandas

• Data Visualization - Power Bi

skytrax_airlines_project's People

Contributors

saquibtechlotraining avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.