Code Monkey home page Code Monkey logo

twitterscraping's Introduction

Twitter Scraping Project

Problem Statement

Today, data is scattered everywhere in the world. Especially in social media, there may be a big quantity of data on Facebook, Instagram, Youtube, Twitter, etc. This consists of pictures and films on Youtube and Instagram as compared to Facebook and Twitter. To get the real facts on Twitter, you want to scrape the data from Twitter. You Need to Scrape the data like (date, id, url, tweet content, user,reply count, retweet count,language, source, like count etc) from twitter.

Directory Structure

  1. This repository consists the file required to create Twitter Scrapper Application
  2. The "main.py" file consists of major code required to develop the applcation. Pycharm IDE is recommended.
  3. The "lottie.py" file has code required for animation of image in Frontend Streamlit User Interface.

Requirements

  1. Python Scripting
  2. Streamlit for Front End UI
  3. Snscrape library to extract data from Twitter
  4. Pandas to create Data Frames
  5. Mongodb to store data in database

Features with Screenshot

  1. This is how the Front end of UI looks like

Page1

  1. After entering the Twitter Hashtag, It wil ask to insert the number of hashtag to search from start date to end date.

page2

  1. This is how the extracted data from twitter looks like with all the required data columns. Here we can upload the data to database, download the data in csv or json format

page3

  1. This is how the downloaded csv file looks like

page4

  1. This is how the downloaded JSON file looks like

page5

  1. This is how the uploaded datas in Mongodb looks like

page6

Video Demo

Here is Demo video link to know more about this project

Click Here to View Demo Video File

GitHub Link

Here is Git hub link to know more about this project

Click Here for Git Hub link

Developer

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.