Light

tirthajyoti / web-database-analytics Goto Github PK

View Code? Open in Web Editor NEW

259.0 17.0 170.0 4.34 MB

Web scrapping and related analytics using Python tools

License: MIT License

Python 0.25% Jupyter Notebook 99.75%

analytics python sqlite3 xml-parser json json-parser sql web-scraping beautifulsoup4 database

web-database-analytics's Introduction

Web scraping, database and related analytics

Dr. Tirthajyoti Sarkar (You can connect with me on LinkedIn)

Requirements

Python 3.5+
NumPy ($ pip install numpy)
Pandas ($ pip install pandas)
requests ($ pip install requests)
BeautifulSoup4 ($ pip install beautifulsoup4)
MatplotLib ($ pip install matplotlib)

My new book on Data wrangling with Python

What type of Notebooks are here?

Web scraping and related analytics using Python tools
Fundamentals of Regular expressions (Regex)
Application of urllib
Application of BeautifulSoup for HTML parsing
Application of ElementTree for XML parsing
Application of Python json library for JSON parsing
Application of Python sqlite library (building a personal movie database)

How to design your own mini-IMDB movie database by scraping web?

Check out this article I wrote on Medium about this topic

How to scrape data from CIA website (this is harmless, I promise) about simple facts on various nations?

Check out this article I wrote on Medium about this topic

How to build a Yelp crawler which can generate interesting word cloud based on a particular city's food cuisine and taste?

How to crawl the Project Gutenberg portal and download 100 most popular books automatically?

How to use a free API to download basic information about countries around the world and build a database?

web-database-analytics's People

Contributors

Stargazers

Watchers

Forkers

coolprojects mohisen dthboyd radhikari54 kapoor28anuj alainottenheimer ymopur daprueba666 ravithejaburugu zappy555 mrb1b0 gridl libardo1 saldridgevcs matuschladek aascode nhatnguyen12 rafayet13 alexewd f-m-grebe jbdatascience lucas-homer saffire33 hoadu mphirke pleabargain pankajmehar mayukhg jaironpz anirband antpham yuseferi sdabic kshailen jawsnfl-xx tanglijhu mahmud83 m3k0ar rajeshpyne tzislam isuhas jaspunkybackpack georgits sangeetha77 hidhineshraja elzaksspro druva18 p-2-r-0-i-1-y-8-a rams502 lucastravi liston rohitkumar93 smurugu khinmaunghtay4ah pravileaf ioannispapadakis patrickestark konutech vinay7820 jeka312 leofn stjordanis nyzo-voting-repo armendk iliassaek juanmunoz00 rshnghost dlm6693 laguer mindaugasvaitkus2 ibrahim85 rodeoblues youheimaoli shekerkamma payalbhatia pmalempati hsharmatech jahmah sifatul22 luyi1678 gshivakrishna ernestcr heenashree daoway ajeetpandeyy bibhuashish rkyuubin ethoian mochinko7 creptt vmnet04 bhumi-29 venk2108 gokhu18 sahil-sharma1024 roadtodatascience asyrofist codeit lpcteste wnocella

web-database-analytics's Issues

the book script does not put the files in the user created directory

Not the end of the world! :) but I think your script is windows specific...
the f print issue is in this notebook
AND
the book files do get downloaded into the main dir but NOT into the user created dir. The book names are prepended by the user supplied name for the folder but the folders are malformed and empty.
see screenshot

f is appearing before many print statements

I'm trying to run your repo and there is a spare 'f' before many print statements.

please supply a bogus APIkeys.json file for reference

I went to the weather site
it gave me a long string
I saved it as APIkeys.json in the directory but the noteboook complains:
`JSONDecodeError: Expecting value: line 1 column 1 (char 0)

`
please supply a bogus APIkeys.json file for reference

thank you

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.