Code Monkey home page Code Monkey logo

sinha-debojyoti / ookla-speedtest.net-crawler Goto Github PK

View Code? Open in Web Editor NEW
16.0 16.0 17.0 4.18 MB

A python web crawler that fetches more than 25 billion pages of https://www.speedtest.net to fetch download speed, upload speed, latency, date, distance, country code, server ID, server name, sponsor name, sponsor URL, connection_mode, isp name, isp rating, test rank, test grade, test rating, path.

License: Apache License 2.0

Python 97.27% Dockerfile 2.73%
bs4 ookla python speedtest webcrawler

ookla-speedtest.net-crawler's People

Contributors

amarkarak1 avatar bobthebuilder06092000 avatar chavarera avatar dependabot-preview[bot] avatar geekypandey avatar harsh0701xd avatar jaiakash avatar piranavs avatar rc-rajat avatar roycoding8 avatar sinha-debojyoti avatar snyk-bot avatar tusharnankani avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

ookla-speedtest.net-crawler's Issues

Readme: Update commands in installation instruction

In the readme file, the installation instruction is causing errors in Ubuntu 20.10.
python -m venv env
env\Scripts\activate.bat

But it should be for correct installation

python3 -m venv env
source env\Scripts\activate.bat

Shall I give a PR for same?

Error in the values interpretation

There is no proper interpretation of values returned from site.
Eg.,
download = 353 (on site it is 0.35 Mbps)
upload = 168 (on site it is 0.17 Mbps)
distance = 1250 (No proper units for interpretation)
etc etc

Need for proper documentation of the value stored.

Can anyone add Parallel Processing?

Parallel processing is a mode of operation where the task is executed simultaneously in multiple processors on the same computer. It is meant to reduce the overall processing time. A pooling of the URLs can be done.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.