Code Monkey home page Code Monkey logo

common-crawl-downloader's Introduction

👋 Hi there, I'm Zhong Zhenyu

Hexo-Blog Gmail-nczzy1997@gmail.com

Top Langs

Meet Zhong Zhenyu, a skilled researcher and developer with extensive expertise in machine learning, AIOps, high-performance computing, and full stack development. I am a smart worker and a determined team player, always looking for interesting research opportunities to build upon my skills.

I graduated from Nankai University in Tianjin, China and is currently pursuing my PhD at the same school. I am a fast learner, always eager to expand my knowledge base. As a keen observer and curious learner, I adds to the team's diversity and thrives in a collaborative environment. If you are looking for a passionate researcher and developer, I am your guy!

💻 What languages do I use?

C C++ Java Python JavaScript PHP LaTeX HTML5 CSS3 MySQL Powershell GNU Bash Markdown JSON YAML

⚡ What weapons do I have?

Operating System

Windows Linux Android

IDE

Visual Studio VS Code JetBrains

Development Framework

Qt Unity Spring Framework TensorFlow PyTorch Scikit-Learn Numpy Pandas Jupyter Django React Vue.js Node.js Hexo Bootstrap

Environment

Docker VMware

common-crawl-downloader's People

Contributors

alumik avatar

Stargazers

avatar avatar avatar avatar avatar

Watchers

avatar

common-crawl-downloader's Issues

An error has occurred: HTTP Error 403: Forbidden

D:\anaconda\envs\common_crawl\python.exe D:/code/common-crawl-downloader-main/src/main.py
[2022-07-08 15:50:06,463] [ INFO] Fetching a new job...
[2022-07-08 15:50:06,533] [ INFO] New job fetched: {id=31, uri=crawl-data/CC-MAIN-2021-10/segments/1614178347293.1/wet/CC-MAIN-20210224165708-20210224195708-00000.warc.wet.gz}.
[2022-07-08 15:50:06,533] [ INFO] Download from https://commoncrawl.s3.amazonaws.com/crawl-data/CC-MAIN-2021-10/segments/1614178347293.1/wet/CC-MAIN-20210224165708-20210224195708-00000.warc.wet.gz
[2022-07-08 15:50:07,774] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:07,774] [ INFO] Retry after 5 seconds (10 left)).
[2022-07-08 15:50:13,987] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:13,987] [ INFO] Retry after 5 seconds (9 left)).
[2022-07-08 15:50:20,207] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:20,207] [ INFO] Retry after 5 seconds (8 left)).
[2022-07-08 15:50:26,405] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:26,405] [ INFO] Retry after 5 seconds (7 left)).
[2022-07-08 15:50:32,617] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:32,617] [ INFO] Retry after 5 seconds (6 left)).
[2022-07-08 15:50:38,813] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:38,813] [ INFO] Retry after 5 seconds (5 left)).
[2022-07-08 15:50:45,029] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:45,029] [ INFO] Retry after 5 seconds (4 left)).
[2022-07-08 15:50:51,311] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:51,311] [ INFO] Retry after 5 seconds (3 left)).
[2022-07-08 15:50:57,536] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:50:57,536] [ INFO] Retry after 5 seconds (2 left)).
[2022-07-08 15:51:03,705] [ ERROR] An error has occurred: HTTP Error 403: Forbidden
[2022-07-08 15:51:03,705] [ INFO] Retry after 5 seconds (1 left)).
[2022-07-08 15:51:10,796] [ ERROR] Job failed.
[2022-07-08 15:51:10,866] [ INFO] Fetching a new job...
[2022-07-08 15:51:10,869] [ INFO] No unclaimed job found. This program is about to exit.

Process finished with exit code 0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.