rkanapka / darbas24 Goto Github PK
View Code? Open in Web Editor NEWDarbas24 - list of job offers from different job site providers.
Darbas24 - list of job offers from different job site providers.
Implement functionality to calculate salary.
Provide full salary information for user.
Scrap cvbankas job offers by category.
Implements back-end functionality for search boxes.
Move parsing from tasks.py to some other file - "parser", "processor"?
Implement all code from functional to OOP?
Create cv.lt scraper
Write unit tests for salary calculation service.
Sort job offers by:
Add new page with chart of job offer count in each category.
Use chart.js
Add type hints to codebase.
Install django stubs.
Scrape cvmarket job offers.
Right now scraped data is saved to list as elements of dict.
After each iteration of data it is saved to database.
Need to create objects first and save it as bulk_create
.
Reference: https://stackoverflow.com/questions/64046422/django-save-multiple-objects-at-once
Remake current "cvbankas" scraper to work asynchronously.
Currently cvbankas 1 page is scraped in ~40second synchronously:
[2024-01-18 22:21:50,890: INFO/ForkPoolWorker-3] Task job.tasks.scrape_and_save_job_offers[487ce137-6724-4198-90c6-6e8f1cfd2d58] succeeded in 39.70813476400508s: None
Run celery task: celery -A darbas24 worker -B -l INFO
Warning:
[2024-01-31 11:53:40,249: WARNING/MainProcess] /usr/local/lib/python3.12/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine
whether broker connection retries are made during startup in Celery 6.0 and above.
If you wish to retain the existing behavior for retrying connections on startup,
you should set broker_connection_retry_on_startup to True.
warnings.warn(
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.