Code Monkey home page Code Monkey logo

app_scraper's Introduction

Avito Scraper, Elasticsearch and Kibana

Make sure to have docker and docker compose installed

Installation

Clone project

  git clone https://github.com/deidax/pfe.git

Build project with docker compose

  docker-compose build --no-cach

Run project

  docker-compose up

Create user to login

  curl -X POST http://127.0.0.1:8000/auth/users/ --data 'username=admin&password=pfe123456'

Open in browser

  http://localhost:8080/

Login

  Username: admin
  password: pfe123456

app_scraper's People

Contributors

deidax avatar

Stargazers

avatar avatar avatar

Watchers

avatar

Forkers

kimovix

app_scraper's Issues

ERROR: Could not build wheels for cffi, lxml, which is required to install pyproject.toml-based projects

Running into the following error when trying to run the docker-compose up
`Building wheel for lxml (pyproject.toml): started
Building wheel for lxml (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error

× Building wheel for lxml (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [86 lines of output]
Building lxml version 4.6.3.
Building without Cython.
Building against libxml2 2.9.10 and libxslt 1.1.34
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/init.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/doctestcompare.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/sax.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/_elementpath.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/builder.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/cssselect.py -> build/lib.linux-x86_64-cpython-311/lxml
creating build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/init.py -> build/lib.linux-x86_64-cpython-311/lxml/includes
creating build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/init.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/clean.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/defs.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/builder.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-cpython-311/lxml/html
copying src/lxml/html/diff.py -> build/lib.linux-x86_64-cpython-311/lxml/html
creating build/lib.linux-x86_64-cpython-311/lxml/isoschematron
copying src/lxml/isoschematron/init.py -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron
copying src/lxml/etree.h -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/etree_api.h -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-cpython-311/lxml
copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/init.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-cpython-311/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-cpython-311/lxml/includes
creating build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources
creating build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/rng
creating build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl
creating build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-cpython-311/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.linux-x86_64-cpython-311
creating build/temp.linux-x86_64-cpython-311/src
creating build/temp.linux-x86_64-cpython-311/src/lxml
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/opt/venv/include -I/usr/local/include/python3.11 -c src/lxml/etree.c -o build/temp.linux-x86_64-cpython-311/src/lxml/etree.o -w
src/lxml/etree.c:289:12: fatal error: longintrepr.h: No such file or directory
289 | #include "longintrepr.h"
| ^~~~~~~~~~~~~~~
compilation terminated.
Compile failed: command '/usr/bin/gcc' failed with exit code 1
creating tmp
cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInit15zksful.c -o tmp/xmlXPathInit15zksful.o
cc tmp/xmlXPathInit15zksful.o -lxml2 -o a.out
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for lxml`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.