pyporn-san / multporn Goto Github PK

View Code? Open in Web Editor NEW

26.0 1.0 1.0 196 KB

A python library used to scrape and download from Multporn.net

License: GNU General Public License v3.0

Python 100.00%

python nsfw scraping-websites pornography python-library rule34 rule34-downloader

multporn's People

Contributors

Stargazers

Watchers

Forkers

horstmedst

multporn's Issues

Default timeout

Additional to #92

Also with proxy connection speed is not so good as i would like. Downloading breaks with timeout with default value 5 sec. Maybe should increase the default value?

Using this script with proxy give some time access to site before cookie is outdated and script fails with error 403. With current script logic, image that was unable to download saves with name fpath.with_name(fpath.name + "_SKIPPED"). This works as expected only in scenarios when server return error 404, file not found. With 403 error empty file saved with '_SKIPPED' and won't be downloaded next time then script started.

In my case script must stop for 'cf_clearance' manually update

Throwing error from the basic snippet provided in readme

Between Friends ['Best', 'Blowjob', 'Cunnilingus', 'Lolicon', 'Oral sex', 'Stockings', 'Straight', 'Straight Shota', 'Virgin'] Traceback (most recent call last): File "api_test.py", line 12, in <module> comic.downloadImages() AttributeError: 'Multporn' object has no attribute 'downloadImages'
Where does the error lies ?

New and Comic page is unsupported

Issue

The current https://multporn.net/new and https://multporn.net/comic is not supported / doesn't work in the Webpage class.

How to reproduce

>>> from multporn import Webpage
>>> page = Webpage("https://multporn.net/new")
>>> print(page.links)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/home/livy/src/multclient/venv/lib/python3.10/site-packages/multporn/multporn.py", line 330, in links
    "table", "views-view-grid").find_all("strong")]
AttributeError: 'NoneType' object has no attribute 'find_all'

Error 403

Multporn is not accesable directly for me(blocked by ISP) so i am using proxy. With proxy i need to process cloudflare check before access the site otherwise i get error 403 - access is denied.

At this time script does not support to specify cookie.
What should i do to make it work:
method get in RequestHandler class
response = self.session.get( url, timeout=self.timeout, params=params, proxies=getproxies(), **kwargs, cookies={'cf_clearance' : 'some_data'})
cf_clearance i set manually after cloudflare check is done

BUG/HELP: cannot pip install package on python 3.11

  Building wheel for python-snappy (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\hadoop_snappy.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_cffi.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_cffi_builder.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_formats.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\__init__.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\__main__.py -> build\lib.win-amd64-cpython-311\snappy
      running build_ext
      building 'snappy._snappy' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for python-snappy
  Running setup.py clean for python-snappy
Failed to build python-snappy
Installing collected packages: python-snappy, soupsieve, pathvalidate, Faker, beautifulsoup4, multporn
  Running setup.py install for python-snappy ... error
  error: subprocess-exited-with-error

  × Running setup.py install for python-snappy did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      running install
      C:\Users\unseeyou\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\hadoop_snappy.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_cffi.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_cffi_builder.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\snappy_formats.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\__init__.py -> build\lib.win-amd64-cpython-311\snappy
      copying src\snappy\__main__.py -> build\lib.win-amd64-cpython-311\snappy
      running build_ext
      building 'snappy._snappy' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> python-snappy

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

This also happened to pygame, which they fixed in an experimental/pre-release. [link to pygame github error: https://github.com/pygame/pygame/issues/3522]

According to that page, you just need a new wheel.

Doesnt handle special characters

With the name of the following comic, it will only continually overwrite the same s.jpg over and over due to the name of the comic being S.EXpedition

Here is the test code

comic = Multporn("https://multporn.net/comics/others/s_expedition")

print(comic)


# Downloads the comic
comic.downloadContent()

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.