Code Monkey home page Code Monkey logo

urlgrabber's Introduction

urlgrabber -- A high-level cross-protocol url-grabber

INSTALLATION INSTRUCTIONS

If you want to install urlgrabber on your system, simply open the package 
and run:

   python setup.py install
   
Take a look at the install options by doing

   python setup.py install --help
   
With no arguments, distutils will add all modules to a 'urlgrabber'
package under your python site-packages directory. 

You can build rpms by running

   python setup.py bdist_rpm 

The rpms (both source and "binary") will be specific to the current
distribution/version and may not be portable to others.  This is
because they will be built for the currently installed python.

keepalive.py and byterange.py are generic urllib2 extension modules and
can be used to add keepalive and range support to any urllib2 
application.

As of version 2.9.x, urlgrabber is no longer tested with python
versions less than 2.2.  It will probably work with 2.0, but will
almost certainly NOT work under prior python versions.

urlgrabber's People

Contributors

skvidal avatar james-antill avatar megaumi avatar tradej avatar scop avatar dmnks avatar hartlinejasper avatar lmacken avatar mewingtree avatar abadger avatar

Stargazers

Nate Schultz avatar

Watchers

Henry Huang avatar avatar

Forkers

pebbie andreipak sambler nickjhathaway

urlgrabber's Issues

python3 compatiblity

Hi what's that status of python 3 compatibility? Do you accept PRs? If yes, what are you requirements, what Python versions would you like to support?

Is pycurl a dependency of the package?

We have some trouble adding urlgrabber as required package in our projects because at install time it throws this error

Downloading/unpacking urlgrabber (from ***==0.0.4)
  Downloading urlgrabber-3.9.1.tar.gz (72Kb): 72Kb downloaded
  Running setup.py egg_info for package urlgrabber
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
      File "/home/test_***/build/urlgrabber/setup.py", line 3, in <module>
        import urlgrabber as _urlgrabber
      File "urlgrabber/__init__.py", line 54, in <module>
        from grabber import urlgrab, urlopen, urlread
      File "urlgrabber/grabber.py", line 427, in <module>
        import pycurl
    ImportError: No module named pycurl
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/home/test_***/build/urlgrabber/setup.py", line 3, in <module>

    import urlgrabber as _urlgrabber

  File "urlgrabber/__init__.py", line 54, in <module>

    from grabber import urlgrab, urlopen, urlread

  File "urlgrabber/grabber.py", line 427, in <module>

    import pycurl

ImportError: No module named pycurl

I think that pycurl is needed by urlgrabber. If this is true, please add pycurl as an install_requires in setup.py.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.