uber / cassette Goto Github PK

Store and replay HTTP requests made in your Python app

Home Page: http://cassette.readthedocs.org/

License: Other

Makefile 0.88% Python 99.12%

cassette's Introduction

Cassette

Deprecation Warning: cassette has some known limitations and is not maintained anymore, we recommend using vcrpy instead.

Cassette stores and replays HTTP requests made in your Python app.

import urllib2

import cassette

with cassette.play("data/responses.yaml"):

    # If the request is not already stored in responses.yaml, cassette
    # will request the URL and store its response in the file.
    r = urllib2.urlopen("http://www.internic.net/domain/named.root")

    # This time, the request response must be in the file. The external
    # request is not made. cassette retrieves the response from the
    # file.
    r = urllib2.urlopen("http://www.internic.net/domain/named.root")

assert "A.ROOT-SERVERS.NET" in r.read(10000)

Cassette also supports the requests library.

import requests

with cassette.play("data/responses.yaml"):
    r = requests.get("http://www.internic.net/domain/named.root")

Note that requests stored between different libraries may not be compatible with each other. That is, a request stored with urllib2 might still trigger an external request is the same URL is requested with requests.

Installation

$ pip install cassette

Documentation

Latest documentation: cassette.readthedocs.org

License

cassette is available under the MIT License.

cassette's People

Contributors

Stargazers

Watchers

Forkers

bradjasper kevin1024 blampe wbbradley carolinevdh dariobottazzi underdogio-archive connectthefuture victorfontes ii0 leongdut afcarl standardgalactic iq-scm

cassette's Issues

Full `HTTPConnection` interface not supported

HTTPConnection.request is just a shortcut for the low-level API (putrequest, putheader, endheaders and send). If that API is used directly, cassette will throw,

'CassetteHTTPConnection' object has no attribute '_cassette_name'

Allow for `indent` as a `json.dumps` parameter

Upon reviewing the documentation, I discovered JSON encode and directory storage which is awesome for cleaning up out of date fixtures. Also, I saw reporting of unused fixtures which is really cool and I want to consider bringing over to nine-track =3

Anyway, I was wondering if we would be open to either setting an indent on json.dumps as a default or allowing it to be set as a configuration option. When scrolling through POST requests, it is hard to see the POST body without beautifying every file I open.

Patching implementation will not support nesting properly

It seems like cassette's patcher implementation (https://github.com/uber/cassette/blob/master/cassette/patcher.py) was lifted directly from the old version of vcr.py https://github.com/kevin1024/vcrpy/blob/master/vcr/patch.py.

There are actually some pretty nasty, though subtle and not often encountered bugs with this technique that vcr.py has since fixed.
see
kevin1024/vcrpy#107
kevin1024/vcrpy#109

None of this is urgent, but I just thought I'd mention it here in case it ever came up in cassette

Does not respect request headers

I'm trying to test varying a request by request headers, expecting a different response based on the header provided. However, it seems that the first response is replayed by cassette, since it does not take request headers into account. Is this correct, or am I missing something?

Idea: Cassette can keep track of the backtrace of the request for historical purposes

It would be helpful if cassette utilized/recorded backtraces for when requests were made, and recorded them into the responses file. This would aid when debugging why or where a test made a particular external request.

Issue With `requests` and HTTPS

Haven't yet tracked down the source of the bug, but the following gives me an error:

import cassette

# We'll try both
import urllib2
import requests

url = 'https://github.com/uber/cassette'

# This call is OK
with cassette.play('foo.yaml'):
    r = urllib2.urlopen(url)

# This call fails
with cassette.play('foo.yaml'):
    r = requests.get(url)

Yields the error:

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/Library/Python/2.7/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 292, in send
    timeout=timeout
  File "/Library/Python/2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 423, in urlopen
    conn = self._get_conn(timeout=pool_timeout)
  File "/Library/Python/2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 238, in _get_conn
    return conn or self._new_conn()
  File "/Library/Python/2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 562, in _new_conn
    strict=self.strict)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in __init__
    source_address)
TypeError: unbound method __init__() must be called with CassetteHTTPConnection instance as first argument (got VerifiedHTTPSConnection instance instead)

This is with:

requests (1.2.3)
cassette (0.2)

Cannot playback `mutlipart/form-data`

I am attempting to use cassette with a multipart/form-data request via requests/urllib3. The requests are saving successfully. However, they are not being resolved via playback.

We ran into the same problem in eight-track. The root of the problem is multipart/form-data operates on boundaries. A boundary is a delimiter used in the POST body that doesn't occur within any other part of the request.

https://github.com/twolfson/css-validator/blob/0.5.1/test/test-files/fake-jigsaw/POST_%252Fcss-validator%252Fvalidator_bc57c9bdeae0aba1f5a961ea5024775f.json

POST http://localhost:1337/css-validator/validator HTTP/1.1
content-type: multipart/form-data; boundary=--------------------------646267990491631137772609
host: 
connection: keep-alive
transfer-encoding: chunked

----------------------------646267990491631137772609
Content-Disposition: form-data; name="output"

soap12
----------------------------646267990491631137772609
Content-Disposition: form-data; name="w3cUrl"

http://localhost:1337/css-validator/validator
----------------------------646267990491631137772609
Content-Disposition: form-data; name="highWaterMark"

1024000
----------------------------646267990491631137772609
Content-Disposition: form-data; name="text"

body {
 background: url(ab'cd');
 -moz-box-sizing: content-box;
 }


----------------------------646267990491631137772609--

Most programmatic implementations generate random strings as their boundaries. In the example above, node-form-data generated ---...646267990491631137772609. In requests/urllib3, we are generating an hexadecimal string (e.g. f7b462765acf413faae8e5c3b29e7385).

https://github.com/kennethreitz/requests/blob/v2.5.1/requests/packages/urllib3/filepost.py#L13-L17

https://github.com/kennethreitz/requests/blob/v2.5.1/requests/packages/urllib3/filepost.py#L70-L74

For reference: To work around this in eight-track, we introduced normalizeFn which is a hook that allows us to normalize request data for consistent hash lookups. Then, we wrote eight-track-normalize-multipart to take care of the multipart/form-data case.

https://github.com/twolfson/eight-track-normalize-multipart

Docs link is broken

This link looks broken in README.rst:

Latest documentation: uber.github.io/cassette/

Issues with [email protected] and HTTPS

It looks like there is a regression with HTTPS requests. When running the tests against requests=2.4.3, I see the following error:

nosetests
..........E...........E...........E...............................E.......SSSSSSSS...
======================================================================
ERROR: Test that HTTPS requests work using requests.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/underdog/github/cassette/cassette/tests/test_cassette.py", line 320, in test_requestslib_https
    resp = self.helper_requestslib(TEST_URL_HTTPS)
  File "/home/underdog/github/cassette/cassette/tests/test_cassette.py", line 299, in helper_requestslib
    r0 = requests.get(url)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/api.py", line 60, in get
    return request('get', url, **kwargs)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/api.py", line 49, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/sessions.py", line 457, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/sessions.py", line 569, in send
    r = adapter.send(request, **kwargs)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/adapters.py", line 362, in send
    timeout=timeout
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 516, in urlopen
    body=body, headers=headers)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 304, in _make_request
    self._validate_conn(conn)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 722, in _validate_conn
    conn.connect()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 195, in connect
    conn = self._new_conn()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 120, in _new_conn
    if self.socket_options:
AttributeError: 'UL3CassetteHTTPSConnection' object has no attribute 'socket_options'
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTPS connection (1): httpbin.org
--------------------- >> end captured logging << ---------------------

NameError: global name 'UL3CassetteHTTPConnection' is not defined

Looks like we have a missing import in http_connection?

Traceback (most recent call last):
...
...
response = urllib2.urlopen(request, timeout=timeout)
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1174, in do_open
h.request(req.get_method(), req.get_selector(), req.data, headers)
File "/home/user/application/env/local/lib/python2.7/site-packages/cassette/http_connection.py", line 39, in request
if (isinstance(self, UL3CassetteHTTPConnection) and
NameError: global name 'UL3CassetteHTTPConnection' is not defined

Pip install doesn't work

I think there is an issue with how this is packaged on pip. When running setup.py develop from api:
Searching for cassette Reading http://pypi.python.org/simple/cassette/ Reading https://github.com/uber/cassette Best match: cassette 0.1.4.macosx-10.8-intel Downloading http://pypi.python.org/packages/any/c/cassette/cassette-0.1.4.macosx-10.8-intel.tar.gz#md5=c87c72a93997602dcdede7c2eb262ff1 Processing cassette-0.1.4.macosx-10.8-intel.tar.gz error: Couldn't find a setup script in /var/folders/tl/_pj0ccs55dj691frkrqvyw480000gn/T/easy_install-XzBBxa/cassette-0.1.4.macosx-10.8-intel.tar.gz

when running pip install cassette:

`Downloading/unpacking cassette
Downloading cassette-0.1.4.macosx-10.8-intel.tar.gz
Running setup.py egg_info for package cassette
Traceback (most recent call last):
File "", line 14, in
IOError: [Errno 2] No such file or directory: '/usr/local/Cellar/uber-home/HEAD/.virtualenvs/api/build/cassette/setup.py'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 14, in

IOError: [Errno 2] No such file or directory: '/usr/local/Cellar/uber-home/HEAD/.virtualenvs/api/build/cassette/setup.py'
`

Issue with `[email protected]`

The latest requests is incompatible with cassette for HTTP requests. When running the test suite, we receive complaints about self.sock not existing:

nosetests
.........EEE.........EEE.........EEE.............................EEE......SSSSSSSS...
======================================================================
ERROR: Test that normal HTTP requests work using requests.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/underdog/github/cassette/cassette/tests/test_cassette.py", line 314, in test_requestslib_http
    resp = self.helper_requestslib(TEST_URL)
  File "/home/underdog/github/cassette/cassette/tests/test_cassette.py", line 305, in helper_requestslib
    r1 = requests.get(url)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/api.py", line 65, in get
    return request('get', url, **kwargs)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/api.py", line 53, in request
    session.close()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/sessions.py", line 651, in close
    v.close()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/adapters.py", line 261, in close
    self.poolmanager.clear()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/poolmanager.py", line 91, in clear
    self.pools.clear()
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/_collections.py", line 93, in clear
    self.dispose_func(value)
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/poolmanager.py", line 65, in <lambda>
    dispose_func=lambda p: p.close())
  File "/home/underdog/.virtualenvs/cassette/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 378, in close
    conn.close()
  File "/usr/lib/python2.7/httplib.py", line 779, in close
    if self.sock:
AttributeError: 'UL3CassetteHTTPConnection' object has no attribute 'sock'
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1
cassette: INFO: File '/home/underdog/github/cassette/cassette/tests/data/responses.temp' does not exist.
cassette: WARNING: Library does not have 'httplib:GET 127.0.0.1:5000/index a5cd67e00e8b59c018bdce7e0aa5a581 None'
cassette: WARNING: Making external HTTP request: httplib:GET 127.0.0.1:5000/index a5cd67e00e8b59c018bdce7e0aa5a581 None
werkzeug: INFO: 127.0.0.1 - - [12/Feb/2015 19:53:15] "GET /index HTTP/1.1" 200 -
requests.packages.urllib3.connectionpool: DEBUG: "GET /index HTTP/1.1" 200 0
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1): 127.0.0.1
cassette: INFO: Library has 'httplib:GET 127.0.0.1:5000/index a5cd67e00e8b59c018bdce7e0aa5a581 None'
requests.packages.urllib3.connectionpool: DEBUG: "GET /index HTTP/1.1" 200 0
--------------------- >> end captured logging << ---------------------

Overwriting self.sock as None instead of deleteing it seems to work as a quick patch but we should find a better solution.

Cassette support for multiprocessing

Currently, if you use cassette within a multiprocess created process, it will not save the recorded responses.

Simple example wherein "responses.yaml" is not created

import multiprocessing
import urllib2

import cassette


def hi(*args):

    r = urllib2.urlopen("http://www.internic.net/domain/named.root")
    r = urllib2.urlopen("http://www.internic.net/domain/named.root")

    assert "A.ROOT-SERVERS.NET" in r.read(10000)


if __name__ == '__main__':
    cassette.insert("responses.yaml")

    p = multiprocessing.Process(target=hi)
    p.start()
    p.join()

    cassette.eject()