garnaat / placebo Goto Github PK

Hi, I have the following test below that puts simple text into an object:

from nose2.compat import unittest
import boto3
import placebo

class S3PlaceboTestCase(unittest.TestCase):

    def setUp(self):
        super().setUp()
        self.session = boto3.Session()


class S3PlaceboRecordCase(S3PlaceboTestCase):

    def setUp(self):
        super().setUp()
        path = 'responses'
        self.pill = placebo.attach(self.session, data_path=path)
        self.pill.record(services='s3')
        self.s3 = self.session.client("s3")

    def tearDown(self):
        super().tearDown()
        self.pill.stop()

    def test_pill_record(self):
        self.s3.put_object(Bucket='my_bucket', ContentType='text/plain', Key='pill_test.txt', Body='Sample text')
        obj = self.s3.get_object(Bucket='my_bucket, Key='pill_test.txt')
        self.s3.delete_object(Bucket='my_bucket', Key='pill_test.txt')
        self.assertTrue(True)

And when I run the test I'm getting:

  File "/usr/share/stuff-site-adcontrol/.test-venv/lib/python3.4/site-packages/placebo/serializer.py", line 60, in serialize
    raise TypeError("Type not serializable")
TypeError: Type not serializable

Which is strange because it's simple text.

Might be related to #5

Hi,

I have a unit test set up where I have recorded the results from a valid run, and an invalid run.

I have 2 recorded placebo files, one in each directory, structured like this ('./goodtest/xxx_1.json', './badtest/xxx_1.json')

I have 2 tests, where each test loads from a separate directory, like so:

def test_verify_invalid(self):
    invalid_session = boto3.Session()
    pill = placebo.attach(invalid_session, data_path='./badtest/')
    pill.playback()

    with Replacer() as r:
        r.replace('src.SESSION', invalid_session)
        self.assertFalse(verify_thing(BADARGS))
        pill.stop()

def test_verify_valid(self):
    valid_session = boto3.Session()
    pill = placebo.attach(valid_session, data_path='./goodtest/')
    pill.playback()

    with Replacer() as r:
        r.replace('src.SESSION', valid_session)
        self.assertTrue(verify_thing(GOODARGS))
        pill.stop()

However, I get the 'bad' event for both tests. I have verified that if I take the 'good' output, and put it into the badtest folder (the one that is loaded first) and name it 'xxx_2.json' , it works fine.

Am I doing something wrong? I could have them both in the same folder, but that would require specific ordering of my tests, which doesn't seem right to me.

Thanks!

boto/botocore#741

When using placebo.attach() on two different methods within a unittest Class, on the second method the code aborts due to maximum recursion depth exceeded.

This is the failing line

File "...placebo/pill.py", line 109, in _init_method
   super_cls.__init__(self, *args, **kwargs)

This was called by botocore/client

During boto3.resource('s3').Object(bucket, key).get() TypeError raised and invalid json is recorded, while a response is totally valid. bytes type is not json serializable. And response body is a file which is of bytes type. But this is not a reason to raise an error and record invalid json. I propose to stub body with empty value and finish recording without error because lack of file is not an issue for unit testing but invalid record is.

Placebo is a great tool but I am finding the behaviour of Placebo to read from response files that are internally numbered from 1..n makes it difficult to structure maintainable unit tests.

Would it be possible to add a feature, maybe some kind of counter reset mechanism perhaps? that would allow us group response files in separate directories on a per test case basis to make it easier to make understandable code? Or is this already possible somehow?

Today I had the problem with the following code in record mode:

            stdout_key = self._s3.Object(bucket_name='somebucket',
                                         key='{}/stdout'.format(location))
            result_output = stdout_key.get()['Body'].read()

No matter what, result_output was always ''. After some digging, I was able to determine that the stream was already read.

(Pdb) test = stdout_key.get()['Body']
(Pdb) test._raw_stream.tell()
5

Important note: the tests run fine and work as expected when in playback mode, and the content of the StreamingBody is saved correctly to the cassette.

My assumption: Placebo is read()ing the StreamingBody while in record mode so that it can save it to the cassette.

Is there any way we can still present the results correctly? I'd like to avoid having my tests fail in record mode and then pass during playback mode. :-P

One thing I'm having trouble wrapping my head around, which maybe should be clarified in the readme: if I have an existing Boto application that I want to write unit tests for, and I want to use Placebo, how do I do so?

The main problem that I'm seeing is that loading and saving the stored responses happens on the boto client object, which is often created inside a class object or function; there's not much utility to passing clients as function arguments, because they're just containers for the API methods. (As far as I know, at least - feel free to correct me if passing them as arguments is better practice.)

If the client isn't being passed to the class or function, then as far as I can tell I have to pass parameters to it instead, telling it whether to save or load the placebo data, before I can access that behavior in my unit test. That also implies adding code to act on those parameters ("if save_placebo, then record and save; if load_placebo, then load the json before making any calls").

My concern with doing it this way is that it pollutes the module being tested with lots of scaffolding code that doesn't actually contribute to the application's purpose, or get used in production. Something like Mock, or Moto, for example, can typically be set up entirely in the test module, without affecting the original code's readability.

@garnaat - if I was going to use Placebo to test, say, a CLI tool - how would you recommend applying it while changing the application code as little as possible?

This could be me but I ran into an edge case where I seem to be required to have placebo statements in my application code.

I have a project structure like this:

▶ find .          
.
./tests
./tests/test_lambda_function.py
./src
./src/lib
./src/lib/__init__.py
./src/lib/foo.py
./src/lambda_function.py

• The call to AWS occurs in src/lib/foo.py.

• I initialise placebo in tests/test_lambda_function.py.

This did not work until I moved these lines into src/lib/foo.py:

boto3.setup_default_session()
session = boto3.DEFAULT_SESSION
pill = placebo.attach(session, data_path="tests/fixtures")

if "BOTO_RECORD" in os.environ:
    pill.record()
elif "BOTO_PLAYBACK" in os.environ:
    pill.playback()

The file foo has content in it like:

import boto3

client = boto3.client("route53")

def get_policy_instance_id(policy_id: str) -> Union[str, None]:
    response = client.list_traffic_policy_instances()
    for policy_instance in response["TrafficPolicyInstances"]:
        if policy_instance["TrafficPolicyId"] == policy_id:
            return policy_instance["Id"]
    return None

def create(hosted_zone_id: str,
           ttl: int,
           domain_name: str,
           traffic_policy_id: str,
           traffic_policy_version: int) -> dict:

    return client.create_traffic_policy_instance(
        HostedZoneId=hosted_zone_id,
        Name=domain_name,
        TTL=ttl,
        TrafficPolicyId=traffic_policy_id,
        TrafficPolicyVersion=traffic_policy_version
    )
...

Is this a known issue, or am I doing something wrong?

some of the s3 request/response handlers in latest boto expect that attribute, though they all seem to short circuit cleanly if the value is None.

    v = method(Bucket=b['Name'])                                                                                                                                                             
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)                                                                                                                                        
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/client.py", line 615, in _make_api_call                                                     
    self.meta.events.emit(                                                              
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)                             
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)                   
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)                               
  File "/Users/kapilt/.pyenv/versions/3.8.2/envs/c7n-poet-v1/lib/python3.8/site-packages/botocore/handlers.py", line 487, in parse_get_bucket_location
    if http_response.raw is None:                                     
AttributeError: 'FakeHttpResponse' object has no attribute 'raw'                            

Using placebo==0.7.1 and following test suite:

import pytest
from py.path import local
import sys


@pytest.fixture
def session():
    import boto3
    return boto3.Session()


@pytest.fixture(scope="session")
def records_path():
    """(py.path.local) directory for recorded responses"""
    path = local("tests/records/logs")
    assert path.exists()
    return path


def get_groups(client):
        paginator = client.get_paginator('describe_log_groups')
        for page in paginator.paginate():
            for group in page.get('logGroups', []):
                yield group['logGroupName']


def test_pill_playback_client(session, records_path):
    import placebo
    pill = placebo.attach(session, data_path=records_path.strpath)
    pill.playback()
    client = session.client("logs")
    groups = list(get_groups(client))
    print("groups", groups)
    pill.stop()


def test_pill_client_playback(session, records_path):
    import placebo
    pill = placebo.attach(session, data_path=records_path.strpath)
    client = session.client("logs")
    pill.playback()
    groups = list(get_groups(client))
    print("groups", groups)
    pill.stop()

the last test fails. It looks like the client must be created after call to pill.playback().

For my case (awslogs with AWSLogs() class which takes care of session and client creation at
__init__() call it does not work well, using placebo for unit tests there would require
modification of AWSLogs class to allow creation of instance passing existing session as
construction parameter.

Is there something I missed? Shall I dive deeper into issue #9 to learn some known workaround?

When recording AWS API's for unit tests. I would love to be able to mask the account number in the ARN with a fake number provided by AWS. 123456789012

It would be as simple as re.sub(':(\d{12}:)', ':123456789012:', response_data) somewhere here https://github.com/garnaat/placebo/blob/develop/placebo/pill.py#L242-L243.

It would be good to have a flag if someone wanted to mask or not mask the account number of the arn.

Thoughts? @garnaat

The README.md references a data_dir argument for setting the save/load directory when attaching to a session. The code actually uses data_path as the argument.

We've been using Placebo in Zappa as a decorator - would it be useful for you to have this moved upstream?

placebo works fine for some of my testcases. others I run into trouble during playback e.g.:

 response = client.describe_stack_events(StackName=stack_id)
            for event in response['StackEvents'][::-1]:
>               if event['EventId'] not in seen_events and event['Timestamp'] > now:
E               TypeError: can't compare offset-naive and offset-aware datetimes

any idea how to tackle this?

new to placebo so this is most likely my fault. Any idea how to fix this?

___________________________________________________________ test_list_stacks ____________________________________________________________

args = (), kwargs = {}, session_kwargs = {'profile_name': 'superuser-dev', 'region_name': 'eu-west-1'}
profile_name = 'superuser-dev', session = Session(region='eu-west-1')

    @functools.wraps(function)
    def wrapper(*args, **kwargs):
        session_kwargs = {
            'region_name': os.environ.get('AWS_DEFAULT_REGION', 'us-east-1')
        }
        profile_name = os.environ.get('PLACEBO_PROFILE', None)
        if profile_name:
            session_kwargs['profile_name'] = profile_name

        session = boto3.Session(**session_kwargs)

>       self = args[0]
E       IndexError: tuple index out of range

../venv/local/lib/python2.7/site-packages/placebo/utils.py:29: IndexError
======================================================= 1 failed in 0.73 seconds ========================================================

this is my simple testcase:

from placebo.utils import placebo_session
...

@pytest.mark.aws
@check_preconditions
@placebo_session
def test_list_stacks(session):
    out = StringIO()
    list_stacks(session, out=out)
    assert_regexp_matches(out.getvalue().strip(), 'listed \d+ stacks')

I am using placebo in record mode like in the docu:

$ PLACEBO_PROFILE=superuser-dp-dev PLACEBO_MODE=record python -m pytest tests/test_aws.py::test_list_stacks

Trying to use placebo in my pytest based suite, I have found, that in some scenarions (recording responses) I have forgotten to pill.stop() recording. For this reason my recording test was not failing when it was supposed to (when I disconnected from Internet).

For this reason I have decided to use @pytest.fixture called recording_pill, which at teardown calls pill.stop().

Anyway, I have found, I need to get the session bound to the pill and the only way to do it now is to ask pill._session breaking the rule "we are all adults" and accessed the private property.

It would be handy to have (possibly read-only) session property on the Pill class.

Here is my current test suite tests/test_logs.py:

import pytest


@pytest.fixture
def session():
    import boto3
    return boto3.Session()


@pytest.fixture(scope="session")
def records_path(tmpdir_factory):
    """(py.path.local) directory for recorded responses"""
    path = tmpdir_factory.mktemp("logs")
    path.ensure_dir()
    return path


@pytest.yield_fixture
def recording_pill(session, records_path):
    import placebo
    pill = placebo.attach(session, data_path=records_path.strpath)
    pill.record()
    yield pill
    # tear down
    pill.stop()


@pytest.yield_fixture
def replaying_pill(session, records_path):
    import placebo
    pill = placebo.attach(session, data_path=records_path.strpath)
    pill.playback()
    yield pill
    # tear down
    pill.stop()


def get_groups(client):
        paginator = client.get_paginator('describe_log_groups')
        for page in paginator.paginate():
            for group in page.get('logGroups', []):
                yield group['logGroupName']


def test_recording(recording_pill):
    session = recording_pill._session  # better access public property here
    client = session.client("logs")
    groups = list(get_groups(client))
    assert groups


def test_playback(replaying_pill):
    session = replaying_pill._session  # better access public property here
    client = session.client("logs")
    groups = list(get_groups(client))
    assert groups

I could possibly pass the session value into test_recording and
test_playback, but this is tricky as I really want to be sure, it is exactly
the session the pill was created for.

I'm wondering if there is anyway to combine placebo functionality with mock. I'd like do something like this:

session = boto3.Session()
pill = placebo.attach(session, data_path=data_path)
pill.playback()
autoscaling = session.client('autoscaling')

# sut setup here
sut.kill_inst_in_asg('foo')
# the system under test (sut) calls:
# autoscaling.terminate_instance_in_auto_scaling_group(InstanceId='foo')

pill.get_method('terminate_instance_in_auto_scaling_group').assert_called_with('foo')
# or
pill.get_method('terminate_instance_in_auto_scaling_group').assert_called_with(InstanceId='foo')

#53

Apparently #15 was an incomplete description of the issue. While data_path was used in the code and data_dir in the docs, the attach interface defined in __init__.py uses data_file.

While the interface works fine with positional arguments, the attach example in the readme uses a named argument and still causes an exception.

This should be an instance variable. Otherwise, if you have multiple clients all responses will be merged into the same dictionary.

placebo provides feature to save recorded requests, but for this it has to talk to real AWS service to get some response what requires some credentials being in place and also breaks the assumption that unit test shall not be dependent on other things around.

In my current PR #37 I have a problem, that the test case is failing in different way in my own environment and at Travis.

In current test suite I also did not find a test, which would save responses by calling some real boto3 operations - all recordings is done by pill.save_response call.

Do you feel, moto could serve as solution for this? We could use moto to create a S3 bucket, put there some objects and expect AWS calls in test case to use this mocked service.

What do you think of it?

The following code causes placebo to crash:

iot_data = session.client(client='iot-data')
.....
iot_data.meta.placebo.save('iot-data')

On line 49 in placebo.py:

_, service_name, operation_name = kwargs['event_name'].split('.')

Where kwargs['event_name'] is "after-call.data.iot.GetThingShadow"

I ran into another issue, one of the API calls returns a botocore.streamingBody, which causes an issue on JSON serialization:

>>>>> python iot_test.py 
{'data.iot.GetThingShadow': {'index': 0, 'responses': [(200, {u'payload': <botocore.response.StreamingBody object at 0x109db4a10>, 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '73d3ede2-cd17-4bf7-afe5-0facc8f194a1'}})]}}
Traceback (most recent call last):
  File "iot_test.py", line 8, in <module>
    iot.meta.placebo.save('my_saved_iot_calls.json')
  File "/usr/local/lib/python2.7/site-packages/placebo.py", line 68, in save
    json.dump(self.mock_responses, fp, indent=4, default=serialize)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 189, in dump
    for chunk in iterable:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/local/lib/python2.7/site-packages/placebo.py", line 164, in serialize
    raise TypeError("Type not serializable")
TypeError: Type not serializable

They both appear to have vanished into the ether.

Is it currently possible to use placebo to test functions that use multiple regions?

My understanding is that a session is bound to a region. If I have a function that uses mutltiple regions and as a result has multiple sessions, should attaching a pill to each session (and playing each session back) function?

Thanks!!

This is due to the encoding of the response into JSON. It looks like using a base64 encoder with image responses would work for saving/loading. I'm not sure what effect that would have on other response types though or how to specify the encoder based on the boto method used.

I'm trying to use Placebo in Python 3, and getting an exception TypeError: a bytes-like object is required, not 'str' when trying to save the API calls.

The issue is with the save method, line 60 of placebo.py:

    def save(self, path):
        with open(path, 'wb') as fp:
            json.dump(self._mock_responses, fp, indent=4)

The target file is being written as binary. In Python2 8-bit strings also handle byte sequences, but in Python3 byte sequences have their own bytes type, causing the TypeError. (https://www.python.org/dev/peps/pep-0404/#strings-and-bytes)

@garnaat Was there a specific reason for the binary mode on the save() and load() methods? If not, the easiest fix is probably to just change them to 'r' and 'w'.

How might one use this framework to also test exceptions being raised? For instance, trying to terminate an instance that is already terminated.

Some API calls in boto3 include datetime objects in the dictionary return values. Datetime is not one of the types that the json package understands, causing an exception when it tries to serialize it into a JSON file for the save() method:

Traceback (most recent call last):
  File "/usr/local/bin/testcode", line 9, in <module>
    load_entry_point('testcode==0.1', 'console_scripts', 'testcode')()
  File "/usr/local/lib/python2.7/site-packages/testcode-0.1-py2.7.egg/testcode/main.py", line 511, in main
    user.save_placebo()
  File "/usr/local/lib/python2.7/site-packages/testcode-0.1-py2.7.egg/testcode/main.py", line 228, in save_placebo
    self.client.meta.placebo.save(path)
  File "build/bdist.macosx-10.10-x86_64/egg/placebo.py", line 60, in save
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 189, in dump
    for chunk in iterable:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2015, 12, 3, 21, 20, 17, 326000, tzinfo=tzutc()) is not JSON serializable

The json.dump() method takes an optional argument default, allowing you to pass a function that returns serializable versions of objects. There's an example for datetime on Stack Overflow here:
http://stackoverflow.com/a/22238613

Similarly, the load() method will need to convert ISO 8601 strings back to datetime objects before using them for mocking. This could get tricky if someone uses timestamps for AWS object names, but maybe they could be marked for conversion. For example, 'placebo_timestamp:2015-12-3-21:20:17:326000Z' is hopefully pretty unlikely as an AWS object name.

I will try to submit a PR for this soon.

I have a Lambda function that I want to test with Placebo. The function (image_search.py) has been simplified, but is similar to this:

import boto3

ec2 = boto3.client("ec2")

def get_images(os_version):
    images = ec2.describe_images(
        Filters=[{"Name": "tag:OS_Version", "Values": [os_version]}], Owners=["self"]
    )
    return images

def lambda_handler(event, context):
    os_version = event["OS_Version"]
    images = get_images(os_version)

From what I've read, it's best to setup the client outside of the handler function to take advantage of Lambda functions that get re-used. I manually recorded responses with Placebo and they're in a folder ready to go.

When I try to unit test with Pytest however, it always fails with "Request has expired" which tells me that it's trying to run live calls. I think it's because when I import the image_search module, it's creating a client before I can attach Placebo to it.

import placebo
from src import image_search

# begin tests here
def test_get_images():
    boto3.setup_default_session()
    session = boto3.DEFAULT_SESSION
    pill = placebo.attach(session, data_path='/path/to/response/directory')
    pill.playback()
    images = rotate_amis.get_images("amazon-linux2")
    tags = images["Images"][0]["Tags"]
    for tag in tags:
        if tag["Key"] == "OS_Version":
            value = tag["Value"]

    assert value == "amazon-linux2"

However if i lazy load the image_search module after I setup Placebo, the tests pass.

import placebo

# begin tests here
def test_get_images():
    boto3.setup_default_session()
    session = boto3.DEFAULT_SESSION
    pill = placebo.attach(session, data_path='/path/to/response/directory')
    pill.playback()
    from src import image_search

    images = rotate_amis.get_images("amazon-linux2")
    tags = images["Images"][0]["Tags"]
    for tag in tags:
        if tag["Key"] == "OS_Version":
            value = tag["Value"]

    assert value == "amazon-linux2"

However this seems really hacky - what is the best way to handle something like this?

As was discussed in #36 (comment) it would be really nice if placebo could automatically capture the parameters/arguments for API calls, and then play the calls back according to the parameters instead of order (perhaps using order only if there are multiple calls with the same parameters).

I hope it's OK to ask a question here. I can't find this explained in the docs.

I have legacy code that does a lot of stuff using the default Boto3 session, e.g.

import boto3
client = boto3.client('ec2')
client.describe_images(DryRun=False, ImageIds=[ami_id])
...

However, docs here at this project seem to imply that I need to manage the session explicity, i.e.

import boto3
session = boto3.Session()
client = session.client('ec2')
client.describe_images(DryRun=False, ImageIds=[ami_id])
...

Then you need to attach Placebo to session.

But is it possible to attach Placebo somehow to the default session -- or would I have to refactor all my legacy code before I could use Placebo on it?

Thanks,

LICENSE should just be Apache2, at the moment setup.py is long forming the license.txt
ie. see https://pypi.python.org/pypi/placebo/0.4.1

Instead it should be long forming the readme as long_description for display on pypi page.

Hi,

Thanks for fixing the last issue! I figured this problem was different enough to need its own issue.

placebo crashes when I make an API call that returns a botocore.response.StreamingBody:

{'data.iot.GetThingShadow': {'index': 0, 'responses': [(200, {u'payload': <botocore.response.StreamingBody object at 0x109db4a10>, 'ResponseMetadata': {'HTTPStatusCode': 200, 'RequestId': '73d3ede2-cd17-4bf7-afe5-0facc8f194a1'}})]}} Traceback (most recent call last): File "iot_test.py", line 8, in <module> iot.meta.placebo.save('my_saved_iot_calls.json') File "/usr/local/lib/python2.7/site-packages/placebo.py", line 68, in save json.dump(self.mock_responses, fp, indent=4, default=serialize) File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 189, in dump for chunk in iterable: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 434, in _iterencode for chunk in _iterencode_dict(o, _current_indent_level): File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict for chunk in chunks: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict for chunk in chunks: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list for chunk in chunks: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list for chunk in chunks: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict for chunk in chunks: File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 442, in _iterencode o = _default(o) File "/usr/local/lib/python2.7/site-packages/placebo.py", line 164, in serialize raise TypeError("Type not serializable") TypeError: Type not serializable

there's a couple of important fixes between the two (#50 and #51), but without a release its unclear if most users will find them except the hard way, and then have to update deps to use git checkouts :-( Much nicer would be releasing 0.8.2 to pypi

I submitted a pull request a while back to add an optional 'prefix' argument, which I needed for tracking multiple sessions with Placebo. It's been merged into the project. but the latest '0.4.3' release does not have the argument.

Can we get a release pushed to PyPI that has the argument included? I already have code written that is depending on it, but I'm stuck importing my local development version until the feature is available in the public release.

I noticed that each time Placebo writes a new response JSON file, it adds a _<number> suffix before the .json, and simply increments the suffix until it finds one that doesn't exist yet. This is better than overwriting previous runs, but doesn't give any way to distinguish multiple runs.

What would be really nice, is if you could optionally set a prefix argument on a Pill object. This prefix would be applied to all filenames written by the pill, like so: prefix.iam.ListUserPolicies_1.json.

I can think of two good use cases for this:

1. Define the prefix uniquely on each execution, using a command line argument, timestamp, or some other generated value. Now individual executions can be reloaded and replayed, or even diffed for comparison.
2. For an application that uses multiple Boto sessions, define a different prefix for each session. For example, I can use base for my initial credentials session, mfa for my new session after MFA authentication, and <rolename> for a session created using an IAM role. I can even combine that with a timestamp (as in case 1) to separate multiple runs.

Error while installing Kappa

Processing dependencies for kappa==0.3.1
Searching for placebo>=0.4.1
Reading https://pypi.python.org/simple/placebo/
Best match: placebo 0.4.2
Downloading https://pypi.python.org/packages/source/p/placebo/placebo-0.4.2.tar.gz#md5=e9889017cd87355684fe9c6f2071d0d7
Processing placebo-0.4.2.tar.gz
Writing /var/folders/9k/4xz7zjt95qgg1_tr6rnvzghr0000gn/T/easy_install-rRI3Av/placebo-0.4.2/setup.cfg
Running placebo-0.4.2/setup.py -q bdist_egg --dist-dir /var/folders/9k/4xz7zjt95qgg1_tr6rnvzghr0000gn/T/easy_install-rRI3Av/placebo-0.4.2/egg-dist-tmp-L8eSzP
error: [Errno 2] No such file or directory: 'README.md'

Hi Garnaat,

I'm trying to configure placebo to test my code, but I think I'm doing something bad. I not be able to configure placebo to work in local enviroment, I must use real credentials and when I use placebo to create a data base, It be created in AWS DynamoDB in my real account, if I use a not real credentials I get a conecttion error "ClientError: An error occurred (UnrecognizedClientException) when calling the CreateTable operation: The security token included in the request is invali"d This is my code, Could you help me please?

import unittest

import boto3
import placebo
from placebo.utils import placebo_session


class TestIntegrationDynamodbTableDao(unittest.TestCase):
    __access_key = "Fake"
    __secret_key = "Fake"
    __token = "Fake"
    __region = "local"
    __table = None
    __resource = None
    __pill = None

    def setUp(self):
        pass

    def tearDown(self):
        self.__pill.stop()

    def test_given_when_then(self):
        # self.__session = boto3.Session(aws_access_key_id=self.__access_key, aws_secret_access_key=self.__secret_key, aws_session_token=self.__token,
        self.__session = boto3.Session(aws_access_key_id=self.__access_key, aws_secret_access_key=self.__secret_key, region_name=self.__region)
        self.__pill = placebo.attach(self.__session, data_path='/home/ivan/tmp')
        # self.__pill.record(services='dynamodb')
        # self.__pill.record()
        self.__resource = self.__session.resource('dynamodb')
        self.__pill.playback()
        self.__resource.create_table(TableName='test',
                                     KeySchema=[{'AttributeName': 'zn', 'KeyType': 'HASH'},
                                                {'AttributeName': 'dt', 'KeyType': 'RANGE'}],
                                     AttributeDefinitions=[{'AttributeName': 'zn', 'AttributeType': 'S'},
                                                           {'AttributeName': 'dt', 'AttributeType': 'S'}],
                                     ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5})
        self.__pill.stop()
        print "hello"


if __name__ == '__main__':
    unittest.main()

When using placebo.utils.placebo_session as a decorator, I had the following error:

__________________________________________ test_publish_incidents_placebo __________________________________________

args = (), kwargs = {}, session_kwargs = {'region_name': 'us-east-1'}, profile_name = None
session = Session(region_name='us-east-1')

    @functools.wraps(function)
    def wrapper(*args, **kwargs):
        session_kwargs = {
            'region_name': os.environ.get('AWS_DEFAULT_REGION', 'us-east-1')
        }
        profile_name = os.environ.get('PLACEBO_PROFILE', None)
        if profile_name:
            session_kwargs['profile_name'] = profile_name

        session = boto3.Session(**session_kwargs)

>       self = args[0]
E       IndexError: tuple index out of range

env/lib/python2.7/site-packages/placebo/utils.py:29: IndexError
======================================== 1 failed, 5 passed in 1.92 seconds ========================================
make: *** [test] Error 1