epsagon / epsagon-python Goto Github PK

Hi, we're looking at using Epsagon in our Lambdas but we're running into an odd issue. When a part of the trace built by Epsagon contains a datetime value, the trace fails to be uploaded to your service. Our suggestion is to use either specialist functions to serialise types like datetime (this is in our patch) and/or to fallback on str if the type is not serialisable (assuming that more types support str than JSON serialisation).

We've prepared a sample to reproduce the issue and an initial patch. I hope both help.

Sample:

from epsagon import lambda_wrapper
from datetime import date, datetime
from typing import Dict, List


@lambda_wrapper
def handler(event: Dict, context: Dict) -> List[Dict]:
    from_date = event.get("from_date") if event else None

    from_date = (
        datetime.strptime(from_date, "%Y-%m-%d")
        if from_date
        else date.today().replace(day=1)
    )

    return get(
        from_date=from_date,
    )

def get(from_date: datetime.date = None):
    # Use the datetime.
    print(from_date.isoformat())
    return [from_date.strftime("%Y-%m-%d")]

Patch:

--- build/epsagon/trace.py    2019-04-30 20:26:01.000000000 +0100
+++ trace.py    2019-04-30 20:21:34.000000000 +0100
@@ -12,6 +12,8 @@
 import pprint
 import simplejson as json

+
+from datetime import date, datetime
 import requests
 import requests.exceptions
 from epsagon.event import BaseEvent
@@ -355,7 +357,8 @@
             trace = json.dumps(
                 self.to_dict(),
                 cls=TraceEncoder,
-                encoding='latin1'
+                encoding='latin1',
+                default=json_serial
             )
             SESSION.post(
                 self.collector_url,
@@ -382,6 +385,10 @@
             if self.debug:
                 pprint.pprint(self.to_dict())

+def json_serial(obj):
+    """JSON serializer for objects not serializable by default json code"""

+    if isinstance(obj, (datetime, date)):
+        return obj.isoformat()
 # pylint: disable=C0103
 tracer = Trace()

Hi, this is not really a issue, I just needed more info on communication from your service to our serverless infra.
I tried asking in help widget but didn't get any reply.
We really liked this service and want to integrate it to monitor our serverless architecture,but our security team raised few questions over communication between your server and our infra.
We are bit confused about how do you send any updates made in dashboard to reflect in our infra, is it via cloudformation custom resource (SNS in this case)?

Thank you.
Sorry for raising this in an inappropriate place.

Hi guys,

While trying to install epsagon I faced issue with couple outdated dependencies:

epsagon 1.0.46 has requirement future==0.16.0, but you'll have future 0.17.1 which is incompatible.
epsagon 1.0.46 has requirement six==1.11.0, but you'll have six 1.12.0 which is incompatible.

It causes dependency conflict with other packages in my project.

Could you please update requirements if it is possible?

After installing epsagon on my FastAPI application, I get the following error: (route) got an unexpected keyword argument 'epsagon_request'

My configuration is pretty straight forward:

epsagon.init(
token='token,
app_name='my-app',
metadata_only=False,
)

Hi there,
I've already talked to @ranrib on Slack about this issue.

I'm posting it here for keeping track of it.

epsagon wrapper for chalice breaks chalice local server:

unicorn_1     | Exception happened during processing of request from ('172.21.0.1', 43164)
unicorn_1     | Traceback (most recent call last):
unicorn_1     |   File "/usr/local/lib/python3.8/socketserver.py", line 650, in process_request_thread
unicorn_1     |     self.finish_request(request, client_address)
unicorn_1     |   File "/usr/local/lib/python3.8/socketserver.py", line 360, in finish_request
unicorn_1     |     self.RequestHandlerClass(request, client_address, self)
unicorn_1     |   File "/tmp/code-_Py8Si6I/lib/python3.8/site-packages/chalice/local.py", line 565, in __init__
unicorn_1     |     self.local_gateway = LocalGateway(app_object, config)
unicorn_1     |   File "/tmp/code-_Py8Si6I/lib/python3.8/site-packages/chalice/local.py", line 436, in __init__
unicorn_1     |     RouteMatcher(list(app_object.routes)),
unicorn_1     | AttributeError: 'LocalChalice' object has no attribute 'routes'
unicorn_1     | ----------------------------------------```

I didn't get the time to dig too far but I can't have epsagon traces with celery.

From what I've seen, the _get_trace function returns None, as the unique_id is None and the singleton_trace is also None.

I'm running celery with the following command:

celery -A <app_name> worker --beat --scheduler django

and I tried both the auto tracing config and calling epsagon.init directly, but the outcome is the same.

I am running a simple python server, with no framework, calling @epsagon.python_wrapper over the main function.
I use boto3 to make Put/Get/Head operations on an S3 bucket. Only Put operation registeres in the traces

I ran with debug and got the following error for each Head/Get:

[EPSAGON_DEBUG] Epsagon exception: Invalid format string
Traceback (most recent call last):
  File "C:\Git\Production2\glvideo\.venv\lib\site-packages\epsagon\modules\general_wrapper.py", line 35, in wrapper
    factory.create_event(
  File "C:\Git\Production2\glvideo\.venv\lib\site-packages\epsagon\events\botocore.py", line 2061, in create_event
    event = event_class(
  File "C:\Git\Production2\glvideo\.venv\lib\site-packages\epsagon\events\botocore.py", line 213, in __init__
    super(BotocoreS3Event, self).__init__(
  File "C:\Git\Production2\glvideo\.venv\lib\site-packages\epsagon\events\botocore.py", line 77, in __init__
    self.update_response(response)
  File "C:\Git\Production2\glvideo\.venv\lib\site-packages\epsagon\events\botocore.py", line 254, in update_response
    response['LastModified'].strftime('%s')
ValueError: Invalid format string
-----

The bug seems to be you are using:
response['LastModified'].strftime('%s')
Instead of:
response['LastModified'].strftime('%S')

I have detected some flaky tests using pytest plugin flake finder, the details of this plugin can be found at:
https://github.com/dropbox/pytest-flakefinder
Flaky test:
test_transports.py::test_httptransport_timeout
location:

def test_httptransport_timeout():
    start_time = time.time()
    # non-routable IP address, will result in a timeout
    http_transport = HTTPTransport('http://10.255.255.1', 'token')
    trace = trace_factory.get_or_create_trace()

    # This will make sure we get TimeoutError and not MaxRetryError
    with pytest.raises(urllib3.exceptions.TimeoutError):
        http_transport.send(trace)

    duration = time.time() - start_time

    # Making sure that an unreachable url will result in duration almost equal to the
    # timeout duration set
    assert http_transport.timeout < duration < http_transport.timeout + 0.3

Where duration exceeds the timeout.
Snapshots of the error report:
FAILED tests/test_transports.py::test_httptransport_timeout[0] - assert 1.6475169658660889 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[1] - assert 1.4870219230651855 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[2] - assert 1.4910881519317627 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[3] - assert 1.4848833084106445 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[4] - assert 1.482806921005249 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[5] - assert 1.4943721294403076 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[6] - assert 1.500823974609375 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[7] - assert 1.4800028800964355 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[8] - assert 1.4843499660491943 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[9] - assert 1.4824299812316895 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[10] - assert 1.4858860969543457 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[11] - assert 1.488236665725708 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[12] - assert 1.505213975906372 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[13] - assert 1.5001637935638428 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[14] - assert 1.4944648742675781 < (1.0 + 0.3)
FAILED tests/test_transports.py::test_httptransport_timeout[15] - assert 1.4844911098480225 < (1.0 + 0.3)
How to re-appear the problem:
in the root directory, install the plugin:
pip install pytest-flakefinder
then run:
pytest --flake-finder
Conclusion:
I have encountered this flaky test, I have tried to read through the code but do not understand exactly why this happened, I am posting this as an issue to see if there is a solution to improve or this is designed on purpose.

Hello. We're using epsagon-python in Django project with gunicorn server (worker_class = "gevent"). Since version 1.49.2, our Slack notifications aren't working as expected. Instead of actual error, we can see only "Error detected" message. I tried it locally with debug option set to True, and I can see, that in version 1.49.1 there was an exception object (with type, message, traceback and so on) inside an event object in trace being sent. In version 1.49.2 and newer, there is no "exception" key in that object.