Intermittent CI fail with ratelimit about django-ratelimit HOT 3 CLOSED

Ratelimits are calculated on staggered, fixed windows, i.e., for a given set of group/path/methods/key-value/period, ratelimit will calculate some value that will be the same every time it's calculated for period seconds (it uses the value to stagger them so, e.g. not all hour ratelimits expire at the top of the hour).

So, imagine the limit is 1/min, the second attempt (&) will be ratelimited, but the 3rd won't be, even though it's within 1 minute of the first:

period:       |             |
attempts:            ^  &    ^

The reason for this is that if you reset the period every time (what ratelimit used to do) then you can end up blocking even good actors basically forever (e.g. they try every 59 seconds, continually pushing the limit out and never being allowed).

So what's probably happening is that every once in a while, you cross the boundary during that loop. The odds are bad—and your flakiness rate is low—but it's entirely possible.

You could probably simplify this by mocking ratelimit.utils.is_ratelimited to always return True during this test, instead of trying to actually trip the limit. Alternatively, set the number of attempts to twice the limit, to make sure you hit it even if you cross the boundary (but you'll have to drop some of the assertions within the loop).

from django-ratelimit.

Thanks a lot, makes complete sense, this had occurred to me but I assumed the limit was reset every time.

Thanks for your help.

from django-ratelimit.

For anyone else having trouble with ratelimt and CI, my solution is:

    wrong_credentials = {'username': '[email protected]', 'password': 'wrong'}
    found_captcha = False
    for _ in range(7):
        response = self.client.post(self.login_url, wrong_credentials)
        self.assertEqual(response.status_code, 200)
        self.assertFalse(response.context['login_form'].is_valid())
        if 'captcha' in response.content:
            found_captcha = True
            break
    self.assertTrue(found_captcha, 'captcha not found on login page after 7 failed login attempts')

from django-ratelimit.