Upgrading to v1.39.1 throws express-rate-limit Errors - CANNOT LOG IN TO UI about n8n HOT 10 CLOSED

More details about this issue are here:
https://github.com/express-rate-limit/express-rate-limit/wiki/Troubleshooting-Proxy-Issues

from n8n.

And also here:
GoogleCloudPlatform/nodejs-docs-samples#3583

from n8n.

It looks like this is the code that's needed to be added to n8n:
https://github.com/GoogleCloudPlatform/nodejs-docs-samples/pull/3586/files#diff-03f690ebd3c288db3f9f3fc236259f272a1a8105d8587580d24d880000eda715R41

But I'm still trying to figure out where that equivalent code lives within the n8n codebase...

from n8n.

So it looks like this param WAS declared here:
https://github.com/n8n-io/n8n/blob/master/packages/cli/src/AbstractServer.ts#L71

And it's feature is anchored to the proxy_hops ENV variable... investigating further...

from n8n.

Okay, so there's an environment variable named N8N_PROXY_HOPS apparently... but it appears to be completely undocumented...

from n8n.

Okay, I set N8N_PROXY_HOPS=1 and it did NOT resolve the issue. This is incredibly frustrating...

from n8n.

Update: So apparently, changing N8N_PROXY_HOPS=1 did fix the first issue. But there was a second issue.

Essentially, for some reason, my browser was using an old (cached?) set of credentials/cookies to access n8n. Because those credentials expired upon upgrade, the client-side code did NOT respect that expiration -- and the browser attempted to retry using that expired cookie.

Because the browser kept trying to query n8n using expired credentials, express started throwing 429 errors (e.g., too many requests)... and that started happening even before I could properly log in to the UI to refresh my credentials/cookies.

So, the ultimate solution was to hard restart Google Chrome browser, manually flush all local cookies, wait a little bit of time (?), and THEN try again.

At that point, I was able to successfully log back into the N8N UI.

Thankfully, all of these issues did NOT seem to impact current running workflows at the time, but it was an incredibly frustrating experience upgrading -- mainly because none of these issues appeared to be documented ahead of time.

I want to upgrade n8n more often, but if this is how upgrades go, I don't think I can unless I set aside 2-3 hours to expect some sort of unexpected breakage.

from n8n.

Ultimately, I think the open item with this issue is to somehow provide documentation about:

• What N8N_PROXY_HOPS is for
• When to use it
• When to not use it
• And also call out in the release notes about this new issue -- if you're running n8n inside something like a Google Cloud Run service

from n8n.

I think it is safe to close this issue as there isn't a bug here but there is a learning on the proxy_hops, Oddly enough the N8N_PROXY_HOPS key was added back in 1.16.0 when we introduced rate limiting for the forgot password endpoint.

I suspect this could be triggered by a mix of the cache causing the lockout originally and the header issue being highlighted by another change.

from n8n.

Hey @Joffcom -- makes sense -- could you guys at least update the n8n documentation (https://docs.n8n.io/hosting/) to explain what N8N_PROXY_HOPS is for?

from n8n.