23Q4 Performance Regressions as Revealed in TechEmpower Results about ntex HOT 5 CLOSED

i remember now, i did experiment with number of worker threads

TechEmpower/FrameworkBenchmarks#8535

i changed it back but new run never completed
TechEmpower/FrameworkBenchmarks#8569

in next run numbers should recover

from ntex.

You can clone the benchmark and run it locally to find out what commit/change is the cause.

from ntex.

i dont remember drop in performance from ntex upgrade, could you include links to results. i remember there was some problems with techempower environment at some point.

from ntex.

Hi @fafhrd91

Yes, sure.

Here are links to several random intermittent runs in 2023, up to and including the one that started Nov 12 2023. They all have Data Update category results for ntex well above 20K/s:
May 7: https://www.techempower.com/benchmarks/#section=test&runid=bc7f7570-a88e-48e3-9874-06d7dc0a0f74
Sep 19: https://www.techempower.com/benchmarks/#section=test&runid=de12400a-863e-4f9e-9b93-dea9eb260422
Oct 24: https://www.techempower.com/benchmarks/#section=test&runid=0d656625-036a-4b2f-b255-cd3ca188a42c
Nov 12: https://www.techempower.com/benchmarks/#section=test&runid=99e0b150-7cac-4fe7-9b10-5eb526d877ef

And here are the only two finished runs that started beginning Nov 19, 2023. Both of them show ntex with ~5K/s fewer Data Updates (more than 20% less) compared to all the tests above:
Nov 19: https://www.techempower.com/benchmarks/#section=test&runid=502e3495-dda5-4232-a89a-7c08755fcf0e
Nov 26: https://www.techempower.com/benchmarks/#section=test&runid=341a0547-80f0-45f8-9245-04ae0e0e1bcf

All other top-20 frameworks stayed within the top 20 across the Nov 12 and Nov 19 results, including Axum, Drogon, and Just-JS (all written in different langs). Only ntex fell from #1 on Nov 12 to #30 on Nov 19 and #32 on Nov 26.

PS: I just noticed TechEmpower hasn't completed any more intermittent runs since the one that started Nov 26, though they have one incomplete run that began in December. Before that, they made several runs per month. A banner at the top of the page at https://tfb-status.techempower.com/ explains they are getting new server hardware (hence the pause).

from ntex.

Awesome. Glad this was taken care of!

from ntex.