pwrdrvr / lambda-dispatch Goto Github PK
View Code? Open in Web Editor NEWLambda Dispatch for AWS Lambda - Avoid cold starts, save up to 80%!
Lambda Dispatch for AWS Lambda - Avoid cold starts, save up to 80%!
/ping
with 1 channel from Router to Extension but 10 concurrent requests into the Router from oha
, is running at 1/4 to 1/3rd the speed of the dotnet extension (e.g. 1k to 1.3k RPS vs 3k RPS)
Test Case | oha -c |
Channels Router <--> Extension |
Rust RPS | Rust CPU | Rust Node.js CPU | DotNet RPS | DotNet CPU | DotNet Node.js CPU | Direct Node.js RPS | Direct Node.js CPU |
---|---|---|---|---|---|---|---|---|---|---|
/ping |
10 | 10 | 19,007 | 90% | 113% | 14,652 | 330-350% | 95% | 20,035 | 115% |
/ping |
10 | 1 | 4,934 | 27% | 36% | 3,603 | 115% | 30% | ^^^ | ^^^ |
/ping |
1 | 10 | 4,997 | 29% | 37% | 3,451 | 111% | 27% | 15,975 | 97% |
/echo 9.3 MB |
10 | 1 | 81 | 76% | 113% | N/A | N/A | N/A | 228 | 179% |
/echo-slow 9.3 MB |
10 | 1 | 54 | 52% | 150% | 51 | 154% | 147% | 117 | 158% |
/echo-slow 9.3 MB |
1 | 1 | 54 | 52% | 150% | 50 | 152% | 150% | 96 | 215% |
/ping
- 10 Concurrent Requests - 10 Channels from Router to Extension✅ Extension CPU is ~80-90% (vs Node.js at 113% or dotnet extension at ~350%) - note that the extension is doing 2x more work than Node.js in this case (it's receiving and proxying the request and response while Node.js is only receiving the request and immediately writing back a static response - this is the worst case scenario where the contained app is using no CPU at all and only shuffling requests).
Note: Tokio set to current thread
oha -c 10 -z 10s http://localhost:5001/ping
Summary:
Success rate: 100.00%
Total: 10.0009 secs
Slowest: 0.0047 secs
Fastest: 0.0002 secs
Average: 0.0005 secs
Requests/sec: 19007.2403
Total data: 742.54 KiB
Size/request: 4 B
Size/sec: 74.25 KiB
Response time histogram:
0.000 [1] |
0.001 [165746] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.001 [19340] |■■■
0.002 [2468] |
0.002 [1803] |
0.002 [511] |
0.003 [182] |
0.003 [19] |
0.004 [7] |
0.004 [8] |
0.005 [5] |
Response time distribution:
10.00% in 0.0004 secs
25.00% in 0.0004 secs
50.00% in 0.0005 secs
75.00% in 0.0006 secs
90.00% in 0.0006 secs
95.00% in 0.0007 secs
99.00% in 0.0016 secs
99.90% in 0.0025 secs
99.99% in 0.0034 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0019 secs, 0.0014 secs, 0.0022 secs
DNS-lookup: 0.0005 secs, 0.0004 secs, 0.0006 secs
Status code distribution:
[200] 190090 responses
/ping
- 10 Concurrent Requests - 1 Channel from Router to Extension✅ Extension CPU is ~27% with current thread Tokio runtime config
oha -c 10 -z 10s http://localhost:5001/ping
Summary:
Success rate: 100.00%
Total: 10.0005 secs
Slowest: 0.0212 secs
Fastest: 0.0013 secs
Average: 0.0020 secs
Requests/sec: 4934.2600
Total data: 192.75 KiB
Size/request: 4 B
Size/sec: 19.27 KiB
Response time histogram:
0.001 [1] |
0.003 [46522] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.005 [882] |
0.007 [557] |
0.009 [530] |
0.011 [580] |
0.013 [239] |
0.015 [12] |
0.017 [9] |
0.019 [0] |
0.021 [13] |
Response time distribution:
10.00% in 0.0016 secs
25.00% in 0.0016 secs
50.00% in 0.0017 secs
75.00% in 0.0017 secs
90.00% in 0.0020 secs
95.00% in 0.0039 secs
99.00% in 0.0105 secs
99.90% in 0.0126 secs
99.99% in 0.0199 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0008 secs, 0.0006 secs, 0.0010 secs
DNS-lookup: 0.0002 secs, 0.0001 secs, 0.0003 secs
Status code distribution:
[200] 49345 responses
/echo
- 10 Concurrent Requests - 1 Channel from Router to ExtensionNote: /echo
sends/receives in duplex while /echo-slow
is all that has been implemented in the dotnet Extension; this is not the fault of dotnet (as the Router handles duplex) it's just that this was not built in the Extension. This is provided for reference as an upper bound on the throughput of the Rust extension.
oha -c 10 -z 10s -m POST -T application/octet-stream -D ./oha http://localhost:5001/echo
Summary:
Success rate: 100.00%
Total: 10.0018 secs
Slowest: 0.1470 secs
Fastest: 0.0319 secs
Average: 0.1215 secs
Requests/sec: 81.7855
Total data: 7.44 GiB
Size/request: 9.32 MiB
Size/sec: 762.18 MiB
Response time histogram:
0.032 [1] |
0.043 [0] |
0.055 [1] |
0.066 [1] |
0.078 [1] |
0.089 [1] |
0.101 [1] |
0.112 [1] |
0.124 [731] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.135 [79] |■■■
0.147 [1] |
Response time distribution:
10.00% in 0.1201 secs
25.00% in 0.1208 secs
50.00% in 0.1217 secs
75.00% in 0.1226 secs
90.00% in 0.1239 secs
95.00% in 0.1247 secs
99.00% in 0.1285 secs
99.90% in 0.1470 secs
99.99% in 0.1470 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0021 secs, 0.0017 secs, 0.0023 secs
DNS-lookup: 0.0009 secs, 0.0008 secs, 0.0009 secs
Status code distribution:
[200] 818 responses
/echo-slow
- 10 Concurrent Requests - 1 Channel from Router to Extensionoha -c 10 -z 10s -m POST -T application/octet-stream -D ./oha http://localhost:5001/echo-slow
Summary:
Success rate: 100.00%
Total: 10.0004 secs
Slowest: 0.2075 secs
Fastest: 0.0421 secs
Average: 0.1833 secs
Requests/sec: 53.9977
Total data: 4.91 GiB
Size/request: 9.32 MiB
Size/sec: 503.22 MiB
Response time histogram:
0.042 [1] |
0.059 [0] |
0.075 [1] |
0.092 [1] |
0.108 [1] |
0.125 [1] |
0.141 [1] |
0.158 [1] |
0.174 [1] |
0.191 [491] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.208 [41] |■■
Response time distribution:
10.00% in 0.1808 secs
25.00% in 0.1817 secs
50.00% in 0.1828 secs
75.00% in 0.1853 secs
90.00% in 0.1899 secs
95.00% in 0.1944 secs
99.00% in 0.2033 secs
99.90% in 0.2075 secs
99.99% in 0.2075 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0017 secs, 0.0012 secs, 0.0021 secs
DNS-lookup: 0.0005 secs, 0.0004 secs, 0.0006 secs
Status code distribution:
[200] 540 responses
/echo-slow
- 1 Concurrent Requests - 1 Channel from Router to Extensionoha -c 1 -z 10s -m POST -T application/octet-stream -D ./oha http://localhost:5001/echo-slow
Summary:
Success rate: 100.00%
Total: 10.0002 secs
Slowest: 0.0319 secs
Fastest: 0.0170 secs
Average: 0.0184 secs
Requests/sec: 54.2988
Total data: 4.94 GiB
Size/request: 9.32 MiB
Size/sec: 506.02 MiB
Response time histogram:
0.017 [1] |
0.018 [393] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.020 [117] |■■■■■■■■■
0.021 [14] |■
0.023 [6] |
0.024 [4] |
0.026 [3] |
0.027 [2] |
0.029 [0] |
0.030 [0] |
0.032 [3] |
Response time distribution:
10.00% in 0.0175 secs
25.00% in 0.0177 secs
50.00% in 0.0180 secs
75.00% in 0.0185 secs
90.00% in 0.0193 secs
95.00% in 0.0201 secs
99.00% in 0.0258 secs
99.90% in 0.0319 secs
99.99% in 0.0319 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0007 secs, 0.0007 secs, 0.0007 secs
DNS-lookup: 0.0002 secs, 0.0002 secs, 0.0002 secs
Status code distribution:
[200] 543 responses
/ping
- 10 Concurrent Requests - 10 Channels from Router to Extension🔴 Extension CPU is 330-350% (vs Node.js at 97%)
oha -c 10 -z 10s http://localhost:5001/ping
Summary:
Success rate: 100.00%
Total: 10.0006 secs
Slowest: 0.0984 secs
Fastest: 0.0002 secs
Average: 0.0007 secs
Requests/sec: 14652.3652
Total data: 572.39 KiB
Size/request: 4 B
Size/sec: 57.24 KiB
Response time histogram:
0.000 [1] |
0.010 [146522] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.020 [0] |
0.030 [0] |
0.039 [0] |
0.049 [0] |
0.059 [0] |
0.069 [0] |
0.079 [0] |
0.089 [0] |
0.098 [10] |
Response time distribution:
10.00% in 0.0004 secs
25.00% in 0.0005 secs
50.00% in 0.0005 secs
75.00% in 0.0007 secs
90.00% in 0.0009 secs
95.00% in 0.0013 secs
99.00% in 0.0037 secs
99.90% in 0.0045 secs
99.99% in 0.0079 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0019 secs, 0.0017 secs, 0.0021 secs
DNS-lookup: 0.0009 secs, 0.0008 secs, 0.0010 secs
Status code distribution:
[200] 146533 responses
/ping
- 10 Concurrent Requests - 1 Channel from Router to Extension🔴 Extension CPU is ~90%
oha -c 10 -z 10s http://localhost:5001/ping
Summary:
Success rate: 100.00%
Total: 10.0008 secs
Slowest: 0.0257 secs
Fastest: 0.0017 secs
Average: 0.0028 secs
Requests/sec: 3603.8977
Total data: 140.79 KiB
Size/request: 4 B
Size/sec: 14.08 KiB
Response time histogram:
0.002 [1] |
0.004 [34294] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.006 [1463] |■
0.009 [152] |
0.011 [66] |
0.014 [53] |
0.016 [3] |
0.018 [0] |
0.021 [2] |
0.023 [2] |
0.026 [6] |
Response time distribution:
10.00% in 0.0022 secs
25.00% in 0.0023 secs
50.00% in 0.0025 secs
75.00% in 0.0030 secs
90.00% in 0.0035 secs
95.00% in 0.0040 secs
99.00% in 0.0062 secs
99.90% in 0.0121 secs
99.99% in 0.0251 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0014 secs, 0.0011 secs, 0.0018 secs
DNS-lookup: 0.0005 secs, 0.0003 secs, 0.0007 secs
Status code distribution:
[200] 36042 responses
/echo-slow
- 10 Concurrent Requests - 1 Channel from Router to Extensionoha -c 10 -z 10s -m POST -T application/octet-stream -D ./oha http://localhost:5001/echo-slow
Summary:
Success rate: 100.00%
Total: 10.0009 secs
Slowest: 0.2978 secs
Fastest: 0.1180 secs
Average: 0.1925 secs
Requests/sec: 51.3954
Total data: 4.68 GiB
Size/request: 9.32 MiB
Size/sec: 478.97 MiB
Response time histogram:
0.118 [1] |
0.136 [0] |
0.154 [1] |
0.172 [1] |
0.190 [250] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.208 [234] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.226 [16] |■■
0.244 [8] |■
0.262 [1] |
0.280 [1] |
0.298 [1] |
Response time distribution:
10.00% in 0.1875 secs
25.00% in 0.1886 secs
50.00% in 0.1900 secs
75.00% in 0.1934 secs
90.00% in 0.1981 secs
95.00% in 0.2112 secs
99.00% in 0.2340 secs
99.90% in 0.2978 secs
99.99% in 0.2978 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0017 secs, 0.0012 secs, 0.0023 secs
DNS-lookup: 0.0005 secs, 0.0003 secs, 0.0006 secs
Status code distribution:
[200] 514 responses
/echo-slow
- 1 Concurrent Request - 1 Channel from Router to Extensionoha -c 1 -z 10s -m POST -T application/octet-stream -D ./oha http://localhost:5001/echo-slow
Summary:
Success rate: 100.00%
Total: 10.0006 secs
Slowest: 0.0773 secs
Fastest: 0.0175 secs
Average: 0.0193 secs
Requests/sec: 51.6968
Total data: 4.71 GiB
Size/request: 9.32 MiB
Size/sec: 481.78 MiB
Response time histogram:
0.017 [1] |
0.023 [502] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.029 [8] |
0.035 [4] |
0.041 [1] |
0.047 [0] |
0.053 [0] |
0.059 [0] |
0.065 [0] |
0.071 [0] |
0.077 [1] |
Response time distribution:
10.00% in 0.0182 secs
25.00% in 0.0185 secs
50.00% in 0.0188 secs
75.00% in 0.0193 secs
90.00% in 0.0199 secs
95.00% in 0.0210 secs
99.00% in 0.0299 secs
99.90% in 0.0773 secs
99.99% in 0.0773 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0005 secs, 0.0005 secs, 0.0005 secs
DNS-lookup: 0.0001 secs, 0.0001 secs, 0.0001 secs
Status code distribution:
[200] 517 responses
LAMBDA_DISPATCH_AllowInsecureControlChannel=true
LAMBDA_DISPATCH_PreferredControlChannelScheme=http
=> LambdaId: 3170030e-384d-46ea-8353-8199743d7e04 => TaskNumber: 8 => ChannelId: 09e65c5b-c262-469d-a791-430e8cbfc552
--
Exception caught in task
System.Net.Http.HttpRequestException: Connection refused (localhost:3000)
---> System.Net.Sockets.SocketException (111): Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError, CancellationToken) + 0x4c
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16) + 0x60
at System.Net.Sockets.Socket.<<ConnectAsync>g__WaitForConnectWithCancellation\|285_0>d.MoveNext() + 0x1bc
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpConnectionPool.<ConnectToTcpHostAsync>d__104.MoveNext() + 0x714
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.<ConnectToTcpHostAsync>d__104.MoveNext() + 0x79c
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpConnectionPool.<ConnectAsync>d__103.MoveNext() + 0x1dc
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpConnectionPool.<CreateHttp11ConnectionAsync>d__105.MoveNext() + 0x1b8
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpConnectionPool.<AddHttp11ConnectionAsync>d__79.MoveNext() + 0x350
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.<WaitWithCancellationAsync>d__1.MoveNext() + 0x178
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__89.MoveNext() + 0xac0
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext() + 0x184
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at System.Net.Http.HttpClient.<<SendAsync>g__Core\|83_0>d.MoveNext() + 0x3d8
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x24
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0x100
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x68
at PwrDrvr.LambdaDispatch.LambdaLB.Function.<>c__DisplayClass9_1.<<FunctionHandler>b__1>d.MoveNext() + 0x740
01:01:25.580 warn: PwrDrvr.LambdaDispatch.LambdaLB.Function[0]
=> LambdaId: 3170030e-384d-46ea-8353-8199743d7e04 => TaskNumber: 8 => ChannelId: 09e65c5b-c262-469d-a791-430e8cbfc552
Connection refused
Starting task
--
/app/dist/app.cjs:476
throw new TypeError("invalid media type");
^
TypeError: invalid media type
at Object.parse2 [as parse] (/app/dist/app.cjs:476:15)
at setCharset (/app/dist/app.cjs:20086:32)
at ServerResponse.send2 [as send] (/app/dist/app.cjs:21472:36)
at ServerResponse.json (/app/dist/app.cjs:21536:19)
at ServerResponse.send2 [as send] (/app/dist/app.cjs:21464:25)
at /app/dist/app.cjs:71216:9
at Layer.handle [as handle_request] (/app/dist/app.cjs:17246:9)
at next (/app/dist/app.cjs:17415:17)
at rawParser (/app/dist/app.cjs:14424:11)
at Layer.handle [as handle_request] (/app/dist/app.cjs:17246:9)
Node.js v18.18.2
01:01:25.580 fail: PwrDrvr.LambdaDispatch.LambdaLB.Function[0]
.dockerignore
) (100+ MB)libicu
files (about 40 MB)LAMBDA_DISPATCH_FunctionName
env var is set, setup a default dispatch groupVERB /path HTTP/1.1
http://localhost:3001/
as the path//
chars and redirects to remove it, resulting in a URL turned into a broken pathProtocolError
exception said nothing about receiving DATA before HEADERSplatform: linux/amd64
# docker-compose -f docker-compose-public.yml up --build
# docker-compose -f docker-compose-public.yml cp lambdalb:/app/bootstrap bootstrap
[+] Copying 1/0
✔ lambda-dispatch-lambdalb-1 copy lambda-dispatch-lambdalb-1:/app/bootstrap to bootstrap Copied
0.1s
# file bootstrap
bootstrap: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=18a5f37f3196919e19489fcbc5cfdf77029258c2, for GNU/Linux 3.2.0, stripped
platform: linux/arm64
# docker-compose -f docker-compose-public.yml up --build
# docker-compose -f docker-compose-public.yml cp lambdalb:/app/bootstrap bootstrap
[+] Copying 1/0
✔ lambda-dispatch-lambdalb-1 copy lambda-dispatch-lambdalb-1:/app/bootstrap to bootstrap Copied
0.1s
# file bootstrap
bootstrap: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=0aeb353ae33f2fd42d1ae87fd6267e1ac3f8de73, for GNU/Linux 3.7.0, stripped
HttpRequestException
(which ProtocolError
might actually be a part of already)FunctionName = "lambda-dispatch-lambdalb"
System.Private.CoreLib!System.Threading.Tasks.Task.SpinThenBlockingWait(int32,value class System.Threading.CancellationToken)
dotnet/runtime#72153 (comment)
Set DOTNET_ThreadPool_UnfairSemaphoreSpinLimit=0
to a value such as 5 or 10, much lower than the default of ~50. This cuts the CPU usage from 400-500% down to 100%.
hey -c 20 -n 100000 http://127.0.0.1:5001/fact
Summary:
Total: 8.9211 secs
Slowest: 0.0271 secs
Fastest: 0.0004 secs
Average: 0.0018 secs
Requests/sec: 11209.4022
Response time histogram:
0.000 [1] |
0.003 [96749] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.006 [2242] |■
0.008 [237] |
0.011 [304] |
0.014 [288] |
0.016 [111] |
0.019 [20] |
0.022 [6] |
0.024 [2] |
0.027 [40] |
Latency distribution:
10% in 0.0011 secs
25% in 0.0013 secs
50% in 0.0016 secs
75% in 0.0020 secs
90% in 0.0024 secs
95% in 0.0028 secs
99% in 0.0058 secs
hey -c 20 -n 10000 http://127.0.0.1:5001/fact
Summary:
Total: 6.2497 secs
Slowest: 1.0220 secs
Fastest: 0.0004 secs
Average: 0.0125 secs
Requests/sec: 1600.0795
Response time histogram:
0.000 [1] |
0.103 [9879] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.205 [0] |
0.307 [0] |
0.409 [0] |
0.511 [20] |
0.613 [0] |
0.716 [1] |
0.818 [39] |
0.920 [0] |
1.022 [60] |
content-length
response header was a number instead of a stringcontent-length
headerCodeGenieApp/serverless-express#563
Had to switch from serverless-express
to serverless-adapter
to get the fix.
Cannot get required symbol EVP_rc2_cbc from libssl
Debugger.Break
just is not supported on Mac/Linux#if false
// Insert a call to raise(SIGTRAP) here
var sw = new System.Diagnostics.Stopwatch();
// Debugger.IsAttached returns immediately on Mac OS X
while (!Debugger.IsAttached)
{
sw.Restart();
System.Console.WriteLine("Waiting for debugger to attach...");
while (sw.ElapsedMilliseconds < 1000); // Spin for 1 second
}
#else
System.Console.WriteLine("Press any key once the debugger is attached...");
System.Console.ReadKey();
System.Console.WriteLine("Trying Debugger.Break()...");
System.Diagnostics.Debugger.Break();
System.Console.WriteLine("Break here if the debugger didn't break...");
#endif
// coreclr debugger cannot catch SIGTRAP (5) on Mac OS X
// DebugHelper.raise(5);
200
409
Discarding connection for X-Lambda-Id
04:58:44.217 fail: PwrDrvr.LambdaDispatch.LambdaLB.HttpReverseRequester[0]
=> LambdaId: e57820b8-cb54-43f5-be21-8a126db443d9 => TaskNumber: 1 => ChannelId: fd6ffbf2-a04b-4dce-88ca-a2ec7f5b378f
Error reading request from response
System.Exception: Invalid request line: Discarding connection for X-Lambda-Id: e57820b8-cb54-43f5-be21-8a126db443d9, X-Channel-Id: fd6ffbf2-a04b-4dce-88ca-a2ec7f5b378f, closing
at PwrDrvr.LambdaDispatch.LambdaLB.HttpReverseRequester.<GetRequest>d__8.MoveNext() + 0xb94
04:58:44.217 info: PwrDrvr.LambdaDispatch.LambdaLB.Function[0]
=> LambdaId: e57820b8-cb54-43f5-be21-8a126db443d9 => TaskNumber: 1
Exiting task
oha
stops with a time boundary it aborts in-progress requestsrouter
catches this and aborts the request/response to the extensionextension
(rust) is throwing a few errors in this caseThe below will cause the open sockets count to go up and up.
Lambda has a 1024 socket descriptor limit so that is probably being exceeded at some point.
# Count sockets for the node.js app
watch "netstat -a -n -p tcp | grep 3001 | wc -l"
# Repeatedly Allow `oha` to Terminate Sockets
while true; do oha --no-tui -m POST -c 10 -z 1s -D ./oha -T "application/octet-stream" http://localhost:5001/echo ; done
/
root path should be passable to an app/
passes through to appsMicrosoft.AspNetCore.Http.DefaultHttpResponse.StartAsync(CancellationToken cancellationToken)
at PwrDrvr.LambdaDispatch.Router.Dispatcher.AddConnectionForLambda(HttpRequest request, HttpResponse response, String lambdaId, String channelId)
at PwrDrvr.LambdaDispatch.Router.ChunkedController.Post(String instanceId) in /app/src/PwrDrvr.LambdaDispatch.Router/ChunkedController.cs:line 126
lambda-dispatch-router-1 | 16:41:51.543 fail: Microsoft.AspNetCore.Server.Kestrel[13] Connection id "0HMVUAN471FMI", Request id "0HMVUAN471FMI:000509E5": An unhandled exception was thrown by the application. System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<FireOnStarting>g__ProcessEvents|241_0(HttpProtocol protocol, Stack`1 events)
lambda-dispatch-router-1 | 16:41:51.569 fail: PwrDrvr.LambdaDispatch.Router.ChunkedController[0] Router.ChunkedController.Post - Exception System.ObjectDisposedException: The response has been aborted due to an unhandled application exception. ---> System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<FireOnStarting>g__ProcessEvents|241_0(HttpProtocol protocol, Stack`1 events) --- End of inner exception stack trace --- at Microsoft.AspNetCore.Http.DefaultHttpResponse.StartAsync(CancellationToken cancellationToken) at PwrDrvr.LambdaDispatch.Router.Dispatcher.AddConnectionForLambda(HttpRequest request, HttpResponse response, String lambdaId, String channelId) at PwrDrvr.LambdaDispatch.Router.ChunkedController.Post(String instanceId) in /app/src/PwrDrvr.LambdaDispatch.Router/ChunkedController.cs:line 126
lambda-dispatch-router-1 | 16:41:51.574 fail: Microsoft.AspNetCore.Server.Kestrel[13] Connection id "0HMVUAN471FMI", Request id "0HMVUAN471FMI:000509E5": An unhandled exception was thrown by the application. System.ObjectDisposedException: The response has been aborted due to an unhandled application exception. ---> System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<FireOnStarting>g__ProcessEvents|241_0(HttpProtocol protocol, Stack`1 events) --- End of inner exception stack trace --- at Microsoft.AspNetCore.Http.DefaultHttpResponse.StartAsync(CancellationToken cancellationToken) at PwrDrvr.LambdaDispatch.Router.Dispatcher.AddConnectionForLambda(HttpRequest request, HttpResponse response, String lambdaId, String channelId) at PwrDrvr.LambdaDispatch.Router.ChunkedController.Post(String instanceId) in /app/src/PwrDrvr.LambdaDispatch.Router/ChunkedController.cs:line 126 at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
oha
aborts the requests when using the -z 10s
parameterContent-Type
Content-Length
Last-Modified
handler
entrypoint for the node.js applambdadirect.ghpublic.pwrdrvr.com
that goes directly to the Lambda target groupCMD
in the Dockerfile or ENTRYPOINT
if it's an AWS Lambda base imagehey -c 10 -n 10000 http://127.0.0.1:5002/public/silly-test-image.jpg
Summary:
Total: 3.2683 secs
Slowest: 0.0671 secs
Fastest: 0.0007 secs
Average: 0.0033 secs
Requests/sec: 3059.6653
Total data: 1681610000 bytes
Size/request: 168161 bytes
Response time histogram:
0.001 [1] |
0.007 [9854] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.014 [131] |■
0.021 [4] |
0.027 [0] |
0.034 [0] |
0.041 [0] |
0.047 [0] |
0.054 [0] |
0.060 [0] |
0.067 [10] |
Latency distribution:
10% in 0.0020 secs
25% in 0.0024 secs
50% in 0.0029 secs
75% in 0.0038 secs
90% in 0.0047 secs
95% in 0.0055 secs
99% in 0.0080 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0000 secs, 0.0007 secs, 0.0671 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
req write: 0.0000 secs, 0.0000 secs, 0.0013 secs
resp wait: 0.0021 secs, 0.0004 secs, 0.0644 secs
resp read: 0.0011 secs, 0.0000 secs, 0.0080 secs
Status code distribution:
[200] 10000 responses
hey -c 10 -n 10000 http://127.0.0.1:5002/public/silly-test-image.jpg
Summary:
Total: 5.0350 secs
Slowest: 0.1223 secs
Fastest: 0.0009 secs
Average: 0.0050 secs
Requests/sec: 1986.0920
Total data: 1681610000 bytes
Size/request: 168161 bytes
Response time histogram:
0.001 [1] |
0.013 [9967] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
0.025 [22] |
0.037 [0] |
0.049 [0] |
0.062 [0] |
0.074 [0] |
0.086 [0] |
0.098 [0] |
0.110 [0] |
0.122 [10] |
Latency distribution:
10% in 0.0022 secs
25% in 0.0030 secs
50% in 0.0043 secs
75% in 0.0069 secs
90% in 0.0080 secs
95% in 0.0087 secs
99% in 0.0109 secs
Details (average, fastest, slowest):
DNS+dialup: 0.0000 secs, 0.0009 secs, 0.1223 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
req write: 0.0000 secs, 0.0000 secs, 0.0007 secs
resp wait: 0.0037 secs, 0.0006 secs, 0.1183 secs
resp read: 0.0013 secs, 0.0000 secs, 0.0091 secs
Status code distribution:
[200] 10000 responses
lambda-dispatch-router-1 | 03:11:20.266 info: PwrDrvr.LambdaDispatch.Router.LambdaInstance[0]
lambda-dispatch-router-1 | Starting Lambda Instance de5b44d3-19f3-4795-98cb-112d4bbf5905
lambda-dispatch-router-1 | 03:11:21.131 info: PwrDrvr.LambdaDispatch.Router.LoggerMetricsReporter[0]
lambda-dispatch-router-1 | Metrics:
lambda-dispatch-router-1 | LambdaInstanceStartingCount: 1 items
lambda-dispatch-router-1 | report_success: 2 items
lambda-dispatch-router-1 | LambdaInstanceCount: 1 items
lambda-dispatch-router-1 | QueuedRequests: 1 req
lambda-dispatch-router-1 | RequestCount: 1 req
lambda-dispatch-router-1 | IncomingRequestTimer: 0 count 0 last 0 mean 0 min 0 max ms
lambda-dispatch-router-1 | 03:11:25.421 info: PwrDrvr.LambdaDispatch.Router.LambdaInstance[0]
lambda-dispatch-router-1 | Starting Lambda Instance 5d540d52-a9bc-448d-9ff4-51409f40ecc0
https
/ http2
listening port for incoming requests from the ALBapp_parts.status.canonical_reason().unwrap()
519
)thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/run.rs:344:59
thread 'main' panicked at 'LambdaId: 484618c5-509b-446c-9b93-924c33f67688 - run - Error in futures::future::try_join_all', src/run.rs:464:17
ERROR Lambda runtime invoke{requestId="abcfa9d6-4f62-4952-a925-b44b4ec060d1" xrayTraceId="Root=1-65a8969d-4779d0686ed209ef72fd30f8;Parent=7360266b34a4c915;Sampled=0;Lineage=5847b82f:0"}: Any { .. }
oha -c 1
and LAMBDA_DISPATCH_MaxConcurrentCount=1
for the router causes 1-2 second pauses every couple of secondsA declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.