Handshake exception in high concurrency about tls HOT 12 CLOSED

There are many potential causes of connection timeouts:

• Most networking equipment is flaky under high load. When load is high, they drop some packets. You can detect these lost packets by capturing packets from both machines. TCP is resilient against dropped packets when the OS uses appropriate TCP retransmit timings.
• Server process not utilizing all cores because of misconfiguration or container restrictions. Check per-core cpu utilization.
• Container network throttling. The container may be dropping the packets. To detect this, capture packets on the container's internal interface.
• Tokio worker threads stuck on blocking IO. Check per-thread cpu utilization.
• Server process doing something expensive in the accept loop or in connection handler before starting TLS negotiation. Logging can cause slowdowns.
• Host machine is custom built and its power supply is too small. This problem shows under heavy load. It can cause all kinds of strange machine errors. Check the kernel logs for strange errors.
• Host machine internal bus congestion. The internal bus serves GPUs, SSD, and NICs. Network throughput can go down when the NIC shares a PCI channel with another device and that device is used heavily. Check the NIC queue lengths.
• Cosmic rays corrupting TCP packets in RAM. This doesn't affect 1-minute tests.
• Bug in core application libraries

It's been 1.5 months since you reported this issue. What happened?

from tls.

@mleonhard Thank you for your reply.
For reasons of energy, I'm not following this issue, but I'm going to go through H2 and get around the situation.

from tls.

Can you give a reproduce example?

from tls.

The server times out when the code "acceptor.accept(io).await" is executed. The io is tokio tcpstream and the certificate is rsa two-way authentication.

from tls.

The client uses the method "connect " of tokio-rustls.

from tls.

the server handshake timout is 30 sec, the client handshake timout is 10 sec. The CPU usage of the client and server accounts for less than a quarter of the system CPU.

from tls.

Faulty packets

from tls.

I cannot reproduce the problem, can you provide your test code?

I don't think that concurrency will affect tls handshake. from your description, handshake fail is actually a timeout instead of TLS error?

from tls.

I'm sorry, I can't provide the current code. However, this error is not caused by handshake timeout. The timeout interval of the client is 10s and that of the server is 30s. If the handshake fails because of timeout, the client reports timeout first. However, the current problem is that the client considers that the handshake is successful, but the server considers that the handshake times out.

from tls.

According to the captured packets, the server (IP 58) may be waiting for the client (IP 53) to send a Change Cipher Sepc packet to the server.

from tls.

@silence-coding If you're not going to pursue the issue further, how about closing it? That would be kind to the project maintainers.

from tls.

@mleonhard All right.

from tls.