Hi. Can you please collect networking logs by following the instructions below?
https://github.com/microsoft/WSL/blob/master/CONTRIBUTING.md#collect-wsl-logs-for-networking-issues

from wsl.

Multiple log files found, using: https://github.com/user-attachments/files/15827620/WslNetworkingLogs-2024-06-13_21-10-41.zip
.wslconfig found
Detected appx version: 2.1.5.0
optional-components.txt not found

from wsl.

View similar issues

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it!

Open similar issues:

• WSL mirrored network Docker container cannot access Windows service (#10926), similarity score: 0.76
• Mirrored networking IPv6 localhost (and localhost addresses) and hostAddressLoopback (for assigned addresses) not working from host (WSL2) (#10803), similarity score: 0.75
• WSL network crash under constant load (#10817), similarity score: 0.75
• Mirrored networking fine at first but drops after only a few minutes (#11369), similarity score: 0.75

Closed similar issues:

• DNS works however pinging/other traffic does not (#7031), similarity score: 0.77

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

Multiple log files found, using: https://github.com/user-attachments/files/15751492/WslLogs-2024-06-09_12-20-00.2.zip
appxpackage.txt not found
optional-components.txt not found
Error while parsing the logs. See action page for details

from wsl.

Do you have any evidence of that @NyaMisty? I saw exactly the same behavior with tools that connected to the internet as I did with systemd enabled, after running that sysctl command. Maybe they aren't directly related, but exhibit the same symptoms.

TL;DR: You are right, and removing net.ipv4.ip_local_port_range statements from /etc/sysctl.conf /etc/sysctl.d/* and /usr/lib/sysctl.d/* will solve the issue.

I'm sorry I made a stupid false assertion in the above reply. I'm not taking your guess because all possible issue that's causing network packet get unexpectedly dropped will cause the above issue.

In additional to @withinboredom 's previous investigation, I changed systemd's startup target from multi-user.target all the way down to emergency.target (which loads none services), which that makes debug a lot easier. However, even using emergency.target is still killing the network in mirrored mode.

Then I opened two terminal, one running a strace-d systemd

sudo strace -T -tt -f /usr/bin/unshare -fp --propagation shared --mount-proc -- systemd 2>&1 | tee systemd.log

and another running a watchdog

while true; do curl -s 192.168.112.1:7890; date +"%T.%N"; done    

With the log I can guess things goes wrong during systemd-sysctl.service, so finally I pinpointed that the issue goes exactly from sysctl.conf.

It turns out that Microsoft is using some black magic to implement forwarding in WSL mirrored network. It seems that it will only forward connection with specific source port, while dropping other connection silently, so when we overrided the port range, the network requests fails immediately.

Removing all ip_local_port_range statements in sysctl will solve the issue.

from wsl.

It's also worth pointing out that cloud-config and snapd were disabled for those logs (to save anyone else any trouble-shooting). Enabling/disabling them doesn't seem to have any effect.

from wsl.

Here's with all of networking working correctly:
WslNetworkingLogs-2024-06-13_21-10-41.zip

and with systemd preventing networking from working:
WslNetworkingLogs-2024-06-13_21-13-10.zip

I performed the same steps as before.

from wsl.

With the new networkingMode=mirrored I had similar issues in wsl 2.1.5 and 2.2.4, hence I left it to nat. This works flawlessly.

VMware Photon OS uses systemd as well and it works in wsl by configuring a rootless user same as the logged-in windows user. See https://github.com/dcasota/photonos-scripts/wiki/Photon-OS-on-WSL2, step 4.

from wsl.

Bump. Anything?

from wsl.

It appears this might be related to #11450 as running

sudo sysctl -w net.ipv4.ip_local_port_range="1024 65535"

and then doing a ton of connections will cause this situation, even without systemd (I just had to run a load test from wsl and needed more than 1024 connections). Combined with the "Address already in use" bug mentioned (but not called out explicitly), eventually, there is port starvation, and no connections can be made.

So, basically, I suspect that systemd just causes port starvation, since enough ports are not allocated to wsl in mirrored mode.

from wsl.

It appears this might be related to #11450 as running

sudo sysctl -w net.ipv4.ip_local_port_range="1024 65535"

and then doing a ton of connections will cause this situation, even without systemd (I just had to run a load test from wsl and needed more than 1024 connections). Combined with the "Address already in use" bug mentioned (but not called out explicitly), eventually, there is port starvation, and no connections can be made.

So, basically, I suspect that systemd just causes port starvation, since enough ports are not allocated to wsl in mirrored mode.

I believe this has ZERO relation to this issue. The ephemeral port issue is a longstanding problem which is definitely not related with this problem.

from wsl.

Do you have any evidence of that @NyaMisty? I saw exactly the same behavior with tools that connected to the internet as I did with systemd enabled, after running that sysctl command. Maybe they aren't directly related, but exhibit the same symptoms.

from wsl.

I will give this a go asap. Thanks for looking into it; and kinda obvious source in retrospect.

from wsl.