Downloading files with urllib results in missing KB about emsdk HOT 3 OPEN

What platform are you seeing these partial downloads occurring on? Presumably its not macOS where we download using the external curl tool? Is it linux or windows?

Sadly the requests library is not part of the core python library so not really possible for us to use it in this script (which is supposed to be highly portable, and kind of boostraps the whole system).

I'm curious how you think the urllib implementation could be failing? I bug in urllib itself seems highly unlikely. And the fact that nobody else has reported this issue in over a decade also seems odd. I think its worth investigating furthur.

from emsdk.

I am using Windows. Due to well-known reasons, Chinese developers often need to use proxies to accelerate access. It's not that urllib has a 'bug', but it seems less capable of handling less stable network environments. That's why I switched to the requests library, which gave me more consistent results.

As for why others haven't reported this issue, there could be various reasons. Many might simply choose to download files manually rather than addressing the root cause of the problem. Also, even when using a proxy, it doesn't necessarily mean the network is unstable - at least my friends can execute the script normally. I hope this additional context helps explain the situation.

I completely understand the importance of maintaining high portability for the script. Your point about requests not being part of the core Python library is valid. For the sake of easy testing, I simply replaced the original implementation. However, if we were to formally integrate this into the project, we should definitely consider this point. For example, we could try the following approach:

try:
    import requests
    USE_REQUESTS = True
except ImportError:
    USE_REQUESTS = False

This way, we maintain portability while offering a more robust solution for those who need it.

from emsdk.

I would like to understand exactly what the problem is with the urllib implementation before making any change here. Is the problem consistent for you? Perhaps we can start by adding some assertions here? Presumably we should never be returning success if not all the bytes were transferred?

from emsdk.