GnuTlsCertificateSetX509SystemTrust causes GNUTLS_E_FILE_ERROR (-64) after sequentially loading provided DLLs on a Windows Azure Function about fluentftp.gnutls HOT 11 CLOSED

@FanDjango Correct me if I'm wrong, but I guess the issue comes from a place that tries to add a certificate to the trusted store of the OS?
Can it be maybe that an Azure Function does not have permission to do so? In the source code of gnutls, I found the following in the add_system_trust method, which seems to be indirectly used by gnutls_certificate_set_x509_system_trust.

if (i == 0)
    store = CertOpenStore(CERT_STORE_PROV_SYSTEM, 0, 0, CERT_SYSTEM_STORE_CURRENT_USER , L"ROOT");
else
    store = CertOpenStore(CERT_STORE_PROV_SYSTEM, 0, 0, CERT_SYSTEM_STORE_CURRENT_USER, L"CA");

if (store == NULL)
    return GNUTLS_E_FILE_ERROR;

from fluentftp.gnutls.

I'll make a release that skips this error - it will allow you continue. Either you then trust "any certificate" or it will fail.

What's failing is loading the certs from the store of trusted certs in the windows place where they are stored, so could be a mapping problem again.

from fluentftp.gnutls.

To really fix the failing "set_x509_system_trust" a lot more research must be done to find out wether that special azure environment you are using also remaps the location of this(these) file in that special directory.

Under Windows, it's not a file - it turns out that this function will enteran Win32 API function call "CertOpenStore". Which then fails.

Anyway, I have modified the code further to implement the following:

Maybe I will modify this later to ignore all failures and just produce an error message

from fluentftp.gnutls.

For this one, I propose to do the following:

1. I will find out from the GnuTLS source, what GnuTlsCertificateSetX509SystemTrust(...) tries to do internally, to gain an understanding of what is going wrong here.

2. I need to know more about the context of this error - is it recurrring, is it recoverable, does it not happen with no parallelisation. Especially that last one. Once again maybe you would profit from locking - and then I would begin the search for possible culprits in the parallel execution of the GnuTlsInternalStream.

I assume this is once again under your Azure deployment/environment as you have described and have had so much trouble with in the previous issues.

I suppose turning on some logging with the apropriate verbosity would be helpful - let's see what else is going on around the time the error occurs. As before, I am sure your help and tenacity will aid in isolationg this problem.

from fluentftp.gnutls.

where it would normally be executed beforehand and thus be nothing to worry about?

Not sure that I understand this, but certainly the first usage of GnuTLS should not be significantly overlapped by the next usage if the load process is still underway in the first incarnation. Perhaps you could try to do a first initial "connect-disconnect" with no transfer on app start just to get everything initialised. If that helps, I can better understand the problem, mayhaps.

from fluentftp.gnutls.

@FanDjango In my local "unit test" I used earlier when investigating the concurrency issue, I cannot reproduce this issue. So I don't assume it is related to concurrency in this case unfortunately.

My test basically does the following:

• use an "external" directory as prefix, away from the app binaries
• delete any lib*.dll in the directory of the app binaries to fully reproduce the situation of the Azure Function on the DLL level
• use Task.WhenAll to do 5 concurrent create-connect-list-disconnect cycles

As an additional test to also validate it wouldn't behave differently on a concurrently executed Azure Function (which I doubt), I'll temporarily adapt my deployed test code to construct a client and do a connect-disconnect as you suggest in a lock before allowing any concurrent logic to be executed.

In the mean time I'm also updating to the latest version as I was still doing the "bulk load" myself on the older version, maybe there's a slight difference there after all

from fluentftp.gnutls.

Ok, released as 1.0.18

That code place was actually meant to just produce an error message on "not successful". But I forgot to add in a list of failures to ignore (because I didn't know what failures to put in the list) - thus it terminates totally. Now we know one failure we would like to ignore, at least :-) Maybe I will modify this later to ignore all failures and just produce an error message. Added to the TODO/INVESTIGATE list.

Try "validate any certificate" setting to true. Not meant as a permanent solution. Just want to see if there are any other, more suprises after that one, so that we can make a list, maybe, of what else turns up.

To really fix the failing "set_x509_system_trust" a lot more research must be done to find out wether that special azure environment you are using also remaps the location of this(these) file in that special directory.

from fluentftp.gnutls.

Oh, and you will be wanting to look at logs now, to see what is happening...

Like so:

Command:  AUTH TLS
Status:   Waiting for response to: AUTH TLS
Response: 234 AUTH TLS successful [<1ms]
Status:   GnuTLS: 0   FluentFTP.GnuTLS 1.0.18.0(Win32NT/.NET 6.0) / GnuTLS 3.7.8
Status:   GnuTLS: 1   Interop : *GnuTlsGlobalSetLogFunction(...)
Status:   GnuTLS: 1   Interop : *GnuTlsGlobalSetLogLevel(...)
Status:   GnuTLS: 1   Interop : *GnuTlsGlobalInit(...)
Status:   GnuTLS: 1   Interop : *.ctor(...):CertificateCredentials
Status:   GnuTLS: 1   Interop : *GnuTlsCertificateSetX509SystemTrust(...)

Settings:

                LogLevel = 2,
                LogMessages = GnuMessage.InteropFunction,

from fluentftp.gnutls.

Hi @FanDjango

I can confirm that everything now works as expected for this case (I already had validate any certificate enabled for communication with this development server -- didn't yet try without it on a server that has a proper certificate set up).

I applied the logging settings you shared but I don't seem to see them. I only see other logs from FluentFTP itself. (FYI, aside from the settings you shared I'm using the IFtpLogger interface to capture the logs)

See them attached (note that some duplication exists due to concurrency)
query_data.csv

EDIT: I just noticed my log level in the app isn't set correctly, small moment
EDIT2: regardless of putting the log level in my app to Debug, I don't see the GnuTLS logs. I do see Debug logs from FluentFTP itself though

from fluentftp.gnutls.

GnuTLS log stuff all comes in to FluentFTP as "verbose" log entries

from fluentftp.gnutls.

Yes, but Verbose is mapped to Debug in my log adapter..

from fluentftp.gnutls.