Async programming with blocking operations about amp HOT 17 CLOSED

A roughly equivalent implementation in Python3 of the same script could be found here:

https://gist.github.com/yoanisgil/85ce168fa7792c484a1f

and as expected it takes ~ 4 secs to complete. However, I do specify the maximum number of available works for computation tasks.

from amp.

Hi Yoanis,

your problem are the blocking operations. As the event loop is single threaded, you'll not gain any improvements when using blocking operations. Fortunately, there's a non-blocking waiting mechanism: yield new Pause($ms).

<?php

require __DIR__ . '/vendor/autoload.php';

use Amp\Pause;
use function Amp\run;
use function Amp\all;
use function Amp\stop;

function asyncMultiply($x, $y) {
    // Create a new promisor
    $deferred = new Amp\Deferred;
    // Resolve the async result one second from now
    Amp\immediately(function() use ($deferred, $x, $y) {
        $sleep = rand(2, 4);
        echo "Sleep $sleep seconds \n";
        yield new Pause(1000 * $sleep);
        $deferred->succeed($x * $y);
    });
    echo "new promise\n";
    return $deferred->promise();
}

run(function() {
    $promises = [];
    for ($i = 0; $i < 5; $i++) {
        $promises[] = asyncMultiply(6, 7);
    }

    $results = (yield all($promises));
    var_dump($results);
    stop();
});

from amp.

So the question is how to turn that blocking operation into a non blocking one. Or is that outside of the scope of the project? Say I have a function MyBlockingFunction which I'd like to run asynchronously and for which there is no equivalent operation within the amp framework. Is it still possible to do that? Like I notice you have several projects which are using the amphp framework to provide asynchronous implementations of DNS, MySQL,HTTP clients (which by default are blocking). Maybe I am missing something but I though that an event loop and promises should be pretty much what is required to run concurrent tasks.

from amp.

For your use case you need https://github.com/amphp/thread which does the async stuff based on the pthreads php-extension.

This will work for blocking operations but writing code non-blocking will be the superior approach if feasible in your context.

Like I notice you have several projects which are using the amphp framework to provide asynchronous implementations of DNS, MySQL,HTTP clients

those dont do blocking IO but non blocking. see e.g. http://php.net/stream_set_blocking

from amp.

That's the difference between asynchronous concurrency and parallel concurrency. There's amphp/thread to execute those blocking operations in worker threads, but it's not yet up to date and doesn't work with any other than the NativeReactor, so it won't work if you have uv or libevent enabled. amphp/amp provides asynchronous concurrency.

Joe Watkins has a good blog post about "sync vs. async vs. parallel": http://blog.krakjoe.ninja/2015/07/the-universe-is-not-aware.html

If you do blocking operations, the task scheduler can't run other things while it's waiting for completion, because you never return control to the scheduler.

from amp.

@rdlowrey We really need some good documentation with images and examples to explain what's going on in such a case, because a lot of people will have exactly this question.

from amp.

@kelunik @staabm Thanks for the information (and yes I noticed there was amp/thread right after I posted my last comment :(). For sure some images and examples will be of a lot of help. Let me know if I can be of any help.

from amp.

Sure, you can (almost) always help. Did you already see our docs repository?

Would be nice if you could provide some feedback e.g. regarding these points:

• How easy is it to get your first "Hello World" running?
• Could you move on from there? What do we need to provide after a "Hello World"?
• What's important that should be on the main page?
• etc.

from amp.

@kelunik I am trying to clone the docs repository and I am getting this:

git clone [email protected]:amphp/amphp.github.io amphp/docs
Cloning into 'amphp/docs'...
The authenticity of host 'github.com (192.30.252.130)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.252.130' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: The remote end hung up unexpectedly

I do not recall any "Hello World" application, I just started reading some code and documentation here and put together a use case scenario. I would be more than glad to contribute but the clone thing kinda blocks me from going further ahead.

from amp.

Looks like you do not use public keys to authenticate against GitHub. I'd recommend setting that up, there's an article in GitHub's help section. Otherwise you can still clone using HTTPS instead, probably the cloning method you're used to.

from amp.

@kelunik I took at look at the home page and I think it would be great if a somewhat more elaborated example is used (not elaborated enough to confuse users ;)). I also read that article about sync vs. async vs. parallel and it was really clarifying, so maybe a reference to amp/thread might help.

Honestly, I was just rushing to get a working example with amp/dns and I went straight ahead and set it up ;). It was just after a quick debate about async programming with one of my colleagues that I thought about this async sleep test and that's when things got a bit confusing. But it's all clear now ;).

On a somehow related subject I was wondering if amp/dns supports DNS resolution of A,TXT, MX, CNAME, etc records?Will I incur in a performance penalty if I launch Amp/run to perform up to 6 DNS queries within the context of an HTTP request?

Right now I am using https://netdns2.com/ since it's feature complete from a DNS point of view but it's 100% sync. I don't think that will scale too well for thousands or even hundred of simultaneous (needs to test though ;)), hence my question(s).

Bests,

Yoanis.

from amp.

We're really just at the beginning of setting up our docs, so any feedback is really valuable, thanks. Regarding DNS, I think it's currently not supported, but @bwoebi wanted to add support for it. Once it's in, you can fire of all 6 requests at once and wait for all 6 then, so they're basically parallel then, because most of the time you'll be waiting for the other party to respond.

from amp.

Currently amp/dns will only resolve A, AAAA and CNAME records. There are plans for a more robust feature set but it just takes time :)

from amp.

That's not that far from my current use case where I need A, CNAME, TXT and MX records. Let me know if there is anything I can do to help.

from amp.

@yoanisgil amphp/dns supports those other records now, anything else open in this issue or can it be closed?

from amp.

I think it can be closed. Can you send me a link so I can take a look at
what was done?

Thanks,

Yoanis

Le jeudi 10 septembre 2015, Niklas Keller [email protected] a
écrit :

@yoanisgil https://github.com/yoanisgil amphp/dns supports those other
records now, anything else open in this issue or can it be closed?

—
Reply to this email directly or view it on GitHub
#34 (comment).

from amp.

@yoanisgil https://github.com/amphp/dns/tree/master/lib ;-)

from amp.