Issue with pcntl_fork() about mongo-php-driver HOT 12 CLOSED

I am using php 7.0.12 and the ext version is 1.2.8, still found this issue, mongodb is 3.4, this happened in php-fpm

from mongo-php-driver.

Do you have more details on your runtime environment (phpinfo() would suffice)?

Also, do you get the same behavior if you initialize the first Manager by doing a query/command/write before forking? A selectServer() call would also trigger initialization (anything that attempts to discover the topology and/or communicate with a server).

from mongo-php-driver.

Apologies, I'll write a fully working demonstration. The issue just seems so prevalent I thought it would be easy to diagnose. It currently happens 100% of the time but is wrapped in a large framework and a big forking TCP server. I'll pull out the relevent code and get back to you guys

from mongo-php-driver.

Inside the attached zip is...

test.php: A working example of the bug, the script creates 8 threads and a new manager in each then tries to perform a find() query on them. Running it with 2-3 threads doesn't seem to create errors, but as the number increases the errors become more common.

phpinfo.html: This is the phpinfo() from my test system.

output.txt: These are the Fatal Errors I receive when running test.php. Example error:

Fatal error: Uncaught exception 'MongoDB\Driver\Exception\RuntimeException' with message 'Invalid response_to. Expected 2, got 1.' in /home/zorg/web/sites/localdev/townwars/thirdparty/MongoDB/Operation/Find.php:165

test.zip

from mongo-php-driver.

If you need anything to help get this issue resolved I'll gladly provide it. It's currently the only issue preventing me from moving from the old MongoClient() code.

from mongo-php-driver.

Sorry for losing track of this, but thanks for the reproduction script. I see from your example script that you are forking before any network IO (which was general advice I received when I asked the libmongoc devs above this), so there's clearly a problem with the driver not segregating sockets from each process. I've opened PHPC-625 to track this.

from mongo-php-driver.

@ZorgUK: I did have one observation from your test script:

for ($i = 0; $i < $threads; $i++) {
    $pid = pcntl_fork();
    if ($pid == 0) {
        // Child process, create a new manager here
        unset($manager1);
        $manager2    = new MongoDB\Driver\Manager('mongodb://localhost:27017');
        $collection2 = new MongoDB\Collection($manager2, "test.test");
        $cursor      = $collection2->find([], []);
    } else {
        $cursor = $collection1->find([], []);
        sleep(1);
    }
}

Did you purposefully mean to fork from forked children? Despite $threads being initialized to 8, each loop iteration starts with a call to pcntl_fork(), which means that the first iteration will fork one child and then execute a query in both the parent and that child. Those two processes then move to the second iteration, which will again fork two more children (we now have four running), all of which do another query. With this logic, we end up with many more than 8 concurrent processes.

This isn't to distract from the driver bug, but it was a curiosity I noticed while testing your script. I modified it a bit in the referenced JIRA ticket, but preserved the behavior above for the time being.

from mongo-php-driver.

That was unintentional, sorry. It's a very simplified script, the actual script that has this problem is a massive project with a multi-threaded TCP server. The child thread should exit() out after performing the conflicting query.

from mongo-php-driver.

I've found a workaround:
$manager2 = new MongoDB\Driver\Manager('mongodb://localhost:27017/?socketTimeoutMS=30001');

So, each time you use connect-after-fork, you should increment timeout.

upd: non-existing options are ok, so
new MongoDB\Driver\Manager("mongodb://localhost:27017/?x=".posix_getpid())

from mongo-php-driver.

@shepik: FYI, your work-around is due to the driver including the URI string (i.e. MongoDB\Driver\Manager's first argument) in its hash for persistent sockets. By altering the URI string each time, each forked process is guaranteed to create and use its own socket. The relevant code for that is here if you're curious.

The decision to use the URI string here was rather arbitrary, and it notably ignores any array options (i.e. MongoDB\Driver\Manager's second argument), which may supersede parameters found in the URI string.

One of the upcoming tickets for 1.2.0 is PHPC-605, which entails changing our internal stream wrappers. That would change the above logic for persisting sockets; however, it may also resolve this issue entirely (as mentioned in (PHPC-625, the corresponding JIRA ticket for this issue).

from mongo-php-driver.

1.2.0alpha2 has been released, which includes significant changes to our connection handling and persistence. The aforementioned PHPC-605 is included in this release.

PHPC-625 has been updated to ensure that we document incompatibilities with forking and also include the PID when persisting libmongoc client objects in memory. The latter bit is similar to @shepik's suggestion above, which works for 1.1.x where the URI was included in the hash for a persisted socket. Please follow PHPC-625 for updates.

from mongo-php-driver.

@tadimy: If you have reproducible errors on the most recent stable release of the driver (1.5.x), please open a new issue so we can investigate and track that going forward. I'm not sure why you would be calling pcntl_fork() from a web SAPI, but note that the driver still does not support sharing its resources across forked PHP processes. The fixes I referred to in #174 (comment) only pertain to including the PID when persisting libmongoc client objects (and their sockets), which ensure that creating a new Manager in a child process with the same URI will not attempt to re-use resources from the parent process.

from mongo-php-driver.