Comments (7)
What are these reports measuring? In the "Web transactions time" graph you shared, what is "2.64k" referring to for the minute starting at 9:08am? The X axis is clearly time but I don't understand what the Y axis is measuring.
The y axis is ms. So, it's showing transaction time over time. The beginning of the graph in the screenshot (8:50 - ~9:15ish) is with 1.16.2 and the drop is after downgrading back to 1.14.2.
- Does "manually install libbson and libmongoc" refer to specifying --with-mongodb-system-libs=yes when configuring the build, as described in the manual installation docs?
- Can you elaborate on "needed" there? What problems did you encounter attempting to build the driver using the bundled submodules for libmongoc and libmongocrypt?
So we use a build script to pull down packages from a variety of locations, unpack them and compile PHP with various flags depending on what version of PHP we're compiling, etc. The script was already running a git clone
of the specified version (based on a config file that includes a number of other extensions) and then running a git submodule update --init
.
The "manually install" part came in when the PHP build would fail (even when --with-mongodb-system-libs=yes
was passed) because libbson and libmongoc weren't found -- so we had to add a yum install -y mongo-c-driver-devel libbson-devel
for our Centos boxes and actually build libmongoc from source for Debian (because the debian version of libmongoc in apt wasn't compatible with 1.16). For both we then ran pkg-config --cflags
and pkg-config --libs
for libbson-1.0
and libmongoc-1.0
before building (with --with-mongodb-system-libs=yes
).
I could go into the whys and wherefores of the above changes that were necessary to add 1.16.x, but the tl:dr; for each is "it didn't build PHP". Does that help? Unfortunately, the repository that has this build script is private, but I could probably pull stuff out into gists if necessary.
The New Relic graph was taken from a customer's active live site. Right now, we don't have a test environment that we can run the same kind of tests on (and non-live environments didn't have enough data to really see the before/after effects), but that is something we can (and want to) set up, using load testing tools to throw traffic at it for brief periods of time so we can analyze the results.
from mongo-php-driver.
I am relatively certain that the customer is not using MongoDB in any way and it was just the driver update the caused the performance overhead. And I'm reasonably confident that that's the case because actual MongoDB usage amongst our customers is rare and we saw this on multiple sites (I can't confirm whether it was all sites, so there may be some common thread among those that we knew about, or it could have actually made PHP slower for everyone and we only heard from the ones that complained the loudest).
I agree that we need to have an environment to throw tests at, so that's in our court, but I wanted to create the issue while it was relatively fresh in mind.
from mongo-php-driver.
I wanted to create the issue while it was relatively fresh in mind.
Happy to leave this issue open until more info comes to light. Two additional thoughts below.
Unfortunately, the repository that has this build script is private, but I could probably pull stuff out into gists if necessary.
Sharing as much of the build process as possible would be helpful. Ideally, a script that shows all of the steps for how you install libmongoc (let's focus on building manually from a release tarball) and then compiling a particular PHP version with mongodb
as a static extension.
Also, version 1.14.2 of the mongodb
extension requires libmongoc 1.22.2+, but any never version of libmongoc will work. To try and isolate the issue between the PHP driver and libmongoc, I'd suggest leaving the extension at 1.14.2 but building against a newer version of libmongoc. We can start with:
- libmongoc 1.24.3, which is required by PHP driver 1.16.2
- libmongoc 1.23.4, which is required by PHP driver 1.15.3
The goal here would be to start with a version that easily reproduces the delay and then jump backwards until we can pinpoint a libmongoc version that introduces the issue (akin to a git bisect to find a bug). This would definitely require a test environment, so something to revisit once that's in place.
In the meantime, I'll peruse libmongoc's fixVersions in JIRA from 1.22 onwards to see if any tickets come up that might relevant for changes to mongoc_init()
.
from mongo-php-driver.
I took a screenshot so that we keep track of the phpinfo (same for the 3 PHP versions)
from mongo-php-driver.
Thanks @GromNaN I only had that thought after I created the ticket. 😅
from mongo-php-driver.
received reports that PHP was taking much longer to resolve according to New Relic data. Rolling back to 1.15.x did not resolve, but the difference can be seen after we reverted back to 1.14.2:
What are these reports measuring? In the "Web transactions time" graph you shared, what is "2.64k" referring to for the minute starting at 9:08am? The X axis is clearly time but I don't understand what the Y axis is measuring.
I will say that after the update, I needed to manually install libbson and libmongoc, so my suspicion is that those updates are related.
A few questions on this:
- Is "after the update" referring to installing version 1.16.x of the
mongodb
extension? - Does "manually install libbson and libmongoc" refer to specifying
--with-mongodb-system-libs=yes
when configuring the build, as described in the manual installation docs? - Can you elaborate on "needed" there? What problems did you encounter attempting to build the driver using the bundled submodules for libmongoc and libmongocrypt?
To walk through exactly what the mongodb
extension does when loaded (assuming the application doesn't interact with it further), we can look at the various init hooks in php_phongo.c
:
- RINIT initializes several internal HashTables
- GINIT initializes another HashTable, increments an internal counter, and resets some memory
- MINIT does a bit more, but the most notable thing is invoking
mongoc_init
mongoc_init
has had few changes in recent versions, but the most notable I can think of is AWS credential caching in mongodb/mongo-c-driver@8eaac64; however, the PHP driver isn't yet using that functionality so it should not even be compiled per the #if defined(MONGOC_ENABLE_MONGODB_AWS_AUTH)
check.
Is there any way for you to get more detail tracing from New Relic that might pinpoint an increase in time spent in various PHP extensions' init hooks? That would help narrow this down further and provide some assurance we're actually looking in the right place.
from mongo-php-driver.
The New Relic graph was taken from a customer's active live site.
We don't actually run MongoDB but provide the driver for users who wish to make MongoDB connections.
Is the customer actually interacting with the MongoDB driver in some way? My original read of the thread was that simply upgrading the mongodb
extension in your PHP image introduced overhead in an application that was not using the driver API, which is why I focused on the init hooks, as those are the only parts of the extension that I expect would be invoked automatically.
That said (and apologies for not clarifying this in my last comment), MINIT is only run once per PHP worker process. So even if mongoc_init
is responsible for some performance degradation, I would only expect that to impact the first request served by each worker. I see no reason to suspect the RINIT and GINIT functions, which could be invoked more frequently (definitely RINIT; GINIT might only be as frequent as MINIT depending on the SAPI you're using).
If the customer is using MongoDB, I think we'd need to get them involved here to actually investigate further.
If they're not using MongoDB (and assuming we can trust them on that), then I think it's probably correct to focus on MINIT, but you'd need to be able to reproduce this in a test environment so we can start diving into possible culprits.
from mongo-php-driver.
Related Issues (20)
- Adding configure options for zstd, snappy, zlib? HOT 3
- l
- apache stop when I added extension=mongodb.so to php.ini HOT 3
- `selectServer` takes too long on official php base image (connections opened sequentially) HOT 13
- libbson Bug Report : variable ret value type error HOT 8
- No suitable servers found (`serverSelectionTryOnce` set) when rs.initiate with localhost HOT 5
- Extract mongodb-1.6.2.tgz error HOT 5
- Sporadic "Authentication failed" errors HOT 8
- Error while compiling on macOS HOT 7
- Major slowdown since version 1.17.0 HOT 8
- PHP Warning: PHP Startup: Unable to load dynamic library 'mongodb.so' HOT 2
- MongoDB reporting "AuthenticationAbandoned: Overridden by new authentication session" for a PHP client HOT 14
- Windows build after 1.13.0 HOT 4
- linux general protection fault , sorry for my poor english HOT 1
- PHP refusing to load MongoDB PHP driver on Windows (on one machine but not another) HOT 1
- Can't download desired extension HOT 2
- Unable to install pecl mongodb-1.15.1 in php 8.2 HOT 1
- Bug & Question: How to compile mongodb statically into PHP? HOT 9
- PHPStorm Stub for Manager class has outdated signature HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mongo-php-driver.