Comments (17)
jrafanie
commented on July 17, 2024
joerafaniello@Joes-MBP-2 slow-request-web-service-pods % request-log-analyzer */production.log
Request-log-analyzer, by Willem van Bergen and Bart ten Brinke - version 1.13.4
Website: http://railsdoctors.com
production.log: 100% [===========================================================================================] Time: 00:00:03
production.log: 100% [===========================================================================================] Time: 00:00:03
production.log: 100% [===========================================================================================] Time: 00:00:03
production.log: 100% [===========================================================================================] Time: 00:00:00
Request summary
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Processed File: /Users/joerafaniello/Downloads/slow-request-web-service-pods/1-web-service-858455f9b4-7hkwk/production.log
Processed File: /Users/joerafaniello/Downloads/slow-request-web-service-pods/1-web-service-858455f9b4-blqzl/production.log
Processed File: /Users/joerafaniello/Downloads/slow-request-web-service-pods/1-web-service-858455f9b4-jw2b4/production.log
Processed File: /Users/joerafaniello/Downloads/slow-request-web-service-pods/1-web-service-858455f9b4-l6wxr/production.log
Parsed lines: 212904
Skipped lines: 1671
Parsed requests: 53231
Skipped requests: 0
Warnings: no_current_request: 1671
First request: 2021-09-01 15:59:03
Last request: 2021-09-10 17:23:05
Total time analyzed: 10 days
Request distribution per hour
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0:00 ┃ 139 hits/day ┃ ░░░
1:00 ┃ 130 hits/day ┃ ░░░
2:00 ┃ 135 hits/day ┃ ░░░
3:00 ┃ 140 hits/day ┃ ░░░
4:00 ┃ 133 hits/day ┃ ░░░
5:00 ┃ 132 hits/day ┃ ░░░
6:00 ┃ 144 hits/day ┃ ░░░
7:00 ┃ 128 hits/day ┃ ░░
8:00 ┃ 133 hits/day ┃ ░░░
9:00 ┃ 143 hits/day ┃ ░░░
10:00 ┃ 142 hits/day ┃ ░░░
11:00 ┃ 137 hits/day ┃ ░░░
12:00 ┃ 150 hits/day ┃ ░░░
13:00 ┃ 223 hits/day ┃ ░░░░
14:00 ┃ 431 hits/day ┃ ░░░░░░░░
15:00 ┃ 402 hits/day ┃ ░░░░░░░░
16:00 ┃ 449 hits/day ┃ ░░░░░░░░░
17:00 ┃ 444 hits/day ┃ ░░░░░░░░░
18:00 ┃ 408 hits/day ┃ ░░░░░░░░
19:00 ┃ 419 hits/day ┃ ░░░░░░░░
20:00 ┃ 363 hits/day ┃ ░░░░░░░
21:00 ┃ 368 hits/day ┃ ░░░░░░░
22:00 ┃ 354 hits/day ┃ ░░░░░░░
23:00 ┃ 242 hits/day ┃ ░░░░░
Most requested
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::AuthController#show.JSON ┃ 19405 hits ┃ 36.5% ┃ ░░░░░░░░░░░░░░░░░░░░░░░░░░
Api::CloudVolumesController#index.JSON ┃ 5793 hits ┃ 10.9% ┃ ░░░░░░░░
Api::ReportsController#index.JSON ┃ 3997 hits ┃ 7.5% ┃ ░░░░░
Api::ServicesController#index.JSON ┃ 2093 hits ┃ 3.9% ┃ ░░░
Api::VmsController#index.JSON ┃ 1965 hits ┃ 3.7% ┃ ░░░
Api::FloatingIpsController#index.JSON ┃ 1964 hits ┃ 3.7% ┃ ░░░
Api::SecurityGroupsController#index.JSON ┃ 1964 hits ┃ 3.7% ┃ ░░░
Api::NetworkRoutersController#index.JSON ┃ 1963 hits ┃ 3.7% ┃ ░░░
Api::CloudSubnetsController#index.JSON ┃ 1963 hits ┃ 3.7% ┃ ░░░
Api::AvailabilityZonesController#index.JSON ┃ 1940 hits ┃ 3.6% ┃ ░░░
Api::ProvidersController#index.JSON ┃ 1932 hits ┃ 3.6% ┃ ░░░
Api::HostsController#index.JSON ┃ 1932 hits ┃ 3.6% ┃ ░░░
Api::DataStoresController#index.JSON ┃ 1931 hits ┃ 3.6% ┃ ░░░
Api::CloudNetworksController#index.JSON ┃ 1931 hits ┃ 3.6% ┃ ░░░
Api::VmsController#show.JSON ┃ 1906 hits ┃ 3.6% ┃ ░░░
Api::AvailabilityZonesController#show.JSON ┃ 458 hits ┃ 0.9% ┃ ░
#. ┃ 53 hits ┃ 0.1% ┃
Api::NotificationsController#index.JSON ┃ 33 hits ┃ 0.1% ┃
Api::ServiceCatalogsController#update.JSON ┃ 2 hits ┃ 0.0% ┃
Api::ServiceDialogsController#show.JSON ┃ 2 hits ┃ 0.0% ┃
HTTP methods
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
GET ┃ 53226 hits ┃ 100.0% ┃ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
POST ┃ 3 hits ┃ 0.0% ┃
OPTIONS ┃ 2 hits ┃ 0.0% ┃
HTTP statuses returned
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
200 ┃ 33781 hits ┃ 65.5% ┃ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
401 ┃ 17784 hits ┃ 34.5% ┃ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Request duration - by sum ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 3h51m44s ┃ 2.44s ┃ 1.10s ┃ 57ms ┃ 8.84s ┃ 755ms-4.59s
Api::VmsController#index.JSON ┃ 1917 ┃ 2h45m08s ┃ 5.17s ┃ 3.53s ┃ 33ms ┃ 1m59s ┃ 1.51s-9.64s
Api::AuthController#show.JSON ┃ 18027 ┃ 1h24m00s ┃ 279ms ┃ 308ms ┃ 15ms ┃ 16.65s ┃ 22ms-696ms
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 52m28s ┃ 1.62s ┃ 663ms ┃ 189ms ┃ 25.32s ┃ 1.06s-2.57s
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 32m30s ┃ 1.02s ┃ 355ms ┃ 337ms ┃ 10.66s ┃ 685ms-1.56s
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 21m27s ┃ 658ms ┃ 226ms ┃ 332ms ┃ 4.52s ┃ 422ms-1.08s
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 16m07s ┃ 494ms ┃ 205ms ┃ 206ms ┃ 4.09s ┃ 264ms-901ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 13m20s ┃ 409ms ┃ 230ms ┃ 72ms ┃ 5.82s ┃ 221ms-731ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 7m58s ┃ 247ms ┃ 1.46s ┃ 49ms ┃ 1m03s ┃ 103ms-450ms
Api::VmsController#show.JSON ┃ 1905 ┃ 6m46s ┃ 212ms ┃ 1.17s ┃ 50ms ┃ 40.66s ┃ 74ms-521ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 6m23s ┃ 183ms ┃ 557ms ┃ 2ms ┃ 19.02s ┃ 2ms-377ms
Api::HostsController#index.JSON ┃ 1932 ┃ 6m12s ┃ 192ms ┃ 515ms ┃ 63ms ┃ 14.26s ┃ 82ms-371ms
Api::DataStoresController#index.JSON ┃ 1928 ┃ 5m56s ┃ 184ms ┃ 220ms ┃ 57ms ┃ 7.82s ┃ 84ms-383ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 5m55s ┃ 183ms ┃ 243ms ┃ 2ms ┃ 9.23s ┃ 85ms-402ms
Api::ReportsController#index.JSON ┃ 3994 ┃ 3m38s ┃ 54ms ┃ 49ms ┃ 3ms ┃ 2.01s ┃ 26ms-136ms
Api::AvailabilityZonesController#show.JSON ┃ 457 ┃ 17.01s ┃ 37ms ┃ 19ms ┃ 17ms ┃ 194ms ┃ 20ms-92ms
#. ┃ 1 ┃ 6.81s ┃ 6.81s ┃ 0ms ┃ 6.81s ┃ 6.81s ┃ 6.76s-6.98s
Api::NotificationsController#index.JSON ┃ 31 ┃ 4.89s ┃ 157ms ┃ 357ms ┃ 16ms ┃ 1.50s ┃ 15ms-1.51s
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 1.23s ┃ 613ms ┃ 355ms ┃ 362ms ┃ 865ms ┃ 359ms-872ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 1.14s ┃ 568ms ┃ 77ms ┃ 514ms ┃ 623ms ┃ 512ms-632ms
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Request duration - by mean ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
#. ┃ 1 ┃ 6.81s ┃ 6.81s ┃ 0ms ┃ 6.81s ┃ 6.81s ┃ 6.76s-6.98s
Api::VmsController#index.JSON ┃ 1917 ┃ 2h45m08s ┃ 5.17s ┃ 3.53s ┃ 33ms ┃ 1m59s ┃ 1.51s-9.64s
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 3h51m44s ┃ 2.44s ┃ 1.10s ┃ 57ms ┃ 8.84s ┃ 755ms-4.59s
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 52m28s ┃ 1.62s ┃ 663ms ┃ 189ms ┃ 25.32s ┃ 1.06s-2.57s
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 32m30s ┃ 1.02s ┃ 355ms ┃ 337ms ┃ 10.66s ┃ 685ms-1.56s
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 21m27s ┃ 658ms ┃ 226ms ┃ 332ms ┃ 4.52s ┃ 422ms-1.08s
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 1.23s ┃ 613ms ┃ 355ms ┃ 362ms ┃ 865ms ┃ 359ms-872ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 1.14s ┃ 568ms ┃ 77ms ┃ 514ms ┃ 623ms ┃ 512ms-632ms
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 16m07s ┃ 494ms ┃ 205ms ┃ 206ms ┃ 4.09s ┃ 264ms-901ms
Api::ServiceDialogsController#create.JSON ┃ 1 ┃ 419ms ┃ 419ms ┃ 0ms ┃ 419ms ┃ 419ms ┃ 415ms-429ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 13m20s ┃ 409ms ┃ 230ms ┃ 72ms ┃ 5.82s ┃ 221ms-731ms
Api::AuthController#show.JSON ┃ 18027 ┃ 1h24m00s ┃ 279ms ┃ 308ms ┃ 15ms ┃ 16.65s ┃ 22ms-696ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 7m58s ┃ 247ms ┃ 1.46s ┃ 49ms ┃ 1m03s ┃ 103ms-450ms
Api::VmsController#show.JSON ┃ 1905 ┃ 6m46s ┃ 212ms ┃ 1.17s ┃ 50ms ┃ 40.66s ┃ 74ms-521ms
Api::HostsController#index.JSON ┃ 1932 ┃ 6m12s ┃ 192ms ┃ 515ms ┃ 63ms ┃ 14.26s ┃ 82ms-371ms
Api::DataStoresController#index.JSON ┃ 1928 ┃ 5m56s ┃ 184ms ┃ 220ms ┃ 57ms ┃ 7.82s ┃ 84ms-383ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 6m23s ┃ 183ms ┃ 557ms ┃ 2ms ┃ 19.02s ┃ 2ms-377ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 5m55s ┃ 183ms ┃ 243ms ┃ 2ms ┃ 9.23s ┃ 85ms-402ms
Api::ZonesController#index.JSON ┃ 1 ┃ 166ms ┃ 166ms ┃ 0ms ┃ 166ms ┃ 166ms ┃ 165ms-171ms
Api::NotificationsController#index.JSON ┃ 31 ┃ 4.89s ┃ 157ms ┃ 357ms ┃ 16ms ┃ 1.50s ┃ 15ms-1.51s
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Partials rendering time - by sum ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Partials rendering time - by mean ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
View rendering time - by sum ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::AuthController#show.JSON ┃ 18027 ┃ 2.49s ┃ 0ms ┃ 0ms ┃ 0ms ┃ 12ms ┃ 0ms-0ms
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 1.53s ┃ 0ms ┃ 1ms ┃ 0ms ┃ 99ms ┃ 0ms-0ms
Api::ReportsController#index.JSON ┃ 3994 ┃ 984ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 11ms ┃ 0ms-0ms
Api::VmsController#show.JSON ┃ 1905 ┃ 580ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 11ms ┃ 0ms-0ms
Api::HostsController#index.JSON ┃ 1932 ┃ 560ms ┃ 0ms ┃ 3ms ┃ 0ms ┃ 117ms ┃ 0ms-0ms
Api::VmsController#index.JSON ┃ 1917 ┃ 532ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 7ms ┃ 0ms-0ms
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 521ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 487ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 10ms ┃ 0ms-0ms
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 470ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 2ms ┃ 0ms-0ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 466ms ┃ 0ms ┃ 1ms ┃ 0ms ┃ 87ms ┃ 0ms-0ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 465ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 5ms ┃ 0ms-0ms
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 458ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 428ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 13ms ┃ 0ms-0ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 416ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 8ms ┃ 0ms-0ms
Api::DataStoresController#index.JSON ┃ 1928 ┃ 366ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 2ms ┃ 0ms-0ms
Api::AvailabilityZonesController#show.JSON ┃ 457 ┃ 88ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
Api::ProvidersController#options.JSON ┃ 2 ┃ 7ms ┃ 3ms ┃ 0ms ┃ 3ms ┃ 3ms ┃ 3ms-3ms
Api::NotificationsController#index.JSON ┃ 31 ┃ 6ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
View rendering time - by mean ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::ProvidersController#options.JSON ┃ 2 ┃ 7ms ┃ 3ms ┃ 0ms ┃ 3ms ┃ 3ms ┃ 3ms-3ms
Api::VmsController#show.JSON ┃ 1905 ┃ 580ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 11ms ┃ 0ms-0ms
Api::HostsController#index.JSON ┃ 1932 ┃ 560ms ┃ 0ms ┃ 3ms ┃ 0ms ┃ 117ms ┃ 0ms-0ms
Api::VmsController#index.JSON ┃ 1917 ┃ 532ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 7ms ┃ 0ms-0ms
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 1.53s ┃ 0ms ┃ 1ms ┃ 0ms ┃ 99ms ┃ 0ms-0ms
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 521ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 487ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 10ms ┃ 0ms-0ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ReportsController#index.JSON ┃ 3994 ┃ 984ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 11ms ┃ 0ms-0ms
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 470ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 2ms ┃ 0ms-0ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 466ms ┃ 0ms ┃ 1ms ┃ 0ms ┃ 87ms ┃ 0ms-0ms
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 458ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 465ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 5ms ┃ 0ms-0ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 428ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 13ms ┃ 0ms-0ms
Api::NotificationsController#index.JSON ┃ 31 ┃ 6ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
#. ┃ 1 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ZonesController#index.JSON ┃ 1 ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 0ms-0ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 416ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 8ms ┃ 0ms-0ms
Api::AvailabilityZonesController#show.JSON ┃ 457 ┃ 88ms ┃ 0ms ┃ 0ms ┃ 0ms ┃ 1ms ┃ 0ms-0ms
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Database time - by sum ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 1h13m08s ┃ 770ms ┃ 436ms ┃ 13ms ┃ 5.53s ┃ 204ms-1.83s
Api::VmsController#index.JSON ┃ 1917 ┃ 49m36s ┃ 1.55s ┃ 1.12s ┃ 8ms ┃ 23.15s ┃ 402ms-3.55s
Api::AuthController#show.JSON ┃ 18027 ┃ 22m40s ┃ 75ms ┃ 256ms ┃ 2ms ┃ 16.32s ┃ 12ms-371ms
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 19m02s ┃ 587ms ┃ 589ms ┃ 147ms ┃ 24.12s ┃ 244ms-1.19s
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 9m19s ┃ 286ms ┃ 197ms ┃ 80ms ┃ 4.11s ┃ 124ms-592ms
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 7m39s ┃ 234ms ┃ 169ms ┃ 57ms ┃ 3.79s ┃ 92ms-521ms
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 7m16s ┃ 227ms ┃ 301ms ┃ 42ms ┃ 9.96s ┃ 100ms-473ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 6m54s ┃ 211ms ┃ 210ms ┃ 46ms ┃ 5.27s ┃ 80ms-443ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 5m26s ┃ 169ms ┃ 1.45s ┃ 7ms ┃ 1m03s ┃ 47ms-316ms
Api::HostsController#index.JSON ┃ 1932 ┃ 4m30s ┃ 139ms ┃ 513ms ┃ 30ms ┃ 14.20s ┃ 44ms-296ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 4m29s ┃ 129ms ┃ 554ms ┃ 0ms ┃ 18.92s ┃ 0ms-296ms
Api::DataStoresController#index.JSON ┃ 1928 ┃ 4m16s ┃ 132ms ┃ 217ms ┃ 30ms ┃ 7.77s ┃ 46ms-311ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 4m06s ┃ 127ms ┃ 238ms ┃ 0ms ┃ 9.12s ┃ 45ms-296ms
Api::VmsController#show.JSON ┃ 1905 ┃ 1m46s ┃ 55ms ┃ 71ms ┃ 9ms ┃ 878ms ┃ 15ms-273ms
Api::ReportsController#index.JSON ┃ 3994 ┃ 1m13s ┃ 18ms ┃ 19ms ┃ 0ms ┃ 714ms ┃ 5ms-59ms
Api::AvailabilityZonesController#show.JSON ┃ 457 ┃ 5.29s ┃ 11ms ┃ 9ms ┃ 3ms ┃ 101ms ┃ 4ms-38ms
#. ┃ 1 ┃ 1.63s ┃ 1.63s ┃ 0ms ┃ 1.63s ┃ 1.63s ┃ 1.61s-1.66s
Api::NotificationsController#index.JSON ┃ 31 ┃ 1.13s ┃ 36ms ┃ 76ms ┃ 3ms ┃ 342ms ┃ 3ms-342ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 518ms ┃ 259ms ┃ 34ms ┃ 234ms ┃ 283ms ┃ 232ms-287ms
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 262ms ┃ 131ms ┃ 50ms ┃ 95ms ┃ 167ms ┃ 94ms-168ms
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Database time - by mean ┃ Hits ┃ Sum ┃ Mean ┃ StdDev ┃ Min ┃ Max ┃ 95 %tile
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
#. ┃ 1 ┃ 1.63s ┃ 1.63s ┃ 0ms ┃ 1.63s ┃ 1.63s ┃ 1.61s-1.66s
Api::VmsController#index.JSON ┃ 1917 ┃ 49m36s ┃ 1.55s ┃ 1.12s ┃ 8ms ┃ 23.15s ┃ 402ms-3.55s
Api::CloudVolumesController#index.JSON ┃ 5692 ┃ 1h13m08s ┃ 770ms ┃ 436ms ┃ 13ms ┃ 5.53s ┃ 204ms-1.83s
Api::FloatingIpsController#index.JSON ┃ 1943 ┃ 19m02s ┃ 587ms ┃ 589ms ┃ 147ms ┃ 24.12s ┃ 244ms-1.19s
Api::SecurityGroupsController#index.JSON ┃ 1953 ┃ 9m19s ┃ 286ms ┃ 197ms ┃ 80ms ┃ 4.11s ┃ 124ms-592ms
Api::ServiceCatalogsController#update.JSON ┃ 2 ┃ 518ms ┃ 259ms ┃ 34ms ┃ 234ms ┃ 283ms ┃ 232ms-287ms
Api::CloudSubnetsController#index.JSON ┃ 1957 ┃ 7m39s ┃ 234ms ┃ 169ms ┃ 57ms ┃ 3.79s ┃ 92ms-521ms
Api::CloudNetworksController#index.JSON ┃ 1914 ┃ 7m16s ┃ 227ms ┃ 301ms ┃ 42ms ┃ 9.96s ┃ 100ms-473ms
Api::NetworkRoutersController#index.JSON ┃ 1955 ┃ 6m54s ┃ 211ms ┃ 210ms ┃ 46ms ┃ 5.27s ┃ 80ms-443ms
Api::ProvidersController#index.JSON ┃ 1927 ┃ 5m26s ┃ 169ms ┃ 1.45s ┃ 7ms ┃ 1m03s ┃ 47ms-316ms
Api::HostsController#index.JSON ┃ 1932 ┃ 4m30s ┃ 139ms ┃ 513ms ┃ 30ms ┃ 14.20s ┃ 44ms-296ms
Api::DataStoresController#index.JSON ┃ 1928 ┃ 4m16s ┃ 132ms ┃ 217ms ┃ 30ms ┃ 7.77s ┃ 46ms-311ms
Api::ServiceDialogsController#show.JSON ┃ 2 ┃ 262ms ┃ 131ms ┃ 50ms ┃ 95ms ┃ 167ms ┃ 94ms-168ms
Api::ServicesController#index.JSON ┃ 2087 ┃ 4m29s ┃ 129ms ┃ 554ms ┃ 0ms ┃ 18.92s ┃ 0ms-296ms
Api::AvailabilityZonesController#index.JSON ┃ 1937 ┃ 4m06s ┃ 127ms ┃ 238ms ┃ 0ms ┃ 9.12s ┃ 45ms-296ms
Api::ServiceDialogsController#create.JSON ┃ 1 ┃ 119ms ┃ 119ms ┃ 0ms ┃ 119ms ┃ 119ms ┃ 118ms-122ms
Api::AuthController#show.JSON ┃ 18027 ┃ 22m40s ┃ 75ms ┃ 256ms ┃ 2ms ┃ 16.32s ┃ 12ms-371ms
Api::VmsController#show.JSON ┃ 1905 ┃ 1m46s ┃ 55ms ┃ 71ms ┃ 9ms ┃ 878ms ┃ 15ms-273ms
Api::NotificationsController#index.JSON ┃ 31 ┃ 1.13s ┃ 36ms ┃ 76ms ┃ 3ms ┃ 342ms ┃ 3ms-342ms
Api::ZonesController#index.JSON ┃ 1 ┃ 19ms ┃ 19ms ┃ 0ms ┃ 19ms ┃ 19ms ┃ 19ms-19ms
Process blockers (> 1 sec duration)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Api::CloudVolumesController#index.JSON ┃ 4926 hits ┃ 50.2% ┃ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
Api::FloatingIpsController#index.JSON ┃ 1914 hits ┃ 19.5% ┃ ░░░░░░░░░░░░░░
Api::VmsController#index.JSON ┃ 1881 hits ┃ 19.2% ┃ ░░░░░░░░░░░░░░
Api::CloudNetworksController#index.JSON ┃ 881 hits ┃ 9.0% ┃ ░░░░░░
Api::AuthController#show.JSON ┃ 67 hits ┃ 0.7% ┃
Api::SecurityGroupsController#index.JSON ┃ 61 hits ┃ 0.6% ┃
Api::CloudSubnetsController#index.JSON ┃ 23 hits ┃ 0.2% ┃
Api::NetworkRoutersController#index.JSON ┃ 14 hits ┃ 0.1% ┃
Api::VmsController#show.JSON ┃ 11 hits ┃ 0.1% ┃
Api::ProvidersController#index.JSON ┃ 8 hits ┃ 0.1% ┃
Api::AvailabilityZonesController#index.JSON ┃ 7 hits ┃ 0.1% ┃
Api::HostsController#index.JSON ┃ 5 hits ┃ 0.1% ┃
Api::ServicesController#index.JSON ┃ 5 hits ┃ 0.1% ┃
Api::DataStoresController#index.JSON ┃ 5 hits ┃ 0.1% ┃
Api::NotificationsController#index.JSON ┃ 2 hits ┃ 0.0% ┃
Api::ReportsController#index.JSON ┃ 2 hits ┃ 0.0% ┃
#. ┃ 1 hits ┃ 0.0% ┃
from manageiq-api.
jrafanie
commented on July 17, 2024
For the first query:
The include for find is:
options[:include_for_find]
=> {:hardware=>{:hostnames=>{}, :ipaddresses=>{}}, "operating_system"=>{}, "tenant"=>{}, :ems_cluster=>:all_relationships, "hardware"=>{}}
At this location:
and the virtual attributes above were:
=> [
"hostnames",
"operating_system.product_name",
"storage.name",
"tenant.name",
"ipaddresses",
"tags",
"v_owning_datacenter",
"last_compliance_status",
"hardware.cpu_sockets",
"hardware.memory_mb",
"active",
"archived",
"hardware.allocated_disk_storage",
"service.id",
"service.guid
"]
If we annotate all of the virtual attributes, we can see why some are preloaded/optimized and others are not:
=> [
"hostnames", # <--- √ preloaded by include_for_find[:hardware][:hostnames]
"operating_system.product_name", # <--- √ preloaded by include_for_find[:operating_system]
"tenant.name", # <--- √ preloaded by include_for_find[:tenant]
"v_owning_datacenter", # <--- √ preloaded by include_for_find[:ems_cluster][:all_relationships]
"last_compliance_status", # <--- √ Seems to be properly joined
"hardware.cpu_sockets", # <--- √ preloaded by include_for_find[:hardware]
"hardware.memory_mb", # <--- √ preloaded by include_for_find[:hardware]
"active", # <--- √ handled by arel in virtual_attribute :active
"archived", # <--- √ handled by arel in virtual_attribute :archived
"ipaddresses", # <--- ! preloaded by include_for_find[:hardware][:ipaddresses] FROM BASE CLASS uses NOT cloud manager/vm uses (therefore doesn't properly preload subclass uses)
"tags", # <--- !!! ???
"storage.name", # <--- !!! NOT preloaded - is rbac'able
"service.id", # <--- !!! NOT preloaded - is rbac'able
"service.guid", # <--- !!! NOT preloaded - is rbac'able
"hardware.allocated_disk_storage" # <--- !!! allocated_disk_storage on vm or template works, hardware does not preload. vm or template has virtual_delegate :allocated_disk_storage with proper uses... hardware has virtual_sum :allocated_disk_storage, :disks, :size but disks are not preloaded
]
So, to conclude:
ipaddressesis expensive because the optimization for this database resides in theusesclause of the subclass but the API only uses the base class's uses so this cannot be optimized as it is now.- tags - NOT sure
- storage and service are both in the list of rbac classes and therefore even though we're doing a
vmscollection API request, we have to circle back for each vm to check the rbac on it's storage and service to ensure the user has permission to see each. hardware.allocated_disk_storageisn't optimized for the API from vms collection whereas the virtualallocated_disk_storageon the requested collection's class is optimized so, substitutingallocated_disk_storageforhardware.allocated_disk_storagecuts out lots of queries, see below:
be miqperf benchmark -a -c 3 "/api/vms?offset=0&limit=1000&expand=resources&attributes=id,href,vendor,type,name,hostname,hostnames,availability_zone_id,ems_id,operating_system.product_name,storage.name,created_on,updated_on,guid,power_state,state_changed_on,tenant.name,ipaddresses,tags,v_owning_datacenter,last_compliance_status,hardware.cpu_sockets,hardware.memory_mb,active,archived,hardware.allocated_disk_storage,cloud,service.id,service.guid"
/api/vms
| ms | queries | query (ms) | rows |
| ---: | ---: | ---: | ---: |
| 5173 | 2047 | 603.7 | 900 |
| 3564 | 2047 | 840.0 | 900 |
| 3031 | 2047 | 520.5 | 900 |
vs.
% be miqperf benchmark -a -c 3 "/api/vms?offset=0&limit=1000&expand=resources&attributes=id,href,vendor,type,name,hostname,hostnames,availability_zone_id,ems_id,operating_system.product_name,storage.name,created_on,updated_on,guid,power_state,state_changed_on,tenant.name,ipaddresses,tags,v_owning_datacenter,last_compliance_status,hardware.cpu_sockets,hardware.memory_mb,active,archived,allocated_disk_storage,cloud,service.id,service.guid"
/api/vms
| 2917 | 1581 | 471.1 | 900 |
| 2610 | 1581 | 458.1 | 900 |
| 2634 | 1581 | 442.8 | 900 |
from manageiq-api.
jrafanie
commented on July 17, 2024
Note, all other API requests after the first are improved drastically and need no other changes if you drop tags from the attribute list, which is also a question in the first API request, so we can ignore the other reported slow requests.
from manageiq-api.
Fryguy
commented on July 17, 2024
So my follow up question would be whether or not reporting actually preloads this stuff properly with similar includes? I've questioned in the past why we don't just reuse the same preloading mechanism from reporting (or maybe we are and both are slow).
from manageiq-api.
jrafanie
commented on July 17, 2024
So, questions left to figure out:
-
hardware.allocated_disk_storagedoesn't properly use the uses on the singular association's virtual attribute whereas the already defined virtual on vmallocated_disk_storage. This is yet another example where thedotnotation is busted. It doesn't support non-singular associations and now demonstrates that it also doesn't properly preload virtuals on singular associations. Do we try to deprecate / remove / push people away from usingdotnotation? Here be dragons. -
why tags doesn't preload ever? It's not in rbac, so doesn't need a roundtrip to see if the user who requested vms is also allowed to see the tags on each vm.
-
Users will not know what classes/associations off the main collection will need to be routed back through rbac to ensure the user can not only see the original collection but the associations off the collection. I don't know how to make this faster but this is a 💣 waiting to go off. Any of these off of a primary collection will cause N+1s.
from manageiq-api.
jrafanie
commented on July 17, 2024
I moved into reporting to verify the problem with 1) hardware.allocated_disk_storage vs. allocated_disk_storage from vms
report = MiqReport.new(
:name => "n",
:title => "t",
:rpt_group => "Custom",
:rpt_type => "Custom",
:db => "VmOrTemplate",
:cols => %w(name allocated_disk_storage),
)
report._async_generate_table(nil, :userid => "admin", :mode => "async",
:report_source => "Requested by user")
puts report.table.data.take(2).inspectResults in:
=> #<MiqReport id: nil, name: "n", title: "t", rpt_group: "Custom", rpt_type: "Custom", priority: nil, db: "VmOrTemplate", cols: ["name", "allocated_disk_storage"], include: nil, col_order: nil, headers: nil, conditions: nil, order: nil, sortby: nil, group: nil, graph: nil, dims: nil, created_on: nil, updated_on: nil, filename: nil, file_mtime: nil, categories: nil, timeline: nil, template_type: nil, where_clause: nil, db_options: nil, generate_cols: nil, generate_rows: nil, col_formats: nil, tz: nil, time_profile_id: nil, display_filter: nil, col_options: nil, rpt_options: nil, miq_group_id: nil, user_id: nil>
MiqTask Load (9.8ms) SELECT "miq_tasks".* FROM "miq_tasks" WHERE "miq_tasks"."id" IS NULL LIMIT $1 [["LIMIT", 1]]
MiqTask Inst Including Associations (0.0ms - 0rows)
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" BETWEEN $1 AND $2 AND "users"."userid" = $3 LIMIT $4 [["id", 0], ["id", 999999999999], ["userid", "admin"], ["LIMIT", 1]]
User Inst Including Associations (0.1ms - 1rows)
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" BETWEEN $1 AND $2 AND "users"."userid" = $3 LIMIT $4 [["id", 0], ["id", 999999999999], ["userid", "admin"], ["LIMIT", 1]]
User Inst Including Associations (0.1ms - 1rows)
MiqGroup Load (0.2ms) SELECT "miq_groups".* FROM "miq_groups" WHERE "miq_groups"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
MiqGroup Inst Including Associations (0.1ms - 1rows)
Entitlement Load (0.2ms) SELECT "entitlements".* FROM "entitlements" WHERE "entitlements"."miq_group_id" = $1 LIMIT $2 [["miq_group_id", 2], ["LIMIT", 1]]
Entitlement Inst Including Associations (0.1ms - 1rows)
Tenant Load (0.3ms) SELECT "tenants".* FROM "tenants" WHERE "tenants"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Tenant Inst Including Associations (0.1ms - 1rows)
MiqUserRole Load (0.4ms) SELECT "miq_user_roles".* FROM "miq_user_roles" INNER JOIN "entitlements" ON "miq_user_roles"."id" = "entitlements"."miq_user_role_id" WHERE "entitlements"."miq_group_id" = $1 LIMIT $2 [["miq_group_id", 2], ["LIMIT", 1]]
MiqUserRole Inst Including Associations (0.1ms - 1rows)
VmOrTemplate Load (8.3ms) SELECT "vms".*, (SELECT (COALESCE((SELECT SUM("disks"."size") FROM "disks" WHERE "disks"."hardware_id" = "hardwares"."id"), 0)) FROM "hardwares" WHERE "hardwares"."vm_or_template_id" = "vms"."id" LIMIT 1) AS "allocated_disk_storage" FROM "vms"
VmOrTemplate Inst Including Associations (60.1ms - 844rows)
=> nil
[#<Ruport::Data::Record:0x00007fdce1c7ee60 @data={"name"=>"rested-squid-qr26x-master", "allocated_disk_storage"=>0, "id"=>1}, @attributes=["name", "allocated_disk_storage", "id"]>, #<Ruport::Data::Record:0x00007fdce1c7e2f8 @data={"name"=>"cluster-acm-up2-vcz75-master", "allocated_disk_storage"=>0, "id"=>2}, @attributes=["name", "allocated_disk_storage", "id"]>]
Vs. using hardware.allocated_disk_storage:
report = MiqReport.new(
:name => "n",
:title => "t",
:rpt_group => "Custom",
:rpt_type => "Custom",
:db => "VmOrTemplate",
:cols => %w(name hardware.allocated_disk_storage),
)
report._async_generate_table(nil, :userid => "admin", :mode => "async",
:report_source => "Requested by user")
puts report.table.data.take(2).inspectresults in:
=> #<MiqReport id: nil, name: "n", title: "t", rpt_group: "Custom", rpt_type: "Custom", priority: nil, db: "VmOrTemplate", cols: ["name", "hardware.allocated_disk_storage"], include: nil, col_order: nil, headers: nil, conditions: nil, order: nil, sortby: nil, group: nil, graph: nil, dims: nil, created_on: nil, updated_on: nil, filename: nil, file_mtime: nil, categories: nil, timeline: nil, template_type: nil, where_clause: nil, db_options: nil, generate_cols: nil, generate_rows: nil, col_formats: nil, tz: nil, time_profile_id: nil, display_filter: nil, col_options: nil, rpt_options: nil, miq_group_id: nil, user_id: nil>
MiqTask Load (0.3ms) SELECT "miq_tasks".* FROM "miq_tasks" WHERE "miq_tasks"."id" IS NULL LIMIT $1 [["LIMIT", 1]]
MiqTask Inst Including Associations (0.0ms - 0rows)
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" BETWEEN $1 AND $2 AND "users"."userid" = $3 LIMIT $4 [["id", 0], ["id", 999999999999], ["userid", "admin"], ["LIMIT", 1]]
User Inst Including Associations (0.1ms - 1rows)
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" BETWEEN $1 AND $2 AND "users"."userid" = $3 LIMIT $4 [["id", 0], ["id", 999999999999], ["userid", "admin"], ["LIMIT", 1]]
User Inst Including Associations (0.1ms - 1rows)
MiqGroup Load (0.2ms) SELECT "miq_groups".* FROM "miq_groups" WHERE "miq_groups"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
MiqGroup Inst Including Associations (0.1ms - 1rows)
Entitlement Load (0.1ms) SELECT "entitlements".* FROM "entitlements" WHERE "entitlements"."miq_group_id" = $1 LIMIT $2 [["miq_group_id", 2], ["LIMIT", 1]]
Entitlement Inst Including Associations (0.1ms - 1rows)
MiqServer Load (0.6ms) SELECT "miq_servers".* FROM "miq_servers" WHERE "miq_servers"."guid" = $1 LIMIT $2 [["guid", "b47e7850-62d6-45c7-ba7e-e67640b8742e"], ["LIMIT", 1]]
MiqServer Inst Including Associations (0.1ms - 1rows)
Tenant Load (0.2ms) SELECT "tenants".* FROM "tenants" WHERE "tenants"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Tenant Inst Including Associations (0.1ms - 1rows)
MiqUserRole Load (0.2ms) SELECT "miq_user_roles".* FROM "miq_user_roles" INNER JOIN "entitlements" ON "miq_user_roles"."id" = "entitlements"."miq_user_role_id" WHERE "entitlements"."miq_group_id" = $1 LIMIT $2 [["miq_group_id", 2], ["LIMIT", 1]]
MiqUserRole Inst Including Associations (0.1ms - 1rows)
VmOrTemplate Load (2.6ms) SELECT "vms".* FROM "vms"
VmOrTemplate Inst Including Associations (53.0ms - 844rows)
Hardware Load (0.2ms) SELECT "hardwares".* FROM "hardwares" WHERE "hardwares"."vm_or_template_id" = $1 LIMIT $2 [["vm_or_template_id", 1], ["LIMIT", 1]]
Hardware Inst Including Associations (0.1ms - 1rows)
Hardware Load (0.1ms) SELECT "hardwares".* FROM "hardwares" WHERE "hardwares"."vm_or_template_id" = $1 LIMIT $2 [["vm_or_template_id", 2], ["LIMIT", 1]]
Hardware Inst Including Associations (0.1ms - 1rows)
Hardware Load (0.2ms) SELECT "hardwares".* FROM "hardwares" WHERE "hardwares"."vm_or_template_id" = $1 LIMIT $2 [["vm_or_template_id", 3], ["LIMIT", 1]]
... HUNDREDS of lines like the above ^
Hardware Load (0.2ms) SELECT "hardwares".* FROM "hardwares" WHERE "hardwares"."vm_or_template_id" = $1 LIMIT $2 [["vm_or_template_id", 841], ["LIMIT", 1]]
Hardware Inst Including Associations (0.1ms - 1rows)
(0.2ms) SELECT SUM("disks"."size") FROM "disks" WHERE "disks"."hardware_id" = $1 [["hardware_id", 1]]
(0.1ms) SELECT SUM("disks"."size") FROM "disks" WHERE "disks"."hardware_id" = $1 [["hardware_id", 2]]
(0.1ms) SELECT SUM("disks"."size") FROM "disks" WHERE "disks"."hardware_id" = $1 [["hardware_id", 3]]
... HUNDREDS more lines like above ^
(0.1ms) SELECT SUM("disks"."size") FROM "disks" WHERE "disks"."hardware_id" = $1 [["hardware_id", 841]]
=> nil
[#<Ruport::Data::Record:0x00007fdcb5f6ca68 @data={"name"=>"rested-squid-qr26x-master", "hardware.allocated_disk_storage"=>0.0, "id"=>1}, @attributes=["name", "hardware.allocated_disk_storage", "id"]>, #<Ruport::Data::Record:0x00007fdcb5f77490 @data={"name"=>"cluster-acm-up2-vcz75-master", "hardware.allocated_disk_storage"=>0.0, "id"=>2}, @attributes=["name", "hardware.allocated_disk_storage", "id"]>]
from manageiq-api.
jrafanie
commented on July 17, 2024
Note, here's the existing bug on the inability to use dot notation on plural associations: #871
from manageiq-api.
NickLaMuro
commented on July 17, 2024
Created a patch for being able to replace service.id and service.type:
https://gist.github.com/NickLaMuro/3fc67cb3db99152dff36891806de59e2
However, those classes particpate in RBac, so might not be something we can actually use. Might need Keenan to jump in and throw out some ideas.
from manageiq-api.
Fryguy
commented on July 17, 2024
I'm curious why that gist wouldn't use a proper association (at least for the service id)...is there no proper relationship?
from manageiq-api.
NickLaMuro
commented on July 17, 2024
@Fryguy for what it is worth, I was mostly copying my work previously from:
https://github.com/ManageIQ/manageiq/blob/master/app/models/service/aggregation.rb
So it is possible it would be done easier, but I am pretty sure I remember that making the Vm.select(:service_id)... type thing work required handling the join directly.
Again, because Service participates in Rbac, I don't think we can use these attributes as they would circumvent any Rbac checks that are currently in place.
from manageiq-api.
jrafanie
commented on July 17, 2024
I wonder if we should review the RBAC'able classes. I don't know if they all still need to be RBAC'able, if others need to be added, and if there are downsides/bugs that could happen by switching which classes are in the participates in rbac list.
Additionally, if there is no easy way to avoid the roundtrip on each rbac'able association from the primary collection, I wonder if it makes sense to deprecate doing more than one or two of these from the primary collection. We allow nearly anything and it makes it tough to prevent amazingly slow API requests.
Along those lines, LIMIT only pertains to the primary collection and not associations/methods on the primary collection that could return greater than the LIMIT results. It feels like there's so many ways to accidentally create slow API requests.
from manageiq-api.
kbrock
commented on July 17, 2024
fun fact
ipaddresses is expensive because the optimization for this database resides in the uses clause of the subclass but the API only uses the base class's uses so this cannot be optimized as it is now.
we lost that when we got rid of that deadlock when you load one class it required that you load another class.
That change meant that the uses no across a delegate no longer inherit the uses of a child class.
We can add that back in per reference, thought that would be a little non-dry.
Another issue is we rbac everything. and the current implementation of rbac requires reloading the child classes.
So even if you preload sub collections. you need to reload those sub collections. the second time taking rbac into account.
And lots of times, when you reload those classes, you loose the preloads that you did on the first objects.
If we were able to change the way that preload works and use an rbac query instead of the raw query, I think we would see big improvements.
@NickLaMuro is the master of preloading associations with a scope and a qualifier.
Maybe the issues I am mentioning have already been fixed and are not the culprit here. Coincidentally, these issues are why I was trying so hard to make rbac sql only and why I was diving into the alternative query options. A, so we could apply rbac on all queries and not do the double trip (loosing data in the process). And B, so we could rbac objects that are already in memory and not access the database.
from manageiq-api.
kbrock
commented on July 17, 2024
I was never comfortable with the preload interface. If you manually loaded an association and tried to stick it into a collection (manually doing the includes()), it was always so cumbersome. Maybe Nick has figured out how to do this more easily when diving into the preload gook.
from manageiq-api.
jrafanie
commented on July 17, 2024
Another issue is we rbac everything. and the current implementation of rbac requires reloading the child classes.
yeah, that's why I mentioned ☝️ this. I don't know that we've reviewed each of the rbac'able classes to review questions like:
-
Does this class need to be passed through rbac? Many are excluded from rbac and it's unclear if that criteria for rbac/not rbac is clearly defined.
-
Does this class need to be passed through rbac if the user is already rbac authorized through an association? For example, if you've rbac checked all the vms, do you really need to rbac again to make sure the user is authorized for the services attached to each vm? I don't know.
-
How can we avoid a nearly unlimited result for each association of the primary collection? When querying services and pulling in vms, any
limiton the services association has no control on the paging or anything on this secondary collection.
from manageiq-api.
NickLaMuro
commented on July 17, 2024
Maybe Nick has figured out how to do this more easily when diving into the preload gook.
Once again, you give me way too much credit...
from manageiq-api.
NickLaMuro
commented on July 17, 2024
Update: Been working on a solution for the tags portion of this today and might have something that we can vet in a bit. Will probably require patching into an environment so we can have it stew and see if it causes any issues, because I fear it has a chance of blowing other things up if we aren't careful.
from manageiq-api.
miq-bot
commented on July 17, 2024
This issue has been automatically marked as stale because it has not been updated for at least 3 months.
If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions! More information about the ManageIQ triage process can be found in the triage process documentation.
from manageiq-api.
Related Issues (20)
- Switch tag subcollection missing HOT 2
- Api for timeline charts. HOT 7
- Need API for Cloud Volume Snapshot Subcollection HOT 3
- Sporadic test failure in authentication_spec HOT 4
- vm_reconfigure request has no effect on vm params HOT 10
- Need Policy Simulation Endpoints for non-VM options HOT 2
- Template API Issue with child_resources HOT 5
- If a resource doesn’t come in as hash or `{}` return bad request error HOT 4
- Sporadic test failure with authentication_spec HOT 12
- API Interface Changes HOT 5
- Issue while creating a Custom Button with expression via API HOT 9
- Incorrect work of role accesses in API HOT 5
- New API endpoints HOT 2
- [OPARIN] API Interface Changes
- Add support for Native Console over the API /vm#request_console
- [RFE] Ability to change Catalog Item Request Info via API HOT 8
- Auth API action
- Can't create a user for multiple groups HOT 7
- Dependency Dashboard
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from manageiq-api.