Code Monkey home page Code Monkey logo

manageiq-api's Introduction

ManageIQ::Api

CI Maintainability Test Coverage

Chat

Build history for master branch

REST API plugin for ManageIQ.

Development

See the section on plugins in the ManageIQ Developer Setup

For quick local setup run bin/setup, which will clone the core ManageIQ repository under the spec directory and setup necessary config files. If you have already cloned it, you can run bin/update to bring the core ManageIQ code up to date.

License

The gem is available as open source under the terms of the Apache License 2.0.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

manageiq-api's People

Contributors

abellotti avatar agrare avatar andyvesel avatar aparnakarve avatar bdunne avatar blomquisg avatar chessbyte avatar chrisarcand avatar d-m-u avatar djberg96 avatar fabiendupont avatar fryguy avatar gtanzillo avatar h-kataria avatar himdel avatar imtayadeway avatar isimluk avatar iv1111 avatar jprause avatar jrafanie avatar jvlcek avatar kbrock avatar lfu avatar lpichler avatar martinpovolny avatar mkanoor avatar mzazrivec avatar nicklamuro avatar skateman avatar tzumainn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

manageiq-api's Issues

Rein in message passing to `Object` in collection config

def [](collection_name)
# Config::Options implements a system method causing self[:system] to error.
# i.e. subcollection name system in an arbitraty path /api/automate/manageiq/system
@cfg[collection_name.to_sym]
rescue
nil
end

As brought up on Gitter by @himdel, we should rein in how we handle the issue fixed by this rescue. We're currently still sending the message (calling the method) unrestricted on the config object, so you can send things like display (which writes to stdout), etc. That's less than ideal.

At the very least, if this is indeed the best solution, I wonder what error is being rescued here.

Unable to login with a newly created user

How to reproduce:

  1. In classic UI, create a new role, new group and a new user
  2. try to login into the UI with the new user

You'll see the login fails.

During login, the following request is sent to API:

I, [2018-04-24T17:23:31.655840 #31920]  INFO -- : Started GET "/api/auth?requester_type=ui" for ::1 at 2018-04-24 17:23:31 +0200
I, [2018-04-24T17:23:31.748765 #31920]  INFO -- : Processing by Api::AuthController#show as JSON
I, [2018-04-24T17:23:31.749126 #31920]  INFO -- :   Parameters: {"requester_type"=>"ui"}
I, [2018-04-24T17:23:31.841456 #31920]  INFO -- : Filter chain halted as :require_api_user_or_token rendered or redirected
I, [2018-04-24T17:23:31.841585 #31920]  INFO -- : Completed 401 Unauthorized in 92ms (ActiveRecord: 10.2ms)

The json that comes back:

{"error":{"kind":"unauthorized","message":"Invalid User user01 specified, User's Group is missing","klass":"Api::AuthenticationError"}}

Inretestingly, logging in with admin user works fine.

[RFE] GraphQL

Adding a graphQL interface for the API would be awesome (this would be an addition to, not a replacement)

Don't stub API config in end-to-end tests

This is a test smell and reduces our confidence in the system's integration.

Link:

# TODO: as the api.yml settings/categories expand in the future to include
# different types of categories, full, partial and single entries, we would no
# longer need this stubbing logic or the above sample_settings as we can
# test the different cases with the default api.yml categories provided.
#
def stub_api_settings_categories(value)
settings_config = Api::ApiConfig.collections["settings"].dup
settings_config["categories"] = value
allow(Api::ApiConfig.collections).to receive("settings") { settings_config }
end

Duplicate user id created when authenticating via ext-auth ldap

In a configuration with external authentication and SSSD/LDAP:

  • logging in to UI via userid in the UI creates one user object
  • authenticating an API request as userid@domain creates a second user object.

This is on Fine release and UPN user objects (userid@domain) should be used for all.

Replace hide=resources for count purposes

We currently use ?hide=resources to hide href's in collections for the purposes of reducing the response when a client only cares about the count of the resources.

Instead of doing this, a more conventional, semantically proper, and performant way to do this is by either including the total count in the headers of the collection request whereby a client can then just request HEAD /resources to return the count without a response body at all, or simply add a proper route with only the count as a response (/resources/count)

I propose the former, as using HTTP semantics is a good thing.

The HTTP HEAD method requests the headers that are returned if the specified resource would be requested with an HTTP GET method. Such a request can be done before deciding to download a large resource to save bandwidth, for example.

A header such as x-resource-count or whatever should be added to collections and clients such as the SUI should be using the header from a HEAD request. This avoids wasting sending a response body at all and makes much more sense semantically on an action that the SUI uses quite often for sidebar counts.

"undefined method `service_dialog_ident'

"undefined method `service_dialog_ident' for #Api::ServicesController:0x007fd53c1b89d8 Did you mean? service_ident",

You can see this error in the SUI when ordering a service dialog from a custom button.

Can't know if password/auth_key changed

For security reasons, API omits on GET /users password field, /providers endpoints password, auth_key (and maybe userid?) fields etc. Makes sense.
This caused several problems when developing ansible modules, that try to use API in idempotent style.

  1. You can't compare existing to desired to decide if to POST the edit. The only way to do an idempotent update is always do the POST. That's actually OK.

  2. You can't report "changed": true / false correctly, which is considered important in ansible. Have to always assume true :-(

    • Not knowing whether anything changed complicates polling for auth validation (that is only queued on changes). I'll elaborate in another issue.

=> The solution I propose is API should return a "changed" boolean. Security wise, the only info it would leak is whether you already knew the credential, which is already something you can test.

cc @yaacov

double slash after host `http://127.0.0.1:3000//api` causes error

$ curl -u admin:smartvm 'http://127.0.0.1:3000//api'
{"error":{"kind":"bad_request","message":"Unsupported Collection api specified","klass":"Api::BadRequestError"}}

Not sure RFC-compliant to pass an extra slash, but easy to do it concatenating parts to base URL and user gives base url ending with /... (seen in ansible based on python client)

Sounds like when it routes to controller it dons normalize to /api, but then when parsing which collection it refers to it does simple "after 2nd slash is collection name" on original url?

api.log:

[----] I, [2017-10-15T11:42:47.015562 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request_initiated)  
[----] I, [2017-10-15T11:42:47.015697 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) API Request:    {:requested_at=>"2017-10-15 08:42:47 UTC", :method=>"GET", :url=>"http://127.0.0.1:3000//api"}
[----] I, [2017-10-15T11:42:47.117727 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) Authentication: {:type=>"basic", :token=>nil, :x_miq_group=>nil, :user=>"admin"}
[----] I, [2017-10-15T11:42:47.119546 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) Authorization:  {:user=>"admin", :group=>"EvmGroup-super_administrator", :role=>"EvmRole-super_administrator", :tenant=>"My Company"}
[----] I, [2017-10-15T11:42:47.120078 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) Request:        {:method=>:get, :action=>"read", :fullpath=>"//api", :url=>"http://127.0.0.1:3000//api", :base=>"http://127.0.0.1:3000", :path=>"//api", :prefix=>"/", :version=>"3.0.0-pre", :api_prefix=>"http://127.0.0.1:3000/", :collection=>"api", :c_suffix=>nil, :collection_id=>nil, :subcollection=>nil, :subcollection_id=>nil}
[----] I, [2017-10-15T11:42:47.120480 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) Parameters:     {"action"=>"index", "controller"=>"api/api", "format"=>"json", "body"=>{}}
[----] E, [2017-10-15T11:42:47.120730 #12446:2ab8b45f8b64] ERROR -- : MIQ(Api::ApiController.api_error) API Error
[----] E, [2017-10-15T11:42:47.120778 #12446:2ab8b45f8b64] ERROR -- : MIQ(Api::ApiController.api_error) Api::BadRequestError: Unsupported Collection api specified
[----] I, [2017-10-15T11:42:47.121233 #12446:2ab8b45f8b64]  INFO -- : MIQ(Api::ApiController.log_request) Response:       {:completed_at=>"2017-10-15 08:42:47 UTC", :size=>"0.112 KBytes", :time_taken=>"0.106 Seconds", :status=>400}

Don't bypass authorization by rendering resources on OPTIONS requests

The OPTIONS response for container deployments was rendering back resources in the response - bypassing both authentication and RBAC. The current implementation made a workaround for this - making authentication optional so as to give some protection to this data while preserving the CORS contract. The resources however are still freely given without going through RBAC.

This needs to be refactored out of the OPTIONS request as it does not qualify as metadata, and sets an extremely bad precedent. There is no reason why these resources cannot be queried through the regular GET method.

Link:

def options
# TODO: this service is rendering resources which (a) require
# authentication (problematic for CORS compatibility), and (b)
# are not being properly filtered by RBAC
if authentication_requested?
require_api_user_or_token
render_options(:container_deployments, ContainerDeploymentService.new.all_data)
else
super
end
end
private
def authentication_requested?
[HttpHeaders::MIQ_TOKEN, HttpHeaders::AUTH_TOKEN, "HTTP_AUTHORIZATION"].any? do |header|
request.headers.include?(header)
end

[RFE] Ability to limit fields returned to those specified in a query

Overview

When querying collections with lots of elements (in my case vms) and there are only a few attributes required to be returned, you still need to use expand=resources. When specifying expand=resources this causes the full base object (VM) to be returned which results in a lot of data, slowing down the query performance.

Example

GET https://server.domain.tld/api/vms?expand=resources&attributes=href,id,name,operating_system.product_name
{
    "name": "vms",
    "count": 361,
    "subcount": 361,
    "resources": [
        {
            "href": "https://server.domain.tld/api/vms/100000000000208",
            "id": 100000000000208,
            "vendor": "vmware",
            "name": "vm01.domain.tld",
            "location": "vm01.domain.tld/vm01.domain.tld.vmx",
            "last_sync_on": "2017-10-05T16:02:44Z",
            "created_on": "2017-08-15T19:08:39Z",
            "updated_on": "2017-10-06T15:00:17Z",
            "guid": "abc123",
            "last_scan_on": "2017-10-05T16:02:44Z",
            "last_scan_attempt_on": "2017-10-05T16:00:46Z",
            "uid_ems": "xyz345",
            "boot_time": "2017-10-06T14:51:48Z",
            "tools_status": "toolsNotRunning",
            "standby_action": "checkpoint",
            "power_state": "unknown",
            "state_changed_on": "2017-10-06T15:00:17Z",
            "previous_state": "poweredOff",
            "connection_state": "connected",
            "last_perf_capture_on": "2017-10-06T14:36:40Z",
            "memory_reserve": 0,
            "memory_reserve_expand": false,
            "memory_limit": -1,
            "memory_shares": 122880,
            "memory_shares_level": "normal",
            "cpu_reserve": 0,
            "cpu_reserve_expand": false,
            "cpu_limit": -1,
            "cpu_shares": 4000,
            "cpu_shares_level": "normal",
            "template": false,
            "ems_ref_obj": "vm-123",
            "miq_group_id": 100000000000001,
            "linked_clone": true,
            "fault_tolerance": false,
            "type": "ManageIQ::Providers::Vmware::InfraManager::Vm",
            "ems_ref": "vm-123",
            "cloud": false,
            "raw_power_state": "unknown",
            "tenant_id": 100000000000001,
            "cpu_hot_add_enabled": false,
            "cpu_hot_remove_enabled": false,
            "memory_hot_add_enabled": false,
            "actions": [
                {
                    "name": "edit",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "add_lifecycle_event",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "add_event",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "refresh",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "shutdown_guest",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "reboot_guest",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "start",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "stop",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "suspend",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "shelve",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "shelve_offload",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "pause",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "request_console",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "reset",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "retire",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "delete",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "set_owner",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "set_ownership",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "scan",
                    "method": "post",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                },
                {
                    "name": "delete",
                    "method": "delete",
                    "href": "https://server.domain.tld/api/vms/100000000000208"
                }
            ]
        },
   ....
    ]
}

Desired Behavior

What i really want is to limit the fields returned to only those specified in the query (output like shown in #241 ). Maybe a syntax of /api/<collection>?attributes=a,b,c ?

GET https://server.domain.tld/api/vms?attributes=href,id,name,operating_system.product_name
{
    "name": "vms",
    "count": 361,
    "subcount": 361,
    "resources": [
        {
            "href": "https://server.domain.tldapi/vms/100000000000208",
            "id": 100000000000208,
            "operating_system": {
                "product_name": "Red Hat Enterprise Linux Server release 7.4 (Maipo)"
            }
        },
  ...
    ]
}

[RFE] Supported actions API

We need an api endpoint that would return the supported actions for VMs (things like snapshot_supported, etc)

Cannot create working VM reconfiguration request using the API

I try to reconfigure a VM using the API using the /api/requests API
I post the following body:

{  
   "action":"create",
   "options":{  
      "src_ids":[  
         "80000000000078"
      ],
      "request_type":"vm_reconfigure",
      "disk_add":[  
         {  
            "disk_size_in_mb":"20",
            "persistent":true,
            "thin_provisioned":false,
            "dependent":true,
            "bootable":false
         }
      ]
   },
   "auto_approve":false
}

The request is accepted, but is not handled by the VMWare provider, I receive the following error message:

Error: add_disk_config_spec: Disk size is required to add a new disk.

Investigating this issue comes down the API inserts the reconfiguration request different than the classic-ui. When looking in the database (miq_requests.options) inserted using the UI:

---
:src_ids:
- '80000000000078'
:disk_add:
- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
  disk_size_in_mb: '15'
  persistent: true
  thin_provisioned: true
  new_controller_type: VirtualLsiLogicController
  dependent: true
  bootable: false
:request_type: :vm_reconfigure

Request inserted using the API in database (miq_requests.options)

---
:src_ids:
- '80000000000078'
:request_type: vm_reconfigure
:disk_add:
- disk_size_in_mb: '20'
  persistent: true
  thin_provisioned: false
  dependent: true
  bootable: false
:delivered_on: 2018-01-04 14:59:14.289655000 Z

Using the API the !ruby/hash:ActiveSupport::HashWithIndifferentAccess is missing.
The providers read the options using tokens leading to the exception.

This could be fixed by changing possible all providers, or inserting the options not as a regular hash.

Expose api setting `max_results_per_page` to the api

As a follow on to #113 UIs would benefit from being able to communicate when there is a hard stop on pagination of results, will prevent showing the user things they can't do

oh hum I am user, i want to see the next 1000000 instances of this thing, whaaaa why do i only see 10 !!!! ITS A BUG ITS A BUG ITS A BUG

<3

PhysicalChassisController - duplicate controller name

PhysicalChassisController already exists, provided by manageiq-ui-classic/app/controllers/physical_chassis_controller.rb.

But thanks to the API defining a controller by the same name, accessing /physical_chassis/show_list now fails:

[----] I, [2018-10-10T14:08:40.918265 #13280:2aeef94b9da4]  INFO -- : Started GET "/physical_chassis/show_list" for ::1 at 2018-10-10 14:08:40 +0000
[----] F, [2018-10-10T14:08:40.956717 #13280:2aeef94b9da4] FATAL -- :                                                                  
[----] F, [2018-10-10T14:08:40.956815 #13280:2aeef94b9da4] FATAL -- : LoadError (Unable to autoload constant PhysicalChassisController, expected /home/himdel/manageiq-api/app/controllers/api/physical_chassis_controller.rb to define it):
[----] F, [2018-10-10T14:08:40.956851 #13280:2aeef94b9da4] FATAL -- :   
[----] F, [2018-10-10T14:08:40.956894 #13280:2aeef94b9da4] FATAL -- : activesupport (5.0.7) lib/active_support/dependencies.rb:513:in `load_missing_constant'
activesupport (5.0.7) lib/active_support/dependencies.rb:203:in `const_missing'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:268:in `const_get'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:268:in `block in constantize'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `each'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `inject'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `constantize'
activesupport (5.0.7) lib/active_support/dependencies.rb:584:in `get'
activesupport (5.0.7) lib/active_support/dependencies.rb:615:in `constantize'
actionpack (5.0.7) lib/action_dispatch/http/request.rb:81:in `controller_class'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:44:in `controller'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:30:in `serve'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:39:in `block in serve'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:26:in `each'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:26:in `serve'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:727:in `call'
/home/himdel/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/manageiq-graphql-5f68621f2791/lib/manageiq/graphql/rest_api_proxy.rb:18:in `call'
secure_headers (3.0.3) lib/secure_headers/middleware.rb:10:in `call'
lib/request_started_on_middleware.rb:12:in `call'
rack (2.0.5) lib/rack/etag.rb:25:in `call'
rack (2.0.5) lib/rack/conditional_get.rb:25:in `call'
rack (2.0.5) lib/rack/head.rb:12:in `call'
rack (2.0.5) lib/rack/session/abstract/id.rb:232:in `context'
rack (2.0.5) lib/rack/session/abstract/id.rb:226:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/cookies.rb:613:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/callbacks.rb:38:in `block in call'
activesupport (5.0.7) lib/active_support/callbacks.rb:97:in `__run_callbacks__'
activesupport (5.0.7) lib/active_support/callbacks.rb:750:in `_run_call_callbacks'
activesupport (5.0.7) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.7) lib/action_dispatch/middleware/callbacks.rb:36:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/debug_exceptions.rb:49:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.0.7) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.0.7) lib/rails/rack/logger.rb:26:in `call'
sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:13:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/request_id.rb:24:in `call'
rack (2.0.5) lib/rack/method_override.rb:22:in `call'
rack (2.0.5) lib/rack/runtime.rb:22:in `call'
activesupport (5.0.7) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/static.rb:136:in `call'
rack (2.0.5) lib/rack/sendfile.rb:111:in `call'
railties (5.0.7) lib/rails/engine.rb:522:in `call'
puma (3.7.1) lib/puma/configuration.rb:232:in `call'
puma (3.7.1) lib/puma/server.rb:578:in `handle_request'
puma (3.7.1) lib/puma/server.rb:415:in `process_client'
puma (3.7.1) lib/puma/server.rb:275:in `block in run'
puma (3.7.1) lib/puma/thread_pool.rb:120:in `block in spawn_thread'
[----] I, [2018-10-10T14:11:05.800290 #13280:2aeef94b9ebc]  INFO -- : Started GET "/physical_chassis/show_list" for ::1 at 2018-10-10 14:11:05 +0000
[----] D, [2018-10-10T14:11:05.830050 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 5, in use: 2, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:05.830386 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 5, in use: 3, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:05.830562 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 5, in use: 4, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:05.830751 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 5, in use: 5, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:15.831000 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkin, connection_pool: size: 5, connections: 5, in use: 4, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:15.831561 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkin, connection_pool: size: 5, connections: 5, in use: 3, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:15.832064 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkin, connection_pool: size: 5, connections: 5, in use: 2, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:15.832617 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkin, connection_pool: size: 5, connections: 5, in use: 1, waiting_in_queue: 0
[----] D, [2018-10-10T14:11:15.833444 #13280:2aeef94b9ebc] DEBUG -- : PostgreSQLAdapter#log_after_checkin, connection_pool: size: 5, connections: 5, in use: 0, waiting_in_queue: 0
I, [2018-10-10T14:11:17.390795 #13280]  INFO -- : Initializing websocket worker!                                                       
[----] F, [2018-10-10T14:11:19.809904 #13280:2aeef94b9ebc] FATAL -- :                                                                  
[----] F, [2018-10-10T14:11:19.810036 #13280:2aeef94b9ebc] FATAL -- : LoadError (Unable to autoload constant PhysicalChassisController, expected /home/himdel/manageiq-api/app/controllers/api/physical_chassis_controller.rb to define it):
[----] F, [2018-10-10T14:11:19.810105 #13280:2aeef94b9ebc] FATAL -- :   
[----] F, [2018-10-10T14:11:19.810244 #13280:2aeef94b9ebc] FATAL -- : activesupport (5.0.7) lib/active_support/dependencies.rb:513:in `load_missing_constant'
activesupport (5.0.7) lib/active_support/dependencies.rb:203:in `const_missing'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:268:in `const_get'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:268:in `block in constantize'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `each'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `inject'
activesupport (5.0.7) lib/active_support/inflector/methods.rb:266:in `constantize'
activesupport (5.0.7) lib/active_support/dependencies.rb:584:in `get'
activesupport (5.0.7) lib/active_support/dependencies.rb:615:in `constantize'
actionpack (5.0.7) lib/action_dispatch/http/request.rb:81:in `controller_class'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:44:in `controller'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:30:in `serve'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:39:in `block in serve'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:26:in `each'
actionpack (5.0.7) lib/action_dispatch/journey/router.rb:26:in `serve'
actionpack (5.0.7) lib/action_dispatch/routing/route_set.rb:727:in `call'
/home/himdel/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/manageiq-graphql-5f68621f2791/lib/manageiq/graphql/rest_api_proxy.rb:18:in `call'
secure_headers (3.0.3) lib/secure_headers/middleware.rb:10:in `call'
lib/request_started_on_middleware.rb:12:in `call'
rack (2.0.5) lib/rack/etag.rb:25:in `call'
rack (2.0.5) lib/rack/conditional_get.rb:25:in `call'
rack (2.0.5) lib/rack/head.rb:12:in `call'
rack (2.0.5) lib/rack/session/abstract/id.rb:232:in `context'
rack (2.0.5) lib/rack/session/abstract/id.rb:226:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/cookies.rb:613:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/callbacks.rb:38:in `block in call'
activesupport (5.0.7) lib/active_support/callbacks.rb:97:in `__run_callbacks__'
activesupport (5.0.7) lib/active_support/callbacks.rb:750:in `_run_call_callbacks'
activesupport (5.0.7) lib/active_support/callbacks.rb:90:in `run_callbacks'
actionpack (5.0.7) lib/action_dispatch/middleware/callbacks.rb:36:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/debug_exceptions.rb:49:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
railties (5.0.7) lib/rails/rack/logger.rb:36:in `call_app'
railties (5.0.7) lib/rails/rack/logger.rb:26:in `call'
sprockets-rails (3.2.1) lib/sprockets/rails/quiet_assets.rb:13:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/request_id.rb:24:in `call'
rack (2.0.5) lib/rack/method_override.rb:22:in `call'
rack (2.0.5) lib/rack/runtime.rb:22:in `call'
activesupport (5.0.7) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/executor.rb:12:in `call'
actionpack (5.0.7) lib/action_dispatch/middleware/static.rb:136:in `call'
rack (2.0.5) lib/rack/sendfile.rb:111:in `call'
railties (5.0.7) lib/rails/engine.rb:522:in `call'
puma (3.7.1) lib/puma/configuration.rb:232:in `call'
puma (3.7.1) lib/puma/server.rb:578:in `handle_request'
puma (3.7.1) lib/puma/server.rb:415:in `process_client'
puma (3.7.1) lib/puma/server.rb:275:in `block in run'
puma (3.7.1) lib/puma/thread_pool.rb:120:in `block in spawn_thread'

href slug refactoring

Spent some (probably too much) time attempting to refactor href slugs, but thought it was time for an issue around it...

Currently, the way of grabbing a resource's href_slug is via the virtual attribute which is essentially calling name for subclass. Name for subclass is problematic here, because it is only looking to see whether the resource class is equal to that of a collection class or a descendant of one - and doesn't do any more searching once it finds one that matches that criteria.

There are multiple scenarios (ie requests, policy_actions, vms) where the href slug is incorrect because the resource class is a descendant of a class specified by a different collection than the request that was made to access the resource. Which brings me to the question - what is the purpose of an href slug?

If the purpose is to be a shortened version of an href, then why do we go through all of this trouble with a virtual attribute that looks through the config? We can base it directly off of the request and build it similarly to the href.

If the purpose is to find the specific href, or collection that matches the resource's class exactly (ie when an Automation Request is returned from /requests, it should return an href slug with /automation_requests/:id), then it still needs to be refactored to look for exact matches and not break when it finds a descendant.

FWIW, QE thinks it should be consistent with the href

cc: @chrisarcand @imtayadeway @abellotti

404's are responded via HTML

If you request any random garbage path, the default "page you were looking for doesn't exist (404)" HTML is rendered back to the client. We should be returning a JSON response.

/providers auth validation in foreground

When adding/editing a provider, it's useful to know whether ManageIQ successfully connects to it.
In UI, you're forced to click [Validate] before you can even press [Add]/[Save].
In API, the create/edit returns immediately, and iff the Authentication changed, it's validated in background.
=> I propose switching to (or adding option to do) foreground validation, returning result immediately in API response.

Motivation: It's very hard to know if background happened, and what was the result using API. We did our best in https://github.com/dkorn/manageiq-ansible-module/blob/master/library/manageiq_provider.py with following pain points:

  1. It's slow! The validation itself is seconds (make a few simple network requests), but through the queue it can take a lot, we had to implement 1min timeout returning "added but not validated" answer... 🐌
  2. No task_id or anything to poll for is returned. On edit, the old Authenciation already had a valid/invalid status, so how do you know when you see new status? The only way we found to detect completion is save last_valid_on & last_invalid_on timestamps before edit and poll for either to change 🚧
  3. It's impossible to distinguish not validated yet vs not trying to validate. Given multiple endpoints & athentications, we had to compare desired/existing to predict which ones are going to be re-validated and poll only those... 🎲
    • Due to #27, we don't always know which ones changed.
    • It's sensitive to how ManageIQ decides what to validate. E.g. last I checked it's after_save hook on Authentication, and if you edit Endpoint port or hostname, it doesn't revalidate. That's a ManageIQ bug, but now it also manifests as API polling side mis-predicting what to poll for...

For now we're omitting auth validation from the first iteration going into upstream ansible ansible/ansible#28273. cc yaacov
UPDATE: it landed without validation and will remain that way. The kludges we attempted in dkorn/manageiq-ansible-module are too brittle, won't re-attempt them unless API is improved.

RUD for MiqTemplate (or just read & update)

Recently @andyvesel created API endpoints for editing Cloud Templates - create #337, read #130, update #341 and delete #328.

However, there is no concept of editing cloud templates in the UI, there is just a generic MiqTemplate editor (VmOrTemplate really).

For a generic MiqTemplate:

  • the only fields that can be currently edited in the UI are name and description
  • it can't be created, only updated

So, we need to have API endpoint/endpoints to edit both kinds, with the same interface (for the shared fields at least), as the one introduced in #341.

(Delete would fit logically together, but the UI is currently not using the API for deletion.)

https://bugzilla.redhat.com/show_bug.cgi?id=1487114

EDIT: Related to ManageIQ/manageiq-ui-classic#3944, see discussion there

Add CloudVolumeType to v2v API for OpenStack

When creating a TransformationMapping, the TransformationMappingItems look something like

{ source: '/api/data_stores/, destination: '/api/data_stores/ }

Assuming mapping creation remains the same in the OSP case, in order for us to be able to map from VMWare Storages to OSP CloudVolumeTypes, they would need to be added to the API, and have the href attribute added to the model

Enable bulk assignment of tags for users

I am now working on tag assignment React component for users in MIQ. You can edit multiple tags for multiple users at once. If i want to use API i have to create one request per user to edit their tags. That is not very pretty sollution.

Is it possible to add bulk assignment of tags to users (like for vms of services)?

Move provider attribute validation to models

A ton of validation is done on attributes for the creation of a provider within the Providers controller. This information is different from provider to provider, and should live within the provider itself so that it is easier to maintain and is more provider-specific.

For example, creating of an Azure provider would like to specify an azure_tenant_id, which is an alias for uid_ems. Because validation is done on the columns, it returns a BadRequestError. By moving the allowed attributes to the providers themselves, we can eliminate this problem and the other problems that exist when storing logic for many different models in an API controller.

Validation: https://github.com/ManageIQ/manageiq-api/blob/master/app/controllers/api/providers_controller.rb#L180

Allowed attributes:
https://github.com/ManageIQ/manageiq-api/blob/master/app/controllers/api/providers_controller.rb#L3-L10

Need the ability to add just created vms to a service

Proposed signature:

POST /api/services/:id

{
  "action" : "add_provider_vms",
  "resource" : {
     "provider" : { "href" : "http://localhost:3000/api/providers/:id" },
     "uid_ems" : [ ... ]
  }
}

provider reference should support

href
id
href_slug

Normal action response signature:

{
  "success": true,
  "message": "Service id:1000000000127 name:'test-service' add_provider_vms",
  "task_id": "1000000000022",
  "task_href": "http://localhost:3000/api/tasks/1000000000022",
  "href": "http://localhost:3000/api/services/1000000000127"
}

Associated Services and Providers are not returned from generic object instance endpoint

the json returned from https://{{manageiq_host}}/api/generic_objects/{{id}} does not contain any associated services or providers, even though the object has them.
Is there a separate endpoint that could provide the associated services and providers?

EX Post body:

{
	"associations": {
		"service": [{
			"href": "https://{{manageiq_host}}/api/services/1"
		}],
		"provider": [{
			"href": "https://{{manageiq_host}}/api/providers/1"
		}]
	},
	"generic_object_definition": {
		"href": "https://{{manageiq_host}}/api/generic_object_definitions/1"
	},
	"name": "my_custom_instance",
	"property_attributes": {
		"engine": "orcale",
		"username": "master",
		"region": "default",
		"type": "db",
		"size": 30
	}
}

EX response:

{
	"href": "https://{{manageiq_host}}/api/generic_objects/1",
	"id": "1",
	"name": "my_custom_instance",
	"uid": null,
	"generic_object_definition_id": "1",
	"created_at": "2018-05-08T10:30:20Z",
	"updated_at": "2018-05-08T10:30:20Z",
	"property_attributes": {
		"size": 30,
		"type": "db",
		"engine": "orcale",
		"region": "default",
		"username": "master"
	},
	"actions": [{
		"name": "edit",
		"method": "post",
		"href": "https://{{manageiq_host}}/api/generic_objects/1"
	}, {
		"name": "delete",
		"method": "post",
		"href": "https://{{manageiq_host}}/api/generic_objects/1"
	}, {
		"name": "delete",
		"method": "delete",
		"href": "https://{{manageiq_host}}/api/generic_objects/1"
	}]
}

n+1 query when requesting services

When I:

GET http://localhost:3000/api/services?expand=resources&limit=20

I see that we have an n+1 problem:

The +1:

D, [2017-10-25T15:02:26.932359 #22628] DEBUG -- :   Service Load (0.3ms)  SELECT  "services".* FROM "services" LIMIT $1 OFFSET $2  [["LIMIT", 20], ["OFFSET", 0]]

For every n services:

D, [2017-10-25T15:02:26.996742 #22628] DEBUG -- :   Service Inst Including Associations (64.1ms - 20rows)
D, [2017-10-25T15:02:29.779592 #22628] DEBUG -- :   ServiceTemplate Load (0.2ms)  SELECT  "service_templates".* FROM "service_templates" WHERE "service_templates"."id" = $1 LIMIT $2  [["id", 10000000000005], ["LIMIT", 1]]
D, [2017-10-25T15:02:29.811939 #22628] DEBUG -- :   ServiceTemplate Inst Including Associations (32.1ms - 1rows)
D, [2017-10-25T15:02:29.814565 #22628] DEBUG -- :   ResourceAction Load (0.5ms)  SELECT  "resource_actions".* FROM "resource_actions" WHERE "resource_actions"."resource_id" = $1 AND "resource_actions"."resource_type" = $2 AND "resource_actions"."action" = $3 LIMIT $4  [["resource_id", 10000000000005], ["resource_type", "ServiceTemplate"], ["action", "Reconfigure"], ["LIMIT", 1]]
D, [2017-10-25T15:02:29.814762 #22628] DEBUG -- :   ResourceAction Inst Including Associations (0.0ms - 0rows)
D, [2017-10-25T15:02:29.817743 #22628] DEBUG -- :   CustomButton Load (0.2ms)  SELECT "custom_buttons".* FROM "custom_buttons" WHERE "custom_buttons"."applies_to_class" = $1 AND "custom_buttons"."applies_to_id" IS NULL  [["applies_to_class", "Service"]]
D, [2017-10-25T15:02:29.817925 #22628] DEBUG -- :   CustomButton Inst Including Associations (0.0ms - 0rows)
D, [2017-10-25T15:02:29.818388 #22628] DEBUG -- :   CustomButton Load (0.1ms)  SELECT "custom_buttons".* FROM "custom_buttons" WHERE "custom_buttons"."applies_to_class" = $1 AND "custom_buttons"."applies_to_id" = $2  [["applies_to_class", "ServiceTemplate"], ["applies_to_id", 10000000000005]]
D, [2017-10-25T15:02:29.818559 #22628] DEBUG -- :   CustomButton Inst Including Associations (0.0ms - 0rows)
D, [2017-10-25T15:02:29.819506 #22628] DEBUG -- :   CACHE (0.0ms)  SELECT "custom_buttons".* FROM "custom_buttons" WHERE "custom_buttons"."applies_to_class" = $1 AND "custom_buttons"."applies_to_id" IS NULL  [["applies_to_class", "Service"]]
D, [2017-10-25T15:02:29.819644 #22628] DEBUG -- :   CustomButton Inst Including Associations (0.0ms - 0rows)
D, [2017-10-25T15:02:29.821337 #22628] DEBUG -- :   CustomButtonSet Load (0.4ms)  SELECT "miq_sets".* FROM "miq_sets" WHERE "miq_sets"."set_type" IN ('CustomButtonSet') AND "miq_sets"."owner_id" = $1 AND "miq_sets"."owner_type" = $2  [["owner_id", 10000000000005], ["owner_type", "ServiceTemplate"]]
D, [2017-10-25T15:02:29.821522 #22628] DEBUG -- :   CustomButtonSet Inst Including Associations (0.0ms - 0rows)

That's 6 round trips to the database for every service we render back. This is almost certainly the one of the root causes of any slow query times reported against the services API

Investigate performance issues with REST API

Note: This is reported by Loic

  • Logging is pretty slow, there is a top progress bar that is providing some information… We should look at some animation at the centre of the page to be more informative..
  • More I am using the UI faster it seems to be.. this is may be an impression…
  • Clicking on My Service is the slowest thing.. changing page in My Service is also Slow
  • Once the UI was completely stuck… I cannot click to anything for few seconds.. .I think the previous video capture this moment.
  • I looked at “time_taken” in the log, which seems to be what we need to look at .. I am not sure what is acceptable time… I see some some valid above 1 second (2-3), below 1 seconds and below 0,1 seconds..
  • My impression by watching logs at the same time I am waiting is the UI is: there are multiple API Calls for some requests and then some processing on the UI side.

Token based access still does a lot of authorization in API

When you access the rest API with a token, it still does a LOT of authorization. It only saves 2 or 3 queries over logging in every time (in particular it saves inserting into the audit log), but you can see it re-access the user/group/tenant/product features/and entitlements over and over.

I think that on getting a token we should cache the users entitlements / product features, and reuse that cache on subsequent access. When the token is invalidated, then the user will be required again and it can be refetched. I'm not sure how well RBAC can reuse this cache, but hopefully we can do this.

If there are any changes to the user/group/tenant/product features/and entitlements, then we can invalidate their token forcing them to fetch a new one, and thus solving the caching problem very simply.


This issue was moved to this repository from ManageIQ/manageiq#11637, originally opened by @Fryguy

[BUG] Service_requests API is not considering same user from remote regions (like manageiq-ui-classic does)

As far as I know, in a ManageIQ with global and remote regions, there are more than one user in users table that represents the same user (one for each region), as you can see below:

vmdb_production=# SELECT id, name, email, userid, current_group_id FROM public.users WHERE name = 'SPMIQ01';
id | name | email | userid | current_group_id
----------------+---------+-------+-------------------------------+------------------
2000000000005 | SPMIQ01 | | [email protected] | 2000000000036
99000000000007 | SPMIQ01 | | [email protected] | 99000000000019
1000000000005 | SPMIQ01 | | [email protected] | 1000000000039
(3 rows)

I noticed that the service_requests api is not considering this and it is always querying for service requests from the logged user in global (99000000000007 in my case). The point here is that the owner of the service requests in fact is the user from the remote region, as you can see below:

SELECT id, description, type, requester_id, requester_name, userid
FROM miq_requests where
requester_name = 'SPMIQ01' AND requester_id = '99000000000007';
id | description | type | requester_id | requester_name | userid

(0 rows)

SELECT id, description, type, requester_id, requester_name, userid
FROM miq_requests where
requester_name = 'SPMIQ01' AND requester_id in ('2000000000005', '1000000000005');
id | description | type | requester_id | requester_name |
userid

1000000000326 | Provisioning Service [Red Hat 7.2] from [Red Hat 7.2] | ServiceTemplateProvisionRequest | 1000000000005 | SPMIQ01 | [email protected]
1000000000327 | Provisioning Service [Red Hat 7.2] from [Red Hat 7.2] | ServiceTemplateProvisionRequest | 1000000000005 | SPMIQ01 | [email protected]
(...)

I found the following in the manageiq-api code, that, in my perspective, should be fixed to consider other users from remote regions [that, in fact, is related to the same logged user]:

From manageiq-api - miq_request_controller.rb:

    def find_service_requests(id)
      klass = collection_class(:service_requests)
      return klass.find(id) if User.current_user.admin_user?
      klass.find_by!(:requester => User.current_user, :id => id)
    end

    def service_requests_search_conditions
      return {} if User.current_user.admin_user?
      {:requester => User.current_user}
    end

From manageiq ui the user is able to see all service requests. As far as I understood this is because of the following code, that considers all users from global and remote [that, in fact, is related to the same logged user]:

From manageiq-ui-classic - miq_request_controller.rb:

  def remote_and_global_requestors(requestor)
    condition = []
    requestors = User.where(:userid => requestor.try(:userid))
    if requestors.count > 1
      condition.push("or" => requestors.collect { |user| {"=" => {"value" => user.id, "field" => "MiqRequest-requester_id"}} })
    else
      condition.push("=" => {"value" => requestor.try(:id), "field" => "MiqRequest-requester_id"})
    end
  end

That said, I believe manageiq-api should have the same behavior as manageiq-ui-classic and should return the service requests for the logged user + users in the regions that represents this same user.

[BUG] Adding attribute 'description' limits fields returned to only those in attributes but doesn't include 'description'

Steps to reproduce

Send HTTP command (Postman)

GET https://server.domain.tld/api/vms?expand=resources&attributes=operating_system.product_name,description

Response

{
    "name": "vms",
    "count": 2,
    "subcount": 2,
    "resources": [
        {
            "href": "https://server.domain.tld/api/vms/100000000000208",
            "id": 100000000000208,
            "operating_system": {
                "product_name": "Red Hat Enterprise Linux Server release 7.4 (Maipo)"
            }
        },
        {
            "href": "https://server.domain.tld/api/vms/100000000000154",
            "id": 100000000000154,
            "operating_system": {
                "product_name": "SUSE Linux Enterprise Server 11 (x86_64)\nVERSION = 11\nPATCHLEVEL = 3"
            }
        },
    ]
}

As you can see no "description" is returned in the objects"

If you remove the description field the full VM object is returned with the additional attributes specified in the query.

Edit

Appears that if i add any attribute present on the "base" object then it outputs just the fields specified in attributes (example: vendor, guid, etc). This is exactly what i'm after in #242 . Is this desired behavior, and can i count on it in a future release?

Still, description is not returned when limited in this way.

Find a better way to test posts with query params

There's an open issue for this in rake-test: rack/rack-test#150. However, I don't see this moving ahead soon.

Alternatively, it may be better to respond to this test pain by figuring out a better way to implement this.

Links:

# TODO: provider_class in params, when supported (https://github.com/brynary/rack-test/issues/150)
post(api_providers_url + '?provider_class=provider', :params => gen_request(:create, sample_foreman))

# TODO: provider_class in params, when supported (https://github.com/brynary/rack-test/issues/150)
api_provider_custom_attributes_url(nil, generic_provider) + '?provider_class=provider'

GET api?attributes=authorization should return more than an array of strings for `groups`

While mildly helpful, the array of strings, better would be an array of full group objects that below to the user when this call is made:GET http://localhost:3000/api?attributes=authorization

  "identity": {
    "userid": "admin",
    "name": "Administrator",
    "user_href": "http://localhost:3000/api/users/10000000000001",
    "group": "EvmGroup-super_administrator",
    "group_href": "http://localhost:3000/api/groups/10000000000002",
    "role": "EvmRole-super_administrator",
    "role_href": "http://localhost:3000/api/roles/10000000000001",
    "tenant": "Red Hat",
    "groups": [
      "Tenant My Company access",
      "EvmGroup-super_administrator"
    ]
  }

@imtayadeway 👾 💅 ‼️

Routes don't utilize Rails::Engine mounting

In the quick repo split, we just threw the routes from the core repo as-is in to the routes file here, where we just call Rails.application.routes directly.

One of the many benefits of Rails::Engine is to be able to isolate namespaces and mount separate routers. We should reorganize the router here to use ManageIQ::Api.routes and add it as a separate router in the core repo. This would enable us, off the top of my head, to:

  • Avoid routing conflicts across repos (which thankfully hasn't happened yet)
  • Control the mount priority (who's getting routed first? A provider? The UI? Who knows? You can't tell as it's based purely on who's loaded first right now)
  • Isolate namespaces, localizing path helpers (amongst other things I'm not remembering I'm sure). For example, the API doesn't need to prefix every single one of it's _url and _path helpers with api_ as the engine can just handle that.

These Engine routing concerns, as well as other Engine things I'm sure I'll create separate issues for, are things that are applicable to all of the provider plugins and the UI as well. cc/ @Fryguy

Fill out README

Doesn't give a description, says it's on RubyGems even though it's not, etc. Should update all this to make things clear to project newcomers.

/providers read/write REST API is very asymmetric

Continuing from #9743 but I want to focus on REST API, then worry about internals.
http://manageiq.org/docs/reference/latest/api/reference/providers
The ways to write provider endpoint & auth info is very different from the ways to read them — there is almost no overlap. Especially for multi-endpoint providers (this part BTW is not documented yet)

GET /api/providers/1?attributes=endpoints,authentications,credentials,hostname,port returns:

  ...
  "name": "63",
  "hostname": "vm-48-63.eng.lab.tlv.redhat.com",
  "port": 8443,
  "endpoints": [
    {
      "role": "default",
      "hostname": "vm-48-63.eng.lab.tlv.redhat.com",
      "port": 8443,
      ...
    },
    {
      "role": "hawkular",
      "hostname": "vm-48-63.eng.lab.tlv.redhat.com",
      "port": 443,
      ...
    }
  ],
  "authentications": [
    {
      "name": "ManageIQ::Providers::Openshift::ContainerManager 63",
      "authtype": "hawkular",
      "type": "AuthToken",
      "status": "Valid",
      ...   # excludes "auth_key", "password" for security
    },
    {
      "authtype": "bearer",
      "type": "AuthToken",
      ...
    }
  ...

POST allows several combos:

  • Single endpoint as top-level hostname, port, ipaddress, single credentials:
  "hostname" : "my_vcenter_50",
  "ipaddress" : "192.168.150.1",
  "credentials" :  {
    "userid" : "vcenter_admin",
    "password" : "vcenter_password"
  }
  ...
  • Single endpoint but compound credentials:
   ...
   "credentials" : [
     {
       "userid"    : "default_userid",
       "password"  : "default_password"
     },
     {
       "userid"    : "metrics_userid",
       "password"  : "metrics_password",
       "auth_type" : "metrics"
     }
  ]
  • Compound endpoints paired with auths in connection_configurations:
   ...
   connection_configurations: [
     {
       endpoint: {
         role: "default",
         hostname: "sample_containers_multi_end_point.provider.com",
         port: 18443
       },
       authentication: {
         role: "bearer",
         auth_key: "good token"
       }
     },
     {
       endpoint: {
         role: "hawkular",
         hostname: "sample_containers_multi_end_point.provider.com",
         port: 443
       },
       authentication: {
         role: "hawkular",
         auth_key: "good token"
       }
     }
   ]

Spot the Differences

  • You can write credentials but not read it.
    • If multiple, distinguished by auth_type.
  • You can read authentications but not write it.
    • Distinguished by authtype (no undescore!)
  • You can write {endpoint, authentication} pairs, but not read them.
    • You still have to supply role for each endpoint and role (not auth[_]type!) for each auth.
      • The default endpoint role sometimes expects different auth role eg. bearer above.
        (According to default_authentication_type method.)
    • The pairs structure you supply is I think ignored, they'll actually be matched by role.

P.S. I think only a few of the Authentication columns can be written, many are silently ignored. Most are not relevant to providers, but should either accept or give error. Anyway that's easy.


Where should we go from here?
I'll add some proposals later, but would love to hear opinions.
A central question this might hinge on: Does 1:1 endpoint:auth pairing suit all providers?

cc @abellotti @durandom @yaacov @dkorn
@miq-bot add-label providers, api


This issue was moved to this repository from ManageIQ/manageiq#13454, originally opened by @cben

When no current_user defined exception raised

When logging in for the first time, there is no User.current_user set and the log_request line blows up with the exception
#<NoMethodError: undefined method `userid' for nil:NilClass>

Which leads to the render being called multiple times

rescue_from(NoMethodError) { |e| api_error(:internal_server_error, e) }

F, [2017-08-09T14:23:54.647550 #4476] FATAL -- : AbstractController::DoubleRenderError (Render and/or redirect were called multiple times in this action. Please note that you may only call render OR redirect, and at most once per action. Also note that neither redirect nor render terminate execution of the action, so if you want to exit an action after redirecting, you need to do something like "redirect_to(...) and return".):

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.