maples7 / kong-config-manager Goto Github PK

We have a lot of routes created on kong, so kong paginates it using offset, when I try to dump the conf from this kong, I receive this error

[INFO] Ready to dump configs of kong instance main... [ERROR] Error: RequestError: Error: Invalid URI "/routes?offset=WyI5ZTdkZDQ5MS1iYTljLTRiMWYtODk1Yy1hMDk3YTQ2NWNiZmMiXQ" at new RequestError (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/errors.js:14:15) at Request.plumbing.callback (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/plumbing.js:87:29) at Request.RP$callback [as _callback] (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/plumbing.js:46:31) at self.callback (/usr/lib/node_modules/kong-config-manager/node_modules/request/request.js:185:22) at Request.emit (events.js:189:13) at Request.init (/usr/lib/node_modules/kong-config-manager/node_modules/request/request.js:273:17) at Request.RP$initInterceptor [as init] (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/configure/request2.js:45:29) at new Request (/usr/lib/node_modules/kong-config-manager/node_modules/request/request.js:127:8) at request (/usr/lib/node_modules/kong-config-manager/node_modules/request/index.js:53:10) at dumpItems (/usr/lib/node_modules/kong-config-manager/lib/dump.js:20:12) at rp.then.body (/usr/lib/node_modules/kong-config-manager/lib/dump.js:32:18) From previous event: at getObjects.then.OBJECTS (/usr/lib/node_modules/kong-config-manager/lib/dump.js:87:20) at runCallback (timers.js:705:18) at tryOnImmediate (timers.js:676:5) at processImmediate (timers.js:658:5)

for what I have looked into, not sure why, but when kcm tries to follow the pagination, the URI it gets is incomplete, it only has the path, so it falls into an if on kong-config-manager/node_modules/request/request.js line 260 and returns the error

Hi,

I have dumped the content of my kong-admin, but I cannot find the keys and credentials, is there a way to dump those?

Best

Hi,

I have services, routes and one plugin in my config. One plugin attached to the service.

When I execute kcm apply commend against newly bootstrapped kong-database, I'm getting the following error

Request for POST - http://kong:8001/plugins gets response StatusCodeError: 400 - {"message":"the foreign key '{id="53b57092-4c99-436b-a3cd-5dc33f592d67"}' does not reference an existing 'services' entity.","name":"foreign key violation","fields":{"service":{"id":"53b57092-4c99-436b-a3cd-5dc33f592d67"}},"code":4} but NOT 2xx

I believe plugins are being inserted before the services, if I execute the same command without any change, it works perfectly.

Is there any way to change this order?

Thank you in advance!

Hello, thank you so much for making this great repo. I am having trouble finding out how to create the JSON file to create a new consumer. An example of how you write the JSON would be awesome. Thank you!

I want to use KCM in my CI pipeline.

To do "apply", I need to run "kcm apply" in a non-git directory (my deployment is just a checkout without a .git in the directory inside due a security good-practice).

When I run KCM without the .git directory I get:

Not a git repository
To compare two paths outside a working tree:
usage: git diff [--no-index] <path> <path>

[ERROR] dir ./main does NOT exist for instance main

Additionally, if git is required to do "apply", my docker image needs git as additional dependency.

I have currently tracked my KONG configurations inside a large git project, but I don't want to mix this project with a "sub-repository" just for KONG config.

Kong version is 1.0.3 - dump works perfectly.
In my current setup I have only services and routes. Then I installed a brand new Kong 1.0.3 with empty DB.

When I run

kcm apply

I get warnings like this:

[WARNING] Request for POST - http://localhost:8001/routes/ gets response StatusCodeError: 400 - {"message":"the foreign key '{id=\"50378d08-0238-4bd9-980b-ad819ea6ff08\"}' does not reference an existing 'services' entity.","name":"foreign key violation","fields":{"service":{"id":"50378d08-0238-4bd9-980b-ad819ea6ff08"}},"code":4} but NOT 2xx
[WARNING] Request for POST - http://localhost:8001/routes/ gets response StatusCodeError: 400 - {"message":"the foreign key '{id=\"c1b79ab9-2227-47f4-b56f-e51e7a026e35\"}' does not reference an existing 'services' entity.","name":"foreign key violation","fields":{"service":{"id":"c1b79ab9-2227-47f4-b56f-e51e7a026e35"}},"code":4} but NOT 2xx

And the reason is quite clear: it tries to create routes before services were created and it can't match IDs. As I could see after apply it does same function like dump - it gets current setup and as result it deletes all routes that were not created, so running kcm apply again doesn't help. I have to stash git changes and only after that I get things working.

I guess the right approach is to run services requests first, then routes after it's done, then plugins.

@Maples7 - this tool is really nice, hope you can improve it =)

Hi, would it be possible to ignore SSL errors, such as this one (Error: self signed certificate), here is my config:

$ cat kcm-config.json
{
  "main": "https://ironic-hyena-kong-admin:8444"
}

And the error

$ DEBUG=kcm:* kcm dump
  kcm:make_program Start to make a commander instance... +0ms
  kcm:make_program Finished! +15ms
[ERROR] Error: RequestError: Error: self signed certificate
    at new RequestError (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/errors.js:14:15)
    at Request.plumbing.callback (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/plumbing.js:87:29)
    at Request.RP$callback [as _callback] (/usr/lib/node_modules/kong-config-manager/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at self.callback (/usr/lib/node_modules/kong-config-manager/node_modules/request/request.js:185:22)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
    at Request.onRequestError (/usr/lib/node_modules/kong-config-manager/node_modules/request/request.js:881:8)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:188:7)
    at TLSSocket.socketErrorListener (_http_client.js:324:9)
    at emitOne (events.js:96:13)
    at TLSSocket.emit (events.js:188:7)
    at emitErrorNT (net.js:1290:8)
    at _combinedTickCallback (internal/process/next_tick.js:80:11)
    at process._tickCallback (internal/process/next_tick.js:104:9)

Hi,

I made a docker image here:

https://github.com/zoobab/kong-config-manager/blob/master/README.md
https://github.com/zoobab/kong-config-manager/blob/master/Dockerfile

Example of a run:

$ mkdir data
$ docker run -v $PWD/data:/data --net=host zoobab/kong-config-manager dump --host https://nonplussed-kangaroo-kong-admin:8444
[INFO] Ready to dump configs of kong instance main...
[INFO] All Finished!
$ ./data/
./data/main
./data/main/services
./data/main/services/076a6890-3c7e-4605-9215-c014bd33b898.json
./data/main/services/555c5c7f-bbc4-4645-b358-fa2d0d85a8ee.json
./data/main/services/64afe15f-04f6-4df6-ab4e-cccf2d85f234.json
./data/main/services/908f0691-c0d1-437d-96ab-32a6fbe38a47.json
./data/main/routes
./data/main/routes/418707fd-c53d-4d4c-a3ff-7c7dadc639ed.json
./data/main/routes/78d7f5e2-5780-47c8-acd1-a2634cc1fe16.json
./data/main/routes/84aa1323-70db-485e-b721-592c527036c4.json
./data/main/routes/fcea97cd-c65e-42ec-9ffd-484c9ab84d40.json
./data/main/consumers
./data/main/consumers/10b6349f-fedd-430c-9acb-192ffc56ba54.json
./data/main/consumers/601f5841-ffa5-4867-9699-d54bd66bbde9.json
./data/main/consumers/8deea650-5589-40f9-a0b9-ed9de2520234.json
./data/main/consumers/95dd2488-0a03-417b-973f-37abe46167cc.json
./data/main/consumers/ad806c21-6521-4191-968d-d21351a6f5fc.json
./data/main/plugins
./data/main/plugins/12e4a336-b048-4dab-964a-6b73f31b10b9.json
./data/main/plugins/6df9acc7-e580-4378-a7a1-7248aa27ae19.json
./data/main/plugins/70b4d593-fbae-40c2-84e3-25430c0326b0.json
./data/main/plugins/78906955-a8bc-4ff0-81a3-6aae880f4cd4.json
./data/main/plugins/7f8e553c-0441-4db8-a545-83d73e1e68dc.json
./data/main/plugins/80f368ed-7c77-4139-ae22-621296ff13af.json
./data/main/plugins/97d49676-e4af-4596-b845-c3c9a097450b.json
./data/main/plugins/9c91fa2a-c379-4cc2-afc9-dbc3d08c33f7.json
./data/main/certificates
./data/main/snis
./data/main/upstreams

Should I submit a PR?