Comments (11)
Doing such a schema change could also fix this ugly problem which is plugins selection "fixed" by bb83890
from kong.
Credentials
I am not happy by the way we handle credentials and I think we should think more carefully how we want to support this. I think having something like the following schema could make sense:
We do have the following generic core entities:
accounts
, the base entity who owns of zero or moreapplications
.applications
, generic credential holder, that can hold different credential types
Each authentication plugin can create a credential type on the datastore, like:
query
basic
header
ldap
oauth
This means that we need to introduce the possibility for plugins to edit the datastore during their installation using a DSL.
Plugins DB DSL
If plugins can modify the datastore, that should be done with a DSL instead of plain simple SQL, to avoid doing illegal operations on the datastore. An example could be:
return { create = {
{
type = "table",
name = "ldap",
properties = [[
id uuid,
key text,
created_at timestamp,
PRIMARY KEY (id)
]]
},
{
type = "datatype",
name = "ldap_credential",
properties = [[
public text,
secret text
]]
}
}
}
By having the DSL we're limiting the number of operations that the plugin can execute on the datastore, like deleting other tables, or modifying existing data.
We could also implement a rollback function to execute DELETE
statements on whatever datastore entity has been created during the provisioning of the plugin (it needs to be explicit because it could cause loss of data).
from kong.
This is even nicer, the action to create
those entities could be implicit, and we don't accept DB-specific constructs to support in the future any other datastore.
{
entities = {
{
name = "ldap",
properties = {
{ name = "id", type = "id" },
{ name = "key", type = "string" },
{ name = "created_at", type = "timestamp" },
{ name = "type", type = "ldap_credential"}
{ primary = "id"}
}
},
{
name = "ldap_credential",
properties = {
{ name = "public", type = "string", unique = true },
{ name = "secret", type = "string" }
}
}
}
}
The example above is a quick demonstration, but a DSL like this could technically be ported to any datastore without having to update the plugins if a new DAO is being introduced. The DAO will take care of translating the DSL to an executable statement.
And as long as the DSL is verbose enough, the DAO can then decide to handle edge-cases (like treating child entities like ldap_credential
as datatypes
in Cassandra, or just another table
in other datastores - it's up to the DAO).
from kong.
On a side note the more I look at the DSL above, the more it resembles the schemas
we already have. If we decide to implement a DSL to DB translation, I wonder if we can automatically generate the migration script file by parsing all of the schemas, thus automating the creation of migrations files.
from kong.
Relations
- Edited original comment ^ to raise the question of paginating a
set
, ie: applications of an accounts.
Credentials
- Each credential could be a UDT for Cassandra. If we have a DSL, we need to make sure we can handle this for other DBs.
DSL
I like the idea, just time consuming for DAOs to implement. Otherwise:
- Could even skip the migration file creation. Migration files could be DAO agnostic and simply DSL files executed on the go.
- If we are talking migrations, the DSL also needs to be able to ALTER or DROP, and that does not protect the DB against malicious plugins either as we discussed before.
- What does protect against that is a plugins-only DSL, that just offers the possibility to create UDTs for the
plugins
table. - But at the end of the day, any Lua code executed under a Kong instance can simply require the factory and call the
drop
method @thefosk. It's up to us to distribute valid, trusted plugins. - Official, trusted plugins could be released and signed by PGP.
- And our DSL could just do anything it wants, since an official plugin is trusted.
from kong.
Regarding the DSL, a few points:
- We could automatically prepend the plugin name in front of any entity that is being created, like
basicauth.keys
. This is a good idea for two reasons:- Avoid name clashes if two plugins want to use the same table name.
- Make sure that a plugin can't change anything that doesn't belong to itself.
- There may be a way to limit the scope of
drop
or other reserved calls just to some special packages. Thus we can block every call that comes from thekong.plugins
package. - We should still sign plugins and avoid running those that are not authorized.
from kong.
Too bad we're not using PostgreSQL: http://leafo.net/lapis/reference/database.html#database-schemas
A great contribution would be implementing a Cassandra adapter to Lapis as mentioned in #80.
from kong.
Cassandra 3.0 will support this: https://issues.apache.org/jira/browse/CASSANDRA-8473
from kong.
Following the discussion we had yesterday, here are the decisions we took:
accounts:
renamed toconsumers
.id
: sameprovider_id
: renamed tocustom_id
(same purpose) required if nousername
username
: required if noprovider_id
extra
: Maybe a field for extra informations
apis
: nothing newplugins
:- They can plug themselves into the lifecycle of a request (this hasn't changed)
- They can access the
consumers
table - They can expand the DB to add tables and perform additional queries of their own
- They can expand the API routes
- Once installed, one can create configuration(s) (said
configuration entry
of that plugin, linked to anapi
and optionally, aconsumer
. This allows a plugin to be enabled on an API, as well as being overridable for a specificconsumer
.
We started talking about having a whitelist/blacklist for configuration entries (to be able to enable/disable a configuration entry for a lot of apis
/consumers
at once, but this ran into implementation issues as illustrated in the following picture.
This discussion was related to #50 (Plugins system), #91 (refactor applications), #93 (Plugins API), #98 (Better API routing)
Here is a pic of the whiteboard:
from kong.
Improvements described in the previous comment are implemented, appart from:
- plugins expand the DB to add tables and perform additional queries of their own
- plugins expand the API routes
Those things need to be done in order to provide a good development environment for plugins but will be part of another discussion: #93.
from kong.
Plugins do expand the API routes, it has been implemented: https://github.com/Mashape/kong/blob/master/kong/api/app.lua#L76
We are waiting for the DAO part to have complete separation.
from kong.
Related Issues (20)
- Error running kong migration list HOT 2
- ACME Plugin in kong with aws-ec2 error message "The ACME server refuses to issue a certificate for this domain name, because it is forbidden by policy" HOT 5
- DNS Resolution failed - socket busy HOT 8
- Python Plugin Server crashes during execution HOT 2
- key-auth vs basic-auth plugin behaviour HOT 1
- [Plugin Server] RPC call has unexpected type <nil> HOT 1
- Kong 3.6: resty.openssl.auxiliary.nginx doesn't support Nginx version 1025003 HOT 5
- Kong manager is broken after upgrade to 3.4.3.4 HOT 4
- Reading issues with array elements in configuration files in DB less mode HOT 6
- request-size-limiting http2 requests are not supported without content-length header HOT 5
- Plugin schema not found on Control Plane HOT 1
- Log message does not match plugin HOT 5
- failed fetching KongUpstreamPolicy after upgrade to Kong 3.6 HOT 8
- Zipkin plugin: Support for Datadog tracing headers
- fail use jwt_parser:base64_decode to decode when upgrade kong 3.4.2 to 3.6.0 HOT 5
- ai-proxy buffers streamed responses HOT 5
- Kong Community - JWT Authentication Based Route 401 HOT 3
- Buggy behavior after failed health check recover HOT 14
- Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "". Strict MIME type checking is enforced for module scripts per HTML spec. HOT 3
- Can't start postgresql 16 over tls HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kong.