maif / otoroshi Goto Github PK

before 1st April

https://auth0.com/docs/libraries/lock/v11

Should be Otoroshi, make it configurable

each service should be able to define a redirection URL

ExposedSubdomain is misspelled everywhere as exposedDubdomain

env.eventsName always sent ...

Select "Priv Apps Sessions" in the Select topbar -> http://.../bo/dashboard/sessions

Not found

• DNS
• redirect to dev
• local redirection

react-table support server side filtering and we should leverage that

If I try first to be log with a fake ID -> "Something is wrong ..." GOOD
I retry with a good ID -> "Something is wrong ... " BAAD

As Otoroshi now supports HTTP/2, we should be able to proxy gRPC calls. That would be a great feature for Otoroshi

Linked akka-http issues

• akka/akka-http@c05d47f
• akka/akka-http#1843
• akka/akka-http#530
• akka/akka-http#1857
• akka/akka-http#1869

Proposal

Can you consider adding an option to choose a different https port for the admin API & UI ?

Benefit

We will be able to bloc traffic from internet on this port with a firewall and open it only for internals IPs and enforce the security.

it will be

• more lightweight
• more embeddable
• more independant from Play changes
• more 'close to the metal'

downside is

• we need to rewrite almost everything

prototype at https://github.com/mathieuancelin/otoroshi-akka-http or https://github.com/mathieuancelin/heimdallr

Otoroshi should verify Origin or Referer headers (if available) in BackOfficeAction to validate that the request actually comes from the BackOffice

after #64

• define custom signing key per service
• support RSA signing
  • with custom definition of public/private key per service
• if not define, use the master key as of today

• finish mvp doc
• add gitter.im page
• add travis build
• add bintray for libraries
• publish otoroshi images on docker repository (bintray)
• write the otoroshi landing page
• rewrite build on CC
• push otoroshi code
• push cli code
• push connectors code
• create a root readme
• create a root/otoroshi readme
• add badges on root readme
• add badges on root manual index
• change jar URL on root manual index
• change jar URL on manual quickstart
• change urls in manual get otoroshi
• shutdown tryout
• write code of conduct
• write contribution guide
• write issue template
• write PR template
• push snapshots somewhere
• close old issues
• provide docker tryout

shields at https://shields.io/

When no TTL, 0L is returned in

• LevelDB
• InMemory
• Cassandra

it should be -1L

Storage key should start with otoroshi

Right now several remote assets are used

• https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css
• https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/fonts/fontawesome-webfont.woff2?v=4.7.0
• https://fonts.gstatic.com/s/raleway/v12/CcKI4k9un7TZVWzRVT-T8wzyDMXhdD8sAj6OAJTFsBI.woff2
• https://fonts.googleapis.com/css?family=Raleway:400,500
• https://fonts.googleapis.com/css?family=Roboto

Send headers like

OtoroshiRequestId       
OtoroshiProxyLatency    
OtoroshiUpstreamLatency 

when enabled

This issue will gather all performances improvements for Otoroshi

https://github.com/MAIFX/opun-otoroshi/issues/139

Should we

• support sending events to elastic using standard elastic api
• improve the elastic connector

We should update play to last version.

But using Akka http as backend could introduce some regressions

• move to play 2.6
• use scala 2.12
• update scala dependencies
• fix performance issues
• merge PR
• update documentation about change config keys, etc ...

a.k.a stop instanciating ActorSystem everywhere

• ProxyActorSystem
• DataStoreActorSystem
• EverythingElseActorSystem

we should support HTTP/2.0 as it comes with Play 2.6 (with the play-akka-http2-support module).

https://www.playframework.com/documentation/2.6.x/AkkaHttpServer

• akka/akka-http#1849

Todo

• add the http2 plugin
• disable http2 by default
• add documentation about it
• documentation about serving https (needed for http2)

need to support :

/api/v1/events/:type/_count
/api/v1/events/:type/:field/_sum
/api/v1/events/:type/:field/_avg
/api/v1/events/:type/:field/_piechart
/api/v1/events/:type/:field/_histogram/stats
/api/v1/events/:type/:field/_histogram/percentiles

restart it every 2 hours or something

Canary info are always sent right now :(

• should be passed in Authorization: Basic header
• should be passed in Otoroshi-Authorization: Basic header

Use an atomic reference to hold a Scala data structure

support all kind of authentication modules

• basic auth
• oauth
• etc ...

We got these kind of error logs recently on our Otoroshi instance :

[error] otoroshi-analytics-actor - SEND_TO_ANALYTICS_ERROR: analytics actor error : Failure(java.lang.IllegalStateException: Stream is terminated. SourceQueue is detached)

Note: we did not activate Analytics.

After a restart, everything seems ok.

Add flag (ENV var + static config) to avoid exposition of admin dashboard and admin API on one specific Otoroshi instance. Another instance will be in charge of handling admin stuff.

Linked to #47

What does the blue button do ? The label says "open this group in a new window" but

• it's always in the same "All services" page, without any filters.
• i'ts not in a new window

Right know, throttling is computed using a time window of 10s. The value should be statically customizable. (cc. @sebprunierserli)

The goal here is to provide a mode where an Otoroshi instance (with a redis or cassandra backend) is the master (that does not handle traffic) and send all its internal state changes to a Kafka topic.

Other Otoroshi instances, the workers (with an in memory storage) will be connected to the same kafka topic. At statup, a worker will ask the state of the master, then will receive the flow of masters internal state changes.

This mode will be a good way to scale easily Otoroshi while providing great performance an in memory backed Otoroshi instance introduce almost no overhead

Tasks

• Remote Worker config
  • Kafka config
  • Kafka topic
  • master / worker
• Expose remote worker config in UI
• Expose remote worker config in Swagger
• Expose remote worker config in CLI
• For master instance
  • listen to messages asking for full internal state
  • listen to messages like alerts to broadcast them ?
  • send commands (from the ApiController maybe ?) to the kafka topic
• For worker instance
  • asking for full internal state at startup
  • sending message to say instance is ready
  • listen to command messages and trigger Api according to it
    • do not mutate workers admin api service from master
    • do not mutate workers admin users from master
    • do not mutate workers admin sessions from master
    • do not mutate workers private sessions from master
    • do not mutate remote worker config when global config changes from master
    • do not mutate auth0 stuff when global config changes from master
    • do not mutate clevercloud stuff when global config changes from master

• JWT apikey can be passed in Authorization: Bearer header
• JWT apikey can be passed in Otoroshi-Authorization: Bearer header
• apikey clientId should be passed using standard iss field instead of custom clientId
• original JWT token should be passed to target if in Authorization header

Sometimes, a 'No ApiKey provided' is returned by Otoroshi even if an actual valid ApiKey is provided with good headers

We got this error today during a few minutes :

[error] otoroshi-error-handler - Server Error Clock is running backward. Sorry :-( on /v1/infos/categories
2018-01-16T16:31:51.197+01:00java.lang.RuntimeException: Clock is running backward. Sorry :-(
2018-01-16T16:31:51.197+01:00at security.IdGenerator$.nextId(generators.scala:27)
2018-01-16T16:31:51.197+01:00at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:346)
2018-01-16T16:31:51.197+01:00at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:345)
2018-01-16T16:31:51.198+01:00at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:99)
2018-01-16T16:31:51.198+01:00at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:346)
2018-01-16T16:31:51.198+01:00at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:345)
2018-01-16T16:31:51.198+01:00at security.IdGenerator.nextId(generators.scala:7)
2018-01-16T16:31:51.198+01:00at gateway.Errors$.craftResponseResult(errors.scala:25)
2018-01-16T16:31:51.198+01:00at gateway.ErrorHandler.onServerError(handlers.scala:56)
2018-01-16T16:31:51.198+01:00at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:100)
2018-01-16T16:31:51.199+01:00at security.IdGenerator$.nextId(generators.scala:27)
2018-01-16T16:31:51.199+01:00at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
2018-01-16T16:31:51.199+01:00at security.IdGenerator.nextId(generators.scala:7)
2018-01-16T16:31:51.199+01:00at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:70)
2018-01-16T16:31:51.199+01:00at gateway.Errors$.craftResponseResult(errors.scala:25)

Our config :

• Otoroshi deployed on Clever Cloud
• 2 M instances

When you wan't to add a service there is a default value in "exposed domain" field (https://myservice.foo.bar)

It's impossible to delete the entire value due to the pattern check, there is always at least one letter left.
(example : https://m)

Of course you can past value in this field or use cunning to change the remaining letter but it's a little weird :)

First steps has been pushed in 99dea2e