openflagr / flagr Goto Github PK
View Code? Open in Web Editor NEWFlagr is a feature flagging, A/B testing and dynamic configuration microservice
Home Page: https://openflagr.github.io/flagr
License: Apache License 2.0
Flagr is a feature flagging, A/B testing and dynamic configuration microservice
Home Page: https://openflagr.github.io/flagr
License: Apache License 2.0
Calling the GET /flags endpoint would return a collection of segments and variants related to each flag
GET /flags endpoints returns empty arrays for segments and variants regardless of whether or not they're present
If there's concern about the size of the response, could potentially have a summary flag as a query parameter that returns a list of flags without all of their child component details
I'm attempting to migrate feature flags and their state from lower environments to higher environments in a code first fashion. Checking the state of existing flags, variants, and segments would make this process easier. As a work around, I can GET /flags and then GET the segments for each flag individually, but would prefer to be able to make one request up front.
flagr version
): 1.0.12uname -a
):Related to #110
When starting the application, if a DB connection can not be initiated, the application should retry before failing and terminating.
If the application fails to connect to a DB it immediately exits.
A simple solution could be to allow for a small number of connection attempts at a fixed interval before calling logrus.Fatal
. A more complicated solution could involve exponential backoff or and/or a configurable wait period before failure.
Configure Flagr to use PostgreSQL or MYSQL database.
Start Flagr before starting database server.
We have Flagr running in Kubernetes on Google Cloud Platform. It is configured to use a PostgreSQL database. In order to access that database, we have a sidecar which runs Cloud SQL Proxy. When deploying the server, Flagr attempts to access the database while the sidecar container is still initializing. This forces Kubernetes to restart the Flagr container upon which it is able to connect to the database through the proxy.
While this approach technically works, it seems like Flagr should be able to tolerate a small delay that could occur when getting SQL servers running in a containerized environment.
As we are constantly upgrading Flagr server, the client SDKs may experience unexpected changes from the server side. The integration tests should better include these SDKs instead of just simple CURL tests.
Specifically, we want to cover
I noticed that you offered some SDKs but none for PHP. I checked the ones you do have and saw that they are all auto-generaged using swagger-codegen. I copied some of the existing work and updated it to generate a PHP version - https://github.com/EspadaV8/phpflagr.
I'm happy for you to take this if you'd like it to be part of the main flagr namespace. If not, would you mind having a look at it and seeing if I have missed anything that should be included. I copied the swagger.yaml
and used the Makefile
and swagger_py.json
files as a base for the ones in the repo.
I was wondering if you had something set up to automatically update the swagger.yaml
files in your repositories when you build a new version of flagr?
Teams need more visibility into how and when flags are updated. For example, Email or Slack notifications can be good channels.
Not sure if this feature is a good addon to the current Flagr setup. Maybe we can build it in Flagr, maybe we can expose Flagr webhooks and let other applications implement it.
For safety checks, we may want to block deletion if there are currently ongoing traffic hitting the flag. I can imagine there are multiple ways of doing so, but storing the real-time information across multiple flagr instances requires some tuning.
Possible solutions
Support Prometheus for better observability
After getting latest code(doing dep ensure, make deps etc) and running make gen, make build throws following error:
# github.com/checkr/flagr/swagger_gen/restapi
swagger_gen/restapi/server.go:174:18: cannot use domainSocket (type *graceful.Server) as type *http.Server in argument to configureServer
swagger_gen/restapi/server.go:206:18: cannot use httpServer (type *graceful.Server) as type *http.Server in argument to configureServer
swagger_gen/restapi/server.go:292:18: cannot use httpsServer (type *graceful.Server) as type *http.Server in argument to configureServer
make: *** [build] Error 2
Generated binary sucessfully
Errors, can't compile
I am trying to add new query parameter for rest api but error prevents me from successfully compiling
flagr version
):uname -a
):Considering that this would be a big part of the infrastructure (getting lots of public requests), it would be interesting to have a health check endpoint, in order to monitor the status. Even more if you consider autoscaling or anything of the sort (knowing when the pod is ready to receive traffic, or if its unhealthy and shouldn't receive much load). e.g.: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
For even more control over the infrastructure, sending metrics to statsd would also be really awesome, in order to integrate with something like Datadog and investigate any latency with more precision.
If its something that you guys think its okay I might be willing to work on PR for both.
It is expected that if FLAGR_WEB_PREFIX
environment variable is used that the UI will be able to retrieve its static assets (js/css files)
In the index.html, all static resources are being retrieved starting at the root (e.g. /static/some.js
)
Wondering if it's possible to either set base href
value to the value of FLAGR_WEB_PREFIX or to make references to assets relative rather than absolute in the web pack config. e.g. static/some.js
instead of /static/some.js
.
docker run -it -p 18000:18000 -e FLAGR_WEB_PREFIX='/foo' checkr/flagr
localhost:18000/foo
(ensure cache is cleared)We are unable to use the flagr-ui behind our load balancer where the uri looks something like
www.domain.com/foo/flagr
flagr version
):1.0.13uname -a
): WindowsSetting query params on the findFlags API point should filter results.
findFlags returns all flags.
We tried each query param, for example:
We tried both the heroku app and the docker image from the README file.
I would open a PR with the fix myself but I lack the go/swagger knowledge..
Thanks!
Tom
As a financial organization, we must limit who has access to view and modify feature switches and A/B tests.
The Flagr admin allows for simple access control via JWT, however, we would expect more granular role-base access control. We would like to control for who can read vs. who can write changes. Potentially even more granularity down to which items could be changed on an a/b test, such as distribution and segmentation.
Organizations with multiple applications could also benefit from access to specific groups of flags. This would allow for large organizations needing different levels of administration control over who can update flags owned by their specific organization.
JWT authorization is currently used for the Flagr admin and allows for a single role, authorized or not authorized.
Implement role-based access for updating/reading flags potentially using casbin or goRBAC for the admin page. Secondly, maybe add a construct of grouping in the FlagR admin -- and allow different access levels for flags owned by different organizations. For example:
Some flags are sensitive, if it's deleted, make sure it's also easy to restore as well. Currently, in gorm, everything is soft deleted. I think this feature will need the following changes:
GET /flags?deleted=true
to get deleted flagsPOST /flags/<id>/restore
to restore deleted flagsDeprecate dep
and leverage go mod
There does not seem to be a way to disable the enableDebug option for Evaluations in a Production environment. It seems like that could lead to some data about the A/B Tests being run to leak to external users.
Is there any plan to create a config to disable that?
Basically, I've started a pod on our cluster with sqlite, everything works well. However, when I switched to postgres nothing works anymore.
time="2018-03-09T15:11:14Z" level=info msg="/go/src/github.com/checkr/flagr/pkg/handler/eval_cache.go:89sql: Scan error on column index 6: Cannot scan {} into Attachment type"
time="2018-03-09T15:11:14Z" level=info msg="/go/src/github.com/checkr/flagr/pkg/handler/eval_cache.go:89sql: Scan error on column index 6: Cannot scan {} into Attachment type"
time="2018-03-09T15:11:14Z" level=info msg="/go/src/github.com/checkr/flagr/pkg/handler/eval_cache.go:89sql: Scan error on column index 6: Cannot scan {} into Attachment type"
time="2018-03-09T15:11:14Z" level=error msg="reload evaluation cache error" err="sql: Scan error on column index 6: Cannot scan {} into Attachment type; sql: Scan error on column index 6: Cannot scan {} into Attachment type; sql: Scan error on column index 6: Cannot scan {} into Attachment type; sql: Scan error on column index 6: Cannot scan {} into Attachment type"
time="2018-03-09T15:11:14Z" level=info msg="/go/src/github.com/checkr/flagr/pkg/handler/eval_cache.go:89sql: Scan error on column ind
Trying to create a new flag with the Postgres driver
flagr version
): latestuname -a
): Linux **-846cbb9848-nd8vt 4.4.111-k8s #1 SMP Sun Jan 14 19:32:08 UTC 2018 x86_64 GNU/Linux
We have encountered an error which caused Flagr to fatally crash and restart the container whenever it tries and fails to send data to Kafka.
Every time an evaluation is processed and cannot reach Kafka, we receive the following error, the evaluation fails, and the Flagr container crashes and is forced to restart.
time="2019-03-20T20:33:39Z" level=fatal msg="Failed to start Sarama producer:" kafka_error="kafka: client has run out of available brokers to talk to (Is your cluster reachable?)"
We cannot recreate the issue in a local Docker environment, but it is extremely consistent on our OpenShift server.
When we add a WebPrefix by defining FLAGR_WEB_PREFIX
, we are expecting the entire app to use the prefix. We found that the APIs are not using the prefix, resulting in calls to /api/v1 instead of /webprefix/api/v1.
If we set up flagr locally, calls to the APIs should include the prefix. For example:
The following should return a 200 OK:
curl -v http://localhost:18000/WEBPREFIX/api/v1/health
The following should return a 404 NOT FOUND:
curl -v http://localhost:18000/api/v1/health
Instead, we get the opposite behavior, ie. works without prefix, but does not work with a prefix.
See Expected Behavior above.
May be able to fix this by ensuring that API_URL is set based on the provided WebPrefix (ie. FLAGR_WEB_PREFIX
).
Run up an instance of flagr locally and set FLAGR_WEB_PREFIX
to /flags. Run the curl commands as per expected behavior.
We are trying to run flagr with other applications and this issue prevents it from working as we are proxying only a particular webprefix to flagr.
Hi @zhouzhuojie,
This is a fantastic project! Do you & the Checkr team have a roadmap or vision for this project?
In no particular order, here are some features I've used in similar projects:
The findFlags endpoint should return an array when a query param key
is given, e.g. from GET /api/v1/flags/?key=aKey
Receiving an error message:
{
"message": "cannot query all flags. Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'key = ?))' at line 1"
}
See above request URL for reproducible API call.
I'm attempting to fetch a flag by the flag's key.
flagr version
): checkr/flagr:latest (might be 1.0.12 or later?)Pinging the healthcheck endpoint does not return the expected JSON response
Hitting the healthcheck endpoint returns a 200 JSON payload, according to the docs.
{
"message": "string"
}
https://try-flagr.herokuapp.com/api/v1/health
Hitting https://try-flagr.herokuapp.com/api/v1/health returns a gzipped file. This happens locally, on your hosted heroku, and when I deploy it on AWS.
I would expect that the bulk evaluation endpoint would return the same results as the individual evaluation endpoint.
It returns the matched segment ID, but none of the variant information.
Always occurs on using bulk evaluate.
We were hoping to load all flag data into our apps on launch to reduce network calls. At the moment, it looks like we'll need to make a network call per flag.
Just tried to update one of our projects to the latest release and got an error, checked Docker Hub and saw that 1.1.1 isn't listed (https://hub.docker.com/r/checkr/flagr/tags), last release was 1.1.0. Any chance of getting 1.1.1 tagged and pushed?
Is there a Java SDK available?
Hi,
Just curious how you determine when to put together new releases and push them to Docker Hub. I expect that I will be getting more involved with this project and it would be nice to have an idea of when things are expected to go from master into an official image.
Currently, Flagr supports basic JWT Authentication from Cookies and Auth Headers but does not provide a way to acquire the token. It would be helpful for Flagr to support the industry-standard OAuth protocol and tokens.
OAuth defines a protocol for acquiring a signed JWT token via Query String and API, and includes an audience attribute to validate that the user is authorized to use the application in question (in this case Flagr). Flagr doesn't provide a way to acquire a JWT and can only validate the signature of the token without testing the token’s attributes
I want to avoid overflowing my logs with:
{"FlagEvalResult":{"evalContext":{"entityID":"a8f155220fad675898b52160340a85e2","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:17Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
{"FlagEvalResult":{"evalContext":{"entityID":"ea132b7d85d5fc333efda94088923f44","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:18Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
{"FlagEvalResult":{"evalContext":{"entityID":"02f026d0f04d03818f5d41f3b73f062b","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:19Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
{"FlagEvalResult":{"evalContext":{"entityID":"fa8f1f5ccbafcae6fe40f4d752019a1c","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:21Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
{"FlagEvalResult":{"evalContext":{"entityID":"50f18fc583ed933f924256d28912cf26","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:22Z","variantAttachment":{},"variantID":1,"variantKey":"star"}}
{"FlagEvalResult":{"evalContext":{"entityID":"59caa1edad40e8147429eeb0c80ba700","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:24Z","variantAttachment":{},"variantID":1,"variantKey":"star"}}
{"FlagEvalResult":{"evalContext":{"entityID":"ebac32864a64579e52d3c9eb0a253a3c","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:26Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
{"FlagEvalResult":{"evalContext":{"entityID":"34c4a87d0e8645fde4d11285c5ebc847","entityType":"user","flagID":1},"evalDebugLog":{"segmentDebugLogs":[]},"flagID":1,"flagSnapshotID":36,"segmentID":1,"timestamp":"2018-04-11T10:21:27Z","variantAttachment":{},"variantID":2,"variantKey":"heart"}}
LOG_LEVEL = warning
-> disables logging of individual requests / flag eval results
It would be great to merge the evalResult struct for kafka and kinesis. It make future vendors for data recording easier.
https://github.com/checkr/flagr/blob/master/pkg/handler/data_recorder_kafka.go#L136
https://github.com/checkr/flagr/blob/master/pkg/handler/data_recorder_kinesis.go#L88
type kafkaMessageFrame struct {
Payload string `json:"payload"`
Encrypted bool `json:"encrypted"`
}
type kinesisMessageFrame struct {
Payload string `json:"payload"`
Encrypted bool `json:"encrypted"`
}
My understanding is that flagr is most likely to be used as a microservice in kubernetes. The server should only accepts requests after data loading (e.g. MySQL loading) is done
Is it possible to add support for this ? or if it's already supported plz let me know
thanks
Hi! Thanks for an awesome library — do you have plans for PHP SDK?
Assume we have a feature flag show_button
with variants show
and hide
.
A segment was on 100% rollout and distribution was 100% for hide
. Now we want to increase the percentage of people who see the button. So we adjust it to be 90% hide
and 10% show
.
What happens to existing users who have already been evaluated as hide
?
Does the change in distribution trigger a re-assessment of everyone in that segment when they evaluate a flag?
Thanks
Getting path / was not found on new install
None 404 response perhaps?
Getting the following response from my browser
root@stretch:~# curl localhost:18000
{"code":404,"message":"path / was not found"}
I wanted to try to play around with the tool. Compiled from source and based on the documentation at https://checkr.github.io/flagr/#/README. Had to install certain extra stuff (build-essential, sqlite3, yarn) and need to install yarn from yarn repo.
root@stretch:~# flagr --port=18000 --host=0.0.0.0
INFO[0000] Serving flagr at http://[::]:18000
flagr version
): no relevant output from this command. Installed based on official documention 10 minutes ago.uname -a
): Linux stretch 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/LinuxDo you have any recommendations on how to use JWT Auth with flagr. I was hoping that this could be handled with the Github OAuth. Is there any similar 3rd party vendors I could use?
I want to use numbers as variant keys and capital letters in my flag keys.
I'm not allowed due to the current regular expression.
Change the regular expression to be more permissive in Util.isSafeKey
:
https://github.com/checkr/flagr/blob/master/pkg/util/util.go
Something like: ^[\w\d-]+$
would allow a user to use lowercase, uppercase, numbers, underscores and dashes in their keys.
We are currently switching from a different system which allowed us to use any test key. We need to rename some of our flag and variant keys in order to work with Flagr. I don't really see the point of having the key names so restrictive.
Flagr can decide which variation to use for an experiment but it does not receive the data generated by the experiment.
For example: If you're evaluating the time to complete a task given an A/B variant, you'd want to log how much time the variant A got vs the variant B.
Does flagr intend to collect these results in the near future? Is it open to contributions?
pkg/repo/db.go not throwing fatal erros when issue in connecting DB .
In kubernetes environment when mysql is not ready flagr pods does not restart automatically since appropriate error is not thrown
Expect flagr should continously retry until mysql connections comes up
even though logs of flagr says unable to connect to DB still pod is running
diff --git a/pkg/repo/db.go b/pkg/repo/db.go
index c8811e5..4ef29ca 100644
--- a/pkg/repo/db.go
+++ b/pkg/repo/db.go
@@ -23,8 +23,7 @@ func GetDB() *gorm.DB {
singletonOnce.Do(func() {
db, err := gorm.Open(config.Config.DBDriver, config.Config.DBConnectionStr)
if err != nil {
logrus.WithField("err", err).Error("failed to connect to db")
panic(err)
logrus.WithField("err", err).Fatal("failed to connect to db")
}
db.SetLogger(logrus.StandardLogger())
db.AutoMigrate(entity.AutoMigrateTables...)
latest flagr version
kubernetes development envrionment
Right now there are a couple of quirks with the way that result data in formatted through the data pipeline. The biggest issue for us right now is the use of MarshalIdent
on the JSON. I see that it was used in the past for some sort of Kafka Connector, but the result leaves us with JSON where each element begins on a new line with a prefix. This breaks a lot of data processing applications that expect JSON elements to be newline delimited.
A second quirk I noticed is that the JSON message frame saves the payload as a string of escaped JSON. I suspect this has something to do with the encryption option for Kafka? However, when things are not encrypted, it would be much cleaner if the all of the JSON was marshaled at once such that the payload field was simply another level of the overall JSON object as opposed to a string containing the JSON data.
I realize that these changes would amount to breaking changes in the data generated by the data pipeline, but would it be reasonable to add some sort of configuration option(s) to allow for these changes? This is something that I could address ASAP if we can figure out a way to cleanly handle this.
Flagr Architecture references many components: UI, Flagr, Sync Worker, Evaluator, ...
https://checkr.github.io/flagr/#/flagr_overview?id=flagr-architecture
However, the doc about deployment and configuration is about a single docker image that seem to correspond to the Flagr Manager tier.
It is possible to deploy a scalable installation as the one described in this document?
Where can I find any documentation on how to do so?
Thanks
I notice that right now for simplicity reasons variant attachments only support string/string key/value pairs. This works reasonably well, but there are some tests that would be made much easier if we were able to simply return a full JSON object. I envision something like the JSON editor used in the debug console which could (optionally?) be used to set a variant attachment. Is this something that should be reasonably achievable or are there some complications that could occur as a result of supporting this?
Suppose I specify flag_key="blah" for flag 1. I delete flag 1, and then try to create a new flag with flag_key="blah". I would expect that I can create the new flag successfully since the flag key is no longer being used.
Instead, I get the error:
Error 1062: Duplicate entry 'thisisatestflag' for key 'idx_flag_key'
This is likely due to GORM using soft deletes, so we're unable to add the new flag because the key still exists in the index.
Temporary workaround we're planning to use is to modify the flag key to a dummy value before deleting any flags. A real solution could involve detecting that the flag key exists under another flag and either:
We have some testing utils that will create a temporary flag (identified by key), run some code that depends on the flag, and then delete the flag. However consecutive attempts to re-use the same key fails due to this issue.
With disk files (YAML or JSON) as the data source, we can check in those files into a version control system like Git, and it's easier to start flagr in testing or staging environment.
Currently, flagr supports sqlite export, it would be great if flagr can export YAML or JSON files from a production instance. We can then use the exported file as the seed data and make changes.
Currently in order to reorder segments, one has to drag and drop the segment around. It's not a good UI experience, especially when there are a lot of segments in a flag.
Wondering if we can have "up"/"down" buttons inside each segment so that reordering can be more intuitive.
We would like to consume Flagr for feature management. We don't want to ship any containers based on Debian due to a local policy.
Instead, we would like to use "Alpine - 3.6" version - https://hub.docker.com/_/alpine/.
Flagr should build and run on Alpine base image.
Untested
None that is offered in the checkr community.
Due to local policy, we cannot use Debian to distribute containers. And we would like Flagr to be built using Alpine.
flagr version
): Latestuname -a
): Alpine 3.6If the entity context matches the constraints in a segment, it should stop evaluating the following segments.
Specifically, for 0% rollout segment, it shouldn't go to the next segments.
For example:
The first segment catches all the entity, and roll out to 0%, so every entity should have null
variant returned.
It falls through to the next segment for non-rolled out users in the previous segments.
Stop evaluating the next segments if we found a match (i.e. satisfied all the constraints in a segment)
Following the instructions at https://checkr.github.io/flagr/#/home the following commands should build flagr:
go get -u github.com/checkr/flagr
cd $GOPATH/src/github.com/checkr/flagr
make all
However, the “build” target in the Makefile (CGO_ENABLED=1 go build -o $(PWD)/flagr github.com/checkr/flagr/swagger_gen/cmd/flagr-server) fails with the following error:
…
Building flagr to /Users/…/go/src/github.com/checkr/flagr/flagr ...
# github.com/checkr/flagr/pkg/mapper/entity_restapi/r2e
pkg/mapper/entity_restapi/r2e/r2e.go:13:25: undefined: models.PutDistributionsRequestDistributions
make: *** [build] Error 2
The macOS version is High Sierra 10.13.4 with Go version go1.10.1.
Tried to build from source according to instructions in the documentation.
flagr version
): Code freshly fetched today from github.com/checkr/flagruname -a
): Darwin ….local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar 5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_6At my company we use mostly Kinesis for streaming, so I would like to start working on a kinesis recorder; @zhouzhuojie wdyt?
My team and I were talking about how we want to have some documentation for the different A/B tests we are running. I thought it would be useful if the flag page had a "Notes" section so that relevant details could be viewed right in the Flagr UI. It could be something as simple as a text area or even allow for some markdown. Opening this issue to gauge interest in this idea.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.