permitio / opal-fetcher-postgres Goto Github PK

View Code? Open in Web Editor NEW

19.0 5.0 17.0 34 KB

An OPAL custom data fetcher to bring authorization state from Postgres

Home Page: https://opal.ac

License: Apache License 2.0

Dockerfile 1.59% Python 96.12% Makefile 2.29%

opal-fetcher-postgres's Introduction

OPAL Fetcher for Postgres

Check out OPAL main repo here.

What's in this repo?

An OPAL custom fetch provider to bring authorization state from Postgres.

This fetcher is both:

A fully functional fetch-provider for Postgres: can be used by OPAL to fetch data from Postgres DB.
Serving as an example how to write custom fetch providers for OPAL and how to publish them as pip packages.

How to try this custom fetcher in one command? (Example docker-compose configuration)

You can test this fetcher with the example docker compose file in this repository root. Clone this repo, cd into the cloned repo, and then run:

docker compose up

this docker compose configuration already correctly configures OPAL to load the Postgres Fetch Provider, and correctly configures OPAL_DATA_CONFIG_SOURCES to include an entry that uses this fetcher.

✏️ How to use this fetcher in your OPAL Setup

1) Build a custom opal-client `Dockerfile`

The official docker image only contains the built-in fetch providers. You need to create your own Dockerfile (that is based on the official docker image), that includes this fetcher's pip package.

Your Dockerfile should look like this:

FROM permitio/opal-client:latest
RUN pip install --no-cache-dir --user opal-fetcher-postgres

2) Build your custom opal-client container

Say your special Dockerfile from step one is called custom_client.Dockerfile.

You must build a customized OPAL container from this Dockerfile, like so:

docker build -t yourcompany/opal-client -f custom_client.Dockerfile .

3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`

Pass the following environment variable to the OPAL client docker container (comma-separated provider modules):

OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_postgres.provider

Notice that OPAL receives a list from where to search for fetch providers. The list in our case includes the built-in providers (opal_common.fetcher.providers) and our custom postgres provider.

4) Using the custom provider in your DataSourceEntry objects

Your DataSourceEntry objects (either in OPAL_DATA_CONFIG_SOURCES or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.

Example value of OPAL_DATA_CONFIG_SOURCES (formatted nicely, but in env var you should pack this to one-line and no-spaces):

{
    "config": {
        "entries": [
            {
                "url": "postgresql://postgres@example_db:5432/postgres",
                "config": {
                    "fetcher": "PostgresFetchProvider",
                    "query": "SELECT * from city;",
                    "connection_params": {
                        "password": "postgres"
                    }
                },
                "topics": [
                    "policy_data"
                ],
                "dst_path": "cities"
            }
        ]
    }
}

Notice how config is an instance of PostgresFetcherConfig (code is in opal_fetcher_postgres/provider.py).

Values for this fetcher config:

The url is actually a postgres dsn. You can set the postgres password in the dsn itself if you want.
connection_params are optional, if you want to include certain overrides outside the dsn.
Your config must include the fetcher key to indicate to OPAL that you use a custom fetcher.
Your config must include the query key to indicate what query to run against postgres.

🚩 Possible User Issues

While trying to send requests to a Postgres data source, you may encounter that the request fails. This can be caused by the format of the config entry URL for which the standard is:

postgresql://<user>:<password>@<host>/<db>

It might be most common that this request fails due to the password field being incorrectly parsed by the underlying library called asyncpg, which is one of the required libraries used within our OPAL custom data fetcher.

In order to solve the issue, you need to change the data source config entry URL to the format shown below:

postgresql://<host>/<db>?user=<user>&password=<password>

📖 About OPAL (Open Policy Administration Layer)

OPAL is an administration layer for Open Policy Agent (OPA), detecting changes to both policy and policy data in realtime and pushing live updates to your agents.

OPAL brings open-policy up to the speed needed by live applications. As your application state changes (whether it's via your APIs, DBs, git, S3 or 3rd-party SaaS services), OPAL will make sure your services are always in sync with the authorization data and policy they need (and only those they need).

Check out OPAL's main site at OPAL.ac.

opal-fetcher-postgres's People

Contributors

Stargazers

Watchers

Forkers

phi1010 hongbo-miao vasica38 jb2001216 karan-s-mittal filipermit tlowerison avo-sepp david-hamilton-bah northvolt rahulaga01 shusaan lcarlos10 isurucuma marianojabdala

opal-fetcher-postgres's Issues

Update to permitio/opal-client

It would be great to update authorizon/opal-client to permitio/opal-client! 😃
https://github.com/permitio/opal-fetcher-postgres/blob/master/Dockerfile#L1

Also note once switch to permitio/opal-client, you will see

SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.

opal-client failed

When I run docker compose up, I get error:

opal-client log

[2021-08-23 17:43:31 +0000] [1] [INFO] Starting gunicorn 20.1.0
[2021-08-23 17:43:31 +0000] [1] [INFO] Listening at: http://0.0.0.0:7000 (1)
[2021-08-23 17:43:31 +0000] [1] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2021-08-23 17:43:31 +0000] [9] [INFO] Booting worker with pid: 9
2021-08-23T17:43:31.756294+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'FastApiRpcFetchProvider' found at: <class 'opal_common.fetcher.providers.fastapi_rpc_fetch_provider.FastApiRpcFetchProvider'>
2021-08-23T17:43:31.756561+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'HttpFetchProvider' found at: <class 'opal_common.fetcher.providers.http_fetch_provider.HttpFetchProvider'>
2021-08-23T17:43:31.776586+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'PostgresFetchProvider' found at: <class 'opal_fetcher_postgres.provider.PostgresFetchProvider'>
2021-08-23T17:43:31.776740+0000 |opal_common.fetcher.fetcher_register    | INFO  | Fetcher Register loaded
2021-08-23T17:43:31.797014+0000 |opal_client.opa.runner                  | INFO  | Launching opa runner
2021-08-23T17:43:31.798399+0000 |opal_client.opa.runner                  | INFO  | Running OPA inline: opa run --server --addr=:8181 --authentication=off --authorization=off --log-level=info
2021-08-23T17:43:31.813046+0000 |opal_client.opa.logger                  | INFO  | Initializing server. {'addrs': [':8181'], 'diagnostic-addrs': [], 'time': '2021-08-23T17:43:31Z'}
2021-08-23T17:43:32.804447+0000 |opal_client.opa.runner                  | INFO  | Running OPA initial start callbacks
2021-08-23T17:43:32.805396+0000 |opal_client.policy.updater              | INFO  | Launching policy updater
2021-08-23T17:43:32.805677+0000 |opal_client.data.updater                | INFO  | Launching data updater
2021-08-23T17:43:32.805880+0000 |opal_client.policy.updater              | INFO  | Subscribing to topics: ['policy:.']
2021-08-23T17:43:32.806317+0000 |opal_client.data.updater                | INFO  | Subscribing to topics: ['policy_data']
2021-08-23T17:43:32.806967+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Trying to connect to Pub/Sub server - ws://opal_server:7002/ws
2021-08-23T17:43:32.807783+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
2021-08-23T17:43:32.811801+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Trying to connect to Pub/Sub server - ws://opal_server:7002/ws
2021-08-23T17:43:32.812388+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
2021-08-23T17:43:32.871213+0000 |opal_client.policy.updater              | INFO  | Connected to server
2021-08-23T17:43:32.871684+0000 |opal_client.policy.updater              | INFO  | Refetching policy code (full bundle)
2021-08-23T17:43:32.873803+0000 |opal_client.data.updater                | INFO  | Connected to server
2021-08-23T17:43:32.873948+0000 |opal_client.data.updater                | INFO  | Performing data configuration, reason: Initial load
2021-08-23T17:43:32.874025+0000 |opal_client.data.updater                | INFO  | Getting data-sources configuration from 'http://opal_server:7002/data/config'
2021-08-23T17:43:32.879804+0000 |opal_client.data.updater                | INFO  | Triggering data update with id: bda17568be5c4c3180f8d62f387e6f0e
2021-08-23T17:43:32.879997+0000 |opal_client.data.updater                | INFO  | Fetching policy data
2021-08-23T17:43:32.880289+0000 |opal_client.data.fetcher                | INFO  | Fetching data from url: postgresql://postgres@example_db:5432/postgres
2021-08-23T17:43:32.896534+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Connected to PubSub server ws://opal_server:7002/ws
2021-08-23T17:43:32.898193+0000 |opal_client.policy.updater              | INFO  | got policy bundle, commit hash: 'c60470ef8046152a8498d81df0eff66ddd1632e2'
2021-08-23T17:43:32.900306+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41390', 'req_id': 1, 'req_method': 'GET', 'req_path': '/v1/policies', 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.901079+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41390', 'req_id': 1, 'req_method': 'GET', 'req_path': '/v1/policies', 'resp_bytes': 13, 'resp_duration': 0.568914, 'resp_status': 200, 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.903906+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41392', 'req_id': 2, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.906962+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41392', 'req_id': 2, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 3.040543, 'resp_status': 400, 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.913913+0000 |opal_client.data.updater                | INFO  | Saving fetched data to policy-store: source url='postgresql://postgres@example_db:5432/postgres', destination path='/cities'
2021-08-23T17:43:32.915890+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41394', 'req_id': 3, 'req_method': 'PUT', 'req_path': '/v1/data/cities', 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.916704+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41394', 'req_id': 3, 'req_method': 'PUT', 'req_path': '/v1/data/cities', 'resp_bytes': 0, 'resp_duration': 0.889005, 'resp_status': 204, 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:32.992279+0000 |opal_client.opa.logger                  | INFO  | OPA is out of date.  {'current_version': '0.30.2', 'download_opa': 'https://openpolicyagent.org/downloads/v0.31.0/opa_linux_amd64', 'latest_version': '0.31.0', 'release_notes': 'https://github.com/open-policy-agent/opa/releases/tag/v0.31.0', 'time': '2021-08-23T17:43:32Z'}
2021-08-23T17:43:34.911369+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41396', 'req_id': 4, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:34Z'}
2021-08-23T17:43:34.912880+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41396', 'req_id': 4, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 1.120678, 'resp_status': 400, 'time': '2021-08-23T17:43:34Z'}
2021-08-23T17:43:34.913772+0000 |opal_client...base_policy_store_client  |WARNING | OPA transaction failed, transaction id=c60470ef8046152a8498d81df0eff66ddd1632e2, actions=['set_policies'], error=RetryError(<Future at 0x7f7fa0672f70 state=finished raised ValueError>)
2021-08-23T17:43:34.914065+0000 |fastapi_websocket_rpc.websocket_rpc_c...|ERROR  | RPC Error
Traceback (most recent call last):
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 45, in __call__
  result = await fn(*args, **kwargs)
                 │   │       └ {'policy_id': 'rbac.rego', 'policy_code': '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This ...
                 │   └ (<opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>,)
                 └ <function OpaClient.set_policy at 0x7f7fa0b1d940>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 173, in set_policy
  return await proxy_response_unless_invalid(opa_response, accepted_status_codes=[status.HTTP_200_OK])
               │                             │                                    │      └ 200
               │                             │                                    └ <module 'starlette.status' from '/root/.local/lib/python3.8/site-packages/starlette/status.py'>
               │                             └ <ClientResponse(http://localhost:8181/v1/policies/rbac.rego) [400 Bad Request]>
               │                               <CIMultiDictProxy('Content-Type': 'applicatio...
               └ <function proxy_response_unless_invalid at 0x7f7fa0b0cca0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 54, in proxy_response_unless_invalid
  raise ValueError("OPA Client: unexpected status code: {}, error: {}".format(response.status_code, error))
                                                                              │        │            └ {'code': 'invalid_parameter', 'message': 'error(s) occurred while compiling module(s)', 'errors': [{'code': 'rego_type_error'...
                                                                              │        └ 400
                                                                              └ <starlette.responses.Response object at 0x7f7fa0672eb0>
ValueError: OPA Client: unexpected status code: 400, error: {'code': 'invalid_parameter', 'message': 'error(s) occurred while compiling module(s)', 'errors': [{'code': 'rego_type_error', 'message': 'undefined function data.utils.hasPermission', 'location': {'file': 'rbac.rego', 'row': 37, 'col': 2}}]}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/.local/bin/gunicorn", line 8, in <module>
  sys.exit(run())
  │   │    └ <function run at 0x7f7fa2b85940>
  │   └ <built-in function exit>
  └ <module 'sys' (built-in)>
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
  WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  └ <class 'gunicorn.app.wsgiapp.WSGIApplication'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run
  super().run()
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run
  Arbiter(self).run()
  │       └ <gunicorn.app.wsgiapp.WSGIApplication object at 0x7f7fa2b5f820>
  └ <class 'gunicorn.arbiter.Arbiter'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 202, in run
  self.manage_workers()
  │    └ <function Arbiter.manage_workers at 0x7f7fa2494160>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 551, in manage_workers
  self.spawn_workers()
  │    └ <function Arbiter.spawn_workers at 0x7f7fa2494280>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 622, in spawn_workers
  self.spawn_worker()
  │    └ <function Arbiter.spawn_worker at 0x7f7fa24941f0>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
  worker.init_process()
  │      └ <function UvicornWorker.init_process at 0x7f7fa1b44b80>
  └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 64, in init_process
  super(UvicornWorker, self).init_process()
        │              └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
        └ <class 'uvicorn.workers.UvicornWorker'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 142, in init_process
  self.run()
  │    └ <function UvicornWorker.run at 0x7f7fa1b44ca0>
  └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 77, in run
  loop.run_until_complete(server.serve(sockets=self.sockets))
  │    │                  │      │             │    └ [<gunicorn.sock.TCPSocket object at 0x7f7fa1b30f10>]
  │    │                  │      │             └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
  │    │                  │      └ <function Server.serve at 0x7f7fa1e1cc10>
  │    │                  └ <uvicorn.server.Server object at 0x7f7fa206b4f0>
  │    └ <method 'run_until_complete' of 'uvloop.loop.Loop' objects>
  └ <uvloop.Loop running=True closed=False debug=False>
> File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/websocket_rpc_client.py", line 127, in __connect__
  await self.channel.on_connect()
        │    │       └ <function RpcChannel.on_connect at 0x7f7fa0b46ee0>
        │    └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>
        └ <fastapi_websocket_rpc.websocket_rpc_client.WebSocketRpcClient object at 0x7f7fa0690d60>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/rpc_channel.py", line 237, in on_connect
  await self.on_handler_event(self._connect_handlers, self)
        │    │                │    │                  └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>
        │    │                │    └ [<bound method PubSubClient._primary_on_connect of <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa069...
        │    │                └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>
        │    └ <function RpcChannel.on_handler_event at 0x7f7fa0b46e50>
        └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/rpc_channel.py", line 234, in on_handler_event
  await asyncio.gather(*(callback(*args, **kwargs) for callback in handlers))
        │       │                  │       │                       └ [<bound method PubSubClient._primary_on_connect of <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa069...
        │       │                  │       └ {}
        │       │                  └ (<fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>,)
        │       └ <function gather at 0x7f7fa1eb5ca0>
        └ <module 'asyncio' from '/usr/local/lib/python3.8/asyncio/__init__.py'>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_pubsub/pub_sub_client.py", line 217, in _primary_on_connect
  await asyncio.gather(*(callback(self, channel) for callback in self._on_connect))
        │       │                 │     │                        │    └ [<bound method PolicyUpdater._on_connect of <opal_client.policy.updater.PolicyUpdater object at 0x7f7fa205f910>>]
        │       │                 │     │                        └ <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa06907c0>
        │       │                 │     └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa06ab820>
        │       │                 └ <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa06907c0>
        │       └ <function gather at 0x7f7fa1eb5ca0>
        └ <module 'asyncio' from '/usr/local/lib/python3.8/asyncio/__init__.py'>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy/updater.py", line 108, in _on_connect
  await self.update_policy()
        │    └ <function PolicyUpdater.update_policy at 0x7f7fa09f63a0>
        └ <opal_client.policy.updater.PolicyUpdater object at 0x7f7fa205f910>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy/updater.py", line 214, in update_policy
  await store_transaction.set_policies(bundle)
        │                 │            └ PolicyBundle(manifest=['data.json', 'rbac.rego', 'utils.rego'], hash='c60470ef8046152a8498d81df0eff66ddd1632e2', old_hash=Non...
        │                 └ <function AbstractPolicyStore.set_policies at 0x7f7fa0b02550>
        └ <opal_client.policy_store.base_policy_store_client.PolicyStoreTransactionContextManager object at 0x7f7fa0672760>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 256, in set_policies
  return await self._set_policies_from_complete_bundle(bundle)
               │    │                                  └ PolicyBundle(manifest=['data.json', 'rbac.rego', 'utils.rego'], hash='c60470ef8046152a8498d81df0eff66ddd1632e2', old_hash=Non...
               │    └ <function OpaClient._set_policies_from_complete_bundle at 0x7f7fa0b22430>
               └ <opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 268, in _set_policies_from_complete_bundle
  await self.set_policy(policy_id=module.path, policy_code=module.rego)
        │    │                    │      │                 │      └ '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This example defines an RBAC model for a Pet St...
        │    │                    │      │                 └ RegoModule(path='rbac.rego', package_name='app.rbac', rego='# Role-based Access Control (RBAC)\n# ---------------------------...
        │    │                    │      └ 'rbac.rego'
        │    │                    └ RegoModule(path='rbac.rego', package_name='app.rbac', rego='# Role-based Access Control (RBAC)\n# ---------------------------...
        │    └ <function AsyncRetrying.wraps.<locals>.async_wrapped at 0x7f7fa0b1daf0>
        └ <opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 79, in async_wrapped
  return await fn(*args, **kwargs)
               │   │       └ {'policy_id': 'rbac.rego', 'policy_code': '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This ...
               │   └ (<opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>,)
               └ <function OpaClient.set_policy at 0x7f7fa0b1d9d0>
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 42, in __call__
  do = self.iter(retry_state=retry_state)
       │    │                └ <tenacity.RetryCallState object at 0x7f7fa0672bb0>
       │    └ <function BaseRetrying.iter at 0x7f7fa0ac5c10>
       └ <AsyncRetrying object at 0x7f7fa0b0ee80 (stop=<tenacity.stop.stop_after_attempt object at 0x7f7fa0b0ed30>, wait=<tenacity.wai...
File "/root/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 373, in iter
  six.raise_from(retry_exc, fut.exception())
  │   │          │          │   └ <function Future.exception at 0x7f7fa1fba550>
  │   │          │          └ <Future at 0x7f7fa0672f70 state=finished raised ValueError>
  │   │          └ RetryError(<Future at 0x7f7fa0672f70 state=finished raised ValueError>)
  │   └ <function raise_from at 0x7f7fa0bd2790>
  └ <module 'six' from '/root/.local/lib/python3.8/site-packages/six.py'>
File "<string>", line 3, in raise_from
tenacity.RetryError: RetryError[<Future at 0x7f7fa0672f70 state=finished raised ValueError>]
2021-08-23T17:43:35.858602+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
2021-08-23T17:43:35.869748+0000 |opal_client.policy.updater              | INFO  | Connected to server
2021-08-23T17:43:35.869938+0000 |opal_client.policy.updater              | INFO  | Refetching policy code (full bundle)
2021-08-23T17:43:35.892245+0000 |opal_client.policy.updater              | INFO  | got policy bundle, commit hash: 'c60470ef8046152a8498d81df0eff66ddd1632e2'
2021-08-23T17:43:35.893954+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41402', 'req_id': 5, 'req_method': 'GET', 'req_path': '/v1/policies', 'time': '2021-08-23T17:43:35Z'}
2021-08-23T17:43:35.894336+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41402', 'req_id': 5, 'req_method': 'GET', 'req_path': '/v1/policies', 'resp_bytes': 13, 'resp_duration': 0.153132, 'resp_status': 200, 'time': '2021-08-23T17:43:35Z'}
2021-08-23T17:43:35.896196+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41404', 'req_id': 6, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:35Z'}
2021-08-23T17:43:35.897293+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41404', 'req_id': 6, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 1.059368, 'resp_status': 400, 'time': '2021-08-23T17:43:35Z'}
2021-08-23T17:43:37.901788+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41406', 'req_id': 7, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:37Z'}
2021-08-23T17:43:37.902706+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41406', 'req_id': 7, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 0.967893, 'resp_status': 400, 'time': '2021-08-23T17:43:37Z'}
2021-08-23T17:43:37.903923+0000 |opal_client...base_policy_store_client  |WARNING | OPA transaction failed, transaction id=c60470ef8046152a8498d81df0eff66ddd1632e2, actions=['set_policies'], error=RetryError(<Future at 0x7f7f9e60d9a0 state=finished raised ValueError>)
2021-08-23T17:43:37.904287+0000 |fastapi_websocket_rpc.websocket_rpc_c...|ERROR  | RPC Error
Traceback (most recent call last):
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 45, in __call__
  result = await fn(*args, **kwargs)
                 │   │       └ {'policy_id': 'rbac.rego', 'policy_code': '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This ...
                 │   └ (<opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>,)
                 └ <function OpaClient.set_policy at 0x7f7fa0b1d940>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 173, in set_policy
  return await proxy_response_unless_invalid(opa_response, accepted_status_codes=[status.HTTP_200_OK])
               │                             │                                    │      └ 200
               │                             │                                    └ <module 'starlette.status' from '/root/.local/lib/python3.8/site-packages/starlette/status.py'>
               │                             └ <ClientResponse(http://localhost:8181/v1/policies/rbac.rego) [400 Bad Request]>
               │                               <CIMultiDictProxy('Content-Type': 'applicatio...
               └ <function proxy_response_unless_invalid at 0x7f7fa0b0cca0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 54, in proxy_response_unless_invalid
  raise ValueError("OPA Client: unexpected status code: {}, error: {}".format(response.status_code, error))
                                                                              │        │            └ {'code': 'invalid_parameter', 'message': 'error(s) occurred while compiling module(s)', 'errors': [{'code': 'rego_type_error'...
                                                                              │        └ 400
                                                                              └ <starlette.responses.Response object at 0x7f7f9e60dd60>
ValueError: OPA Client: unexpected status code: 400, error: {'code': 'invalid_parameter', 'message': 'error(s) occurred while compiling module(s)', 'errors': [{'code': 'rego_type_error', 'message': 'undefined function data.utils.hasPermission', 'location': {'file': 'rbac.rego', 'row': 37, 'col': 2}}]}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/root/.local/bin/gunicorn", line 8, in <module>
  sys.exit(run())
  │   │    └ <function run at 0x7f7fa2b85940>
  │   └ <built-in function exit>
  └ <module 'sys' (built-in)>
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
  WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  └ <class 'gunicorn.app.wsgiapp.WSGIApplication'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run
  super().run()
File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run
  Arbiter(self).run()
  │       └ <gunicorn.app.wsgiapp.WSGIApplication object at 0x7f7fa2b5f820>
  └ <class 'gunicorn.arbiter.Arbiter'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 202, in run
  self.manage_workers()
  │    └ <function Arbiter.manage_workers at 0x7f7fa2494160>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 551, in manage_workers
  self.spawn_workers()
  │    └ <function Arbiter.spawn_workers at 0x7f7fa2494280>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 622, in spawn_workers
  self.spawn_worker()
  │    └ <function Arbiter.spawn_worker at 0x7f7fa24941f0>
  └ <gunicorn.arbiter.Arbiter object at 0x7f7fa2b5f940>
File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
  worker.init_process()
  │      └ <function UvicornWorker.init_process at 0x7f7fa1b44b80>
  └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 64, in init_process
  super(UvicornWorker, self).init_process()
        │              └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
        └ <class 'uvicorn.workers.UvicornWorker'>
File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 142, in init_process
  self.run()
  │    └ <function UvicornWorker.run at 0x7f7fa1b44ca0>
  └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 77, in run
  loop.run_until_complete(server.serve(sockets=self.sockets))
  │    │                  │      │             │    └ [<gunicorn.sock.TCPSocket object at 0x7f7fa1b30f10>]
  │    │                  │      │             └ <uvicorn.workers.UvicornWorker object at 0x7f7fa1b34160>
  │    │                  │      └ <function Server.serve at 0x7f7fa1e1cc10>
  │    │                  └ <uvicorn.server.Server object at 0x7f7fa206b4f0>
  │    └ <method 'run_until_complete' of 'uvloop.loop.Loop' objects>
  └ <uvloop.Loop running=True closed=False debug=False>
> File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/websocket_rpc_client.py", line 127, in __connect__
  await self.channel.on_connect()
        │    │       └ <function RpcChannel.on_connect at 0x7f7fa0b46ee0>
        │    └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>
        └ <fastapi_websocket_rpc.websocket_rpc_client.WebSocketRpcClient object at 0x7f7fa0690d60>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/rpc_channel.py", line 237, in on_connect
  await self.on_handler_event(self._connect_handlers, self)
        │    │                │    │                  └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>
        │    │                │    └ [<bound method PubSubClient._primary_on_connect of <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa069...
        │    │                └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>
        │    └ <function RpcChannel.on_handler_event at 0x7f7fa0b46e50>
        └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_rpc/rpc_channel.py", line 234, in on_handler_event
  await asyncio.gather(*(callback(*args, **kwargs) for callback in handlers))
        │       │                  │       │                       └ [<bound method PubSubClient._primary_on_connect of <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa069...
        │       │                  │       └ {}
        │       │                  └ (<fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>,)
        │       └ <function gather at 0x7f7fa1eb5ca0>
        └ <module 'asyncio' from '/usr/local/lib/python3.8/asyncio/__init__.py'>
File "/root/.local/lib/python3.8/site-packages/fastapi_websocket_pubsub/pub_sub_client.py", line 217, in _primary_on_connect
  await asyncio.gather(*(callback(self, channel) for callback in self._on_connect))
        │       │                 │     │                        │    └ [<bound method PolicyUpdater._on_connect of <opal_client.policy.updater.PolicyUpdater object at 0x7f7fa205f910>>]
        │       │                 │     │                        └ <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa06907c0>
        │       │                 │     └ <fastapi_websocket_rpc.rpc_channel.RpcChannel object at 0x7f7fa063e4f0>
        │       │                 └ <fastapi_websocket_pubsub.pub_sub_client.PubSubClient object at 0x7f7fa06907c0>
        │       └ <function gather at 0x7f7fa1eb5ca0>
        └ <module 'asyncio' from '/usr/local/lib/python3.8/asyncio/__init__.py'>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy/updater.py", line 108, in _on_connect
  await self.update_policy()
        │    └ <function PolicyUpdater.update_policy at 0x7f7fa09f63a0>
        └ <opal_client.policy.updater.PolicyUpdater object at 0x7f7fa205f910>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy/updater.py", line 214, in update_policy
  await store_transaction.set_policies(bundle)
        │                 │            └ PolicyBundle(manifest=['data.json', 'rbac.rego', 'utils.rego'], hash='c60470ef8046152a8498d81df0eff66ddd1632e2', old_hash=Non...
        │                 └ <function AbstractPolicyStore.set_policies at 0x7f7fa0b02550>
        └ <opal_client.policy_store.base_policy_store_client.PolicyStoreTransactionContextManager object at 0x7f7f9e60dbe0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 256, in set_policies
  return await self._set_policies_from_complete_bundle(bundle)
               │    │                                  └ PolicyBundle(manifest=['data.json', 'rbac.rego', 'utils.rego'], hash='c60470ef8046152a8498d81df0eff66ddd1632e2', old_hash=Non...
               │    └ <function OpaClient._set_policies_from_complete_bundle at 0x7f7fa0b22430>
               └ <opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 268, in _set_policies_from_complete_bundle
  await self.set_policy(policy_id=module.path, policy_code=module.rego)
        │    │                    │      │                 │      └ '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This example defines an RBAC model for a Pet St...
        │    │                    │      │                 └ RegoModule(path='rbac.rego', package_name='app.rbac', rego='# Role-based Access Control (RBAC)\n# ---------------------------...
        │    │                    │      └ 'rbac.rego'
        │    │                    └ RegoModule(path='rbac.rego', package_name='app.rbac', rego='# Role-based Access Control (RBAC)\n# ---------------------------...
        │    └ <function AsyncRetrying.wraps.<locals>.async_wrapped at 0x7f7fa0b1daf0>
        └ <opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 79, in async_wrapped
  return await fn(*args, **kwargs)
               │   │       └ {'policy_id': 'rbac.rego', 'policy_code': '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This ...
               │   └ (<opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>,)
               └ <function OpaClient.set_policy at 0x7f7fa0b1d9d0>
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 42, in __call__
  do = self.iter(retry_state=retry_state)
       │    │                └ <tenacity.RetryCallState object at 0x7f7f9e60de80>
       │    └ <function BaseRetrying.iter at 0x7f7fa0ac5c10>
       └ <AsyncRetrying object at 0x7f7fa0b0ee80 (stop=<tenacity.stop.stop_after_attempt object at 0x7f7fa0b0ed30>, wait=<tenacity.wai...
File "/root/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 373, in iter
  six.raise_from(retry_exc, fut.exception())
  │   │          │          │   └ <function Future.exception at 0x7f7fa1fba550>
  │   │          │          └ <Future at 0x7f7f9e60d9a0 state=finished raised ValueError>
  │   │          └ RetryError(<Future at 0x7f7f9e60d9a0 state=finished raised ValueError>)
  │   └ <function raise_from at 0x7f7fa0bd2790>
  └ <module 'six' from '/root/.local/lib/python3.8/site-packages/six.py'>
File "<string>", line 3, in raise_from
tenacity.RetryError: RetryError[<Future at 0x7f7f9e60d9a0 state=finished raised ValueError>]
2021-08-23T17:43:39.286038+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
2021-08-23T17:43:39.297423+0000 |opal_client.policy.updater              | INFO  | Connected to server
2021-08-23T17:43:39.297608+0000 |opal_client.policy.updater              | INFO  | Refetching policy code (full bundle)
2021-08-23T17:43:39.328751+0000 |opal_client.policy.updater              | INFO  | got policy bundle, commit hash: 'c60470ef8046152a8498d81df0eff66ddd1632e2'
2021-08-23T17:43:39.331631+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41412', 'req_id': 8, 'req_method': 'GET', 'req_path': '/v1/policies', 'time': '2021-08-23T17:43:39Z'}
2021-08-23T17:43:39.332070+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41412', 'req_id': 8, 'req_method': 'GET', 'req_path': '/v1/policies', 'resp_bytes': 13, 'resp_duration': 0.132113, 'resp_status': 200, 'time': '2021-08-23T17:43:39Z'}
2021-08-23T17:43:39.334898+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41414', 'req_id': 9, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:39Z'}
2021-08-23T17:43:39.336371+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41414', 'req_id': 9, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 0.946037, 'resp_status': 400, 'time': '2021-08-23T17:43:39Z'}
2021-08-23T17:43:41.342537+0000 |opal_client.opa.logger                  | INFO  | Received request.    {'client_addr': '127.0.0.1:41416', 'req_id': 10, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'time': '2021-08-23T17:43:41Z'}
2021-08-23T17:43:41.344037+0000 |opal_client.opa.logger                  | INFO  | Sent response.       {'client_addr': '127.0.0.1:41416', 'req_id': 10, 'req_method': 'PUT', 'req_path': '/v1/policies/rbac.rego', 'resp_bytes': 315, 'resp_duration': 1.443397, 'resp_status': 400, 'time': '2021-08-23T17:43:41Z'}
2021-08-23T17:43:41.345722+0000 |opal_client...base_policy_store_client  |WARNING | OPA transaction failed, transaction id=c60470ef8046152a8498d81df0eff66ddd1632e2, actions=['set_policies'], error=RetryError(<Future at 0x7f7f9e58b640 state=finished raised ValueError>)
2021-08-23T17:43:41.346221+0000 |fastapi_websocket_rpc.websocket_rpc_c...|ERROR  | RPC Error
Traceback (most recent call last):
File "/root/.local/lib/python3.8/site-packages/tenacity/_asyncio.py", line 45, in __call__
  result = await fn(*args, **kwargs)
                 │   │       └ {'policy_id': 'rbac.rego', 'policy_code': '# Role-based Access Control (RBAC)\n# --------------------------------\n#\n# This ...
                 │   └ (<opal_client.policy_store.opa_client.OpaClient object at 0x7f7fa0deb4f0>,)
                 └ <function OpaClient.set_policy at 0x7f7fa0b1d940>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 173, in set_policy
  return await proxy_response_unless_invalid(opa_response, accepted_status_codes=[status.HTTP_200_OK])
               │                             │                                    │      └ 200
               │                             │                                    └ <module 'starlette.status' from '/root/.local/lib/python3.8/site-packages/starlette/status.py'>
               │                             └ <ClientResponse(http://localhost:8181/v1/policies/rbac.rego) [400 Bad Request]>
               │                               <CIMultiDictProxy('Content-Type': 'applicatio...
               └ <function proxy_response_unless_invalid at 0x7f7fa0b0cca0>
File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.11-py3.8.egg/opal_client/policy_store/opa_client.py", line 54, in proxy_response_unless_invalid
  raise ValueError("OPA Client: unexpected status code: {}, error: {}".format(response.status_code, error))
                                                                              │        │            └ {'code': 'invalid_parameter', 'message': 'error(s) occurred while compiling module(s)', 'errors': [{'code': 'rego_type_error'...
                                                                              │        └ 400
                                                                              └ <starlette.responses.Response object at 0x7f7f9e58bac0>

I tried to have a clean setup by

docker system prune
docker rm -f $(docker ps -a -q)  # Delete all containers
docker volume rm $(docker volume ls -q)  # Delete all volumes

and run again, but still same. Any idea? thanks!

How to turn on the logs?

I saw some code in the provider file.

from opal_common.logger import logger
logger.debug('...')

I also tried to add some

print('...')

However, they never got printed after I docker compose up --build.

How can I turn on the logs? Thanks!

Update data

is there a way to update data in the postgres db using the opal server api?

opal client error

Hi OPAL team, hmm, I started to get this error today for this repo. Any idea? Thanks!

➜ docker-compose up

[+] Building 13.2s (10/10) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 31B                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/authorizon/opal-client:latest                                                                                                                                                                                 3.2s
 => [1/5] FROM docker.io/authorizon/opal-client:latest@sha256:1da4cc56616d777cef36d26de4d82916fc392afeba2aae5519afea19c7dccbcf                                                                                                                           0.0s
 => [internal] load build context                                                                                                                                                                                                                        0.1s
 => => transferring context: 1.30MB                                                                                                                                                                                                                      0.1s
 => CACHED [2/5] WORKDIR /app/                                                                                                                                                                                                                           0.0s
 => [3/5] COPY . ./                                                                                                                                                                                                                                      0.1s
 => [4/5] RUN python setup.py install                                                                                                                                                                                                                    1.5s
 => [5/5] RUN pip install wait-for-it                                                                                                                                                                                                                    8.1s
 => exporting to image                                                                                                                                                                                                                                   0.1s
 => => exporting layers                                                                                                                                                                                                                                  0.1s
 => => writing image sha256:4b5659aa6f2b54c312ba3b93c55bc0d3c509181ef872aff0847ab07c2d7ba52b                                                                                                                                                             0.0s
 => => naming to docker.io/library/opal-fetcher-postgres_opal_client                                                                                                                                                                                     0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 4/4
 ⠿ Container opal-fetcher-postgres_broadcast_channel_1  Created                                                                                                                                                                                          0.0s
 ⠿ Container opal-fetcher-postgres_example_db_1         Created                                                                                                                                                                                          0.0s
 ⠿ Container opal-fetcher-postgres_opal_server_1        Created                                                                                                                                                                                          0.0s
 ⠿ Container opal-fetcher-postgres_opal_client_1        Recreated                                                                                                                                                                                        0.1s
Attaching to broadcast_channel_1, example_db_1, opal_client_1, opal_server_1
broadcast_channel_1  |
broadcast_channel_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
broadcast_channel_1  |
broadcast_channel_1  | 2021-09-09 17:30:19.918 UTC [1] LOG:  starting PostgreSQL 13.4 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20210424) 10.3.1 20210424, 64-bit
broadcast_channel_1  | 2021-09-09 17:30:19.918 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
broadcast_channel_1  | 2021-09-09 17:30:19.918 UTC [1] LOG:  listening on IPv6 address "::", port 5432
broadcast_channel_1  | 2021-09-09 17:30:19.921 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
broadcast_channel_1  | 2021-09-09 17:30:19.924 UTC [21] LOG:  database system was shut down at 2021-09-09 17:29:57 UTC
broadcast_channel_1  | 2021-09-09 17:30:19.937 UTC [1] LOG:  database system is ready to accept connections
example_db_1         |
example_db_1         | PostgreSQL Database directory appears to contain a database; Skipping initialization
example_db_1         |
example_db_1         | 2021-09-09 17:30:20.076 UTC [1] LOG:  starting PostgreSQL 13.4 (Debian 13.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
example_db_1         | 2021-09-09 17:30:20.076 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
example_db_1         | 2021-09-09 17:30:20.076 UTC [1] LOG:  listening on IPv6 address "::", port 5432
example_db_1         | 2021-09-09 17:30:20.079 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
example_db_1         | 2021-09-09 17:30:20.084 UTC [27] LOG:  database system was shut down at 2021-09-09 17:29:57 UTC
example_db_1         | 2021-09-09 17:30:20.090 UTC [1] LOG:  database system is ready to accept connections
opal_server_1        | [2021-09-09 17:30:21 +0000] [1] [INFO] Starting gunicorn 20.1.0
opal_server_1        | [2021-09-09 17:30:21 +0000] [1] [INFO] Listening at: http://0.0.0.0:7002 (1)
opal_server_1        | [2021-09-09 17:30:21 +0000] [1] [INFO] Using worker: uvicorn.workers.UvicornWorker
opal_server_1        | [2021-09-09 17:30:21 +0000] [9] [INFO] Booting worker with pid: 9
opal_server_1        | [2021-09-09 17:30:21 +0000] [10] [INFO] Booting worker with pid: 10
opal_server_1        | [2021-09-09 17:30:21 +0000] [11] [INFO] Booting worker with pid: 11
opal_server_1        | [2021-09-09 17:30:21 +0000] [12] [INFO] Booting worker with pid: 12
opal_server_1        | 2021-09-09T17:30:21.657897+0000 |opal_common.authentication.signer       | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server_1        | 2021-09-09T17:30:21.709369+0000 |opal_common.authentication.signer       | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server_1        | 2021-09-09T17:30:21.736570+0000 |opal_server.server                      | INFO  | triggered startup event
opal_server_1        | 2021-09-09T17:30:21.736989+0000 |opal_common.topics.publisher            | INFO  | started topic publisher
opal_server_1        | 2021-09-09T17:30:21.737376+0000 |opal_server.server                      | INFO  | leadership lock acquired, leader pid: 9
opal_server_1        | 2021-09-09T17:30:21.737518+0000 |opal_server.server                      | INFO  | listening on webhook topic: 'webhook'
opal_server_1        | 2021-09-09T17:30:21.738816+0000 |fastapi_websocket_pubsub.event_notifier | INFO  | New subscription {'id': '41deef66f4714d4da2edec6ef1078b0b', 'subscriber_id': '9080fb2e6a82434095246fa9cda352b7', 'topic': 'webhook', 'callback': functools.partial(<function trigger_repo_watcher_pull at 0x7fc532fb0310>, <opal_server.policy.watcher.task.RepoWatcherTask object at 0x7fc532fb3640>), 'notifier_id': None}
opal_server_1        | 2021-09-09T17:30:21.739108+0000 |opal_server.policy.watcher.task         | INFO  | Launching repo watcher
opal_server_1        | 2021-09-09T17:30:21.739778+0000 |opal_common.git.repo_cloner             | INFO  | Cloning repo from 'https://github.com/authorizon/opal-example-policy-repo' to '/regoclone'
opal_server_1        | 2021-09-09T17:30:21.739956+0000 |opal_common.git.repo_cloner             | INFO  | Repo already exists in '/regoclone'
opal_server_1        | 2021-09-09T17:30:21.800354+0000 |opal_server.server                      | INFO  | triggered startup event
opal_server_1        | 2021-09-09T17:30:21.800808+0000 |opal_common.topics.publisher            | INFO  | started topic publisher
opal_server_1        | 2021-09-09T17:30:21.854116+0000 |opal_common.authentication.signer       | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server_1        | 2021-09-09T17:30:21.906423+0000 |opal_server.server                      | INFO  | triggered startup event
opal_server_1        | 2021-09-09T17:30:21.906625+0000 |opal_common.topics.publisher            | INFO  | started topic publisher
opal_server_1        | 2021-09-09T17:30:21.956982+0000 |opal_common.authentication.signer       | INFO  | OPAL was not provided with JWT encryption keys, cannot verify api requests!
opal_server_1        | 2021-09-09T17:30:22.013798+0000 |opal_server.server                      | INFO  | triggered startup event
opal_server_1        | 2021-09-09T17:30:22.014042+0000 |opal_common.topics.publisher            | INFO  | started topic publisher
opal_client_1        | [2021-09-09 17:30:22 +0000] [9] [INFO] Starting gunicorn 20.1.0
opal_client_1        | [2021-09-09 17:30:22 +0000] [9] [INFO] Listening at: http://0.0.0.0:7000 (9)
opal_client_1        | [2021-09-09 17:30:22 +0000] [9] [INFO] Using worker: uvicorn.workers.UvicornWorker
opal_client_1        | [2021-09-09 17:30:22 +0000] [11] [INFO] Booting worker with pid: 11
opal_client_1        | 2021-09-09T17:30:22.776529+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'FastApiRpcFetchProvider' found at: <class 'opal_common.fetcher.providers.fastapi_rpc_fetch_provider.FastApiRpcFetchProvider'>
opal_client_1        | 2021-09-09T17:30:22.776886+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'HttpFetchProvider' found at: <class 'opal_common.fetcher.providers.http_fetch_provider.HttpFetchProvider'>
opal_client_1        | 2021-09-09T17:30:22.802421+0000 |opal_common.fetcher.fetcher_register    | INFO  | Loading FetcherProvider 'PostgresFetchProvider' found at: <class 'opal_fetcher_postgres.provider.PostgresFetchProvider'>
opal_client_1        | 2021-09-09T17:30:22.802631+0000 |opal_common.fetcher.fetcher_register    | INFO  | Fetcher Register loaded
opal_client_1        | 2021-09-09T17:30:22.825239+0000 |opal_client.opa.runner                  | INFO  | Launching opa runner
opal_client_1        | 2021-09-09T17:30:22.826799+0000 |opal_client.opa.runner                  | INFO  | Running OPA inline: opa run --server --addr=:8181 --authentication=off --authorization=off --log-level=info
opal_client_1        | 2021-09-09T17:30:22.842910+0000 |opal_client.opa.logger                  | INFO  | Initializing server. {'addrs': [':8181'], 'diagnostic-addrs': [], 'time': '2021-09-09T17:30:22Z'}
opal_server_1        | 2021-09-09T17:30:23.451689+0000 |opal_common.git.repo_watcher            | INFO  | Launching polling task, interval: 30 seconds
opal_client_1        | 2021-09-09T17:30:23.834381+0000 |opal_client.opa.runner                  | INFO  | Running OPA initial start callbacks
opal_client_1        | 2021-09-09T17:30:23.834992+0000 |opal_client.data.updater                | INFO  | Launching data updater
opal_client_1        | 2021-09-09T17:30:23.835351+0000 |opal_client.policy.updater              | INFO  | Launching policy updater
opal_client_1        | 2021-09-09T17:30:23.835596+0000 |opal_client.data.updater                | INFO  | Subscribing to topics: ['policy_data']
opal_client_1        | 2021-09-09T17:30:23.836373+0000 |opal_client.policy.updater              | INFO  | Subscribing to topics: ['policy:.']
opal_client_1        | 2021-09-09T17:30:23.837027+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Trying to connect to Pub/Sub server - ws://opal_server:7002/ws
opal_client_1        | 2021-09-09T17:30:23.837917+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
opal_client_1        | 2021-09-09T17:30:23.843181+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Trying to connect to Pub/Sub server - ws://opal_server:7002/ws
opal_client_1        | 2021-09-09T17:30:23.843570+0000 |fastapi_websocket_rpc.websocket_rpc_c...| INFO  | Trying server - ws://opal_server:7002/ws
opal_server_1        | 2021-09-09T17:30:23.849333+0000 |fastapi_websocket_pubsub.event_broadc...| INFO  | Listening for incoming events from broadcast channel (first listener started)
opal_server_1        | 2021-09-09T17:30:23.849735+0000 |fastapi_websocket_pubsub.event_broadc...| INFO  | Spawning broadcast listen task
opal_server_1        | 2021-09-09T17:30:23.874777+0000 |fastapi_websocket_pubsub.event_broadc...| INFO  | Subscribing to ALL TOPICS, and sharing messages with broadcast channel
opal_server_1        | 2021-09-09T17:30:23.875580+0000 |fastapi_websocket_pubsub.event_notifier | INFO  | New subscription {'id': '1d521aff17ce49cba19c32833ff51f7c', 'subscriber_id': '95cfad20c966449c93fd01fe5eae0fae', 'topic': '__EventNotifier_ALL_TOPICS__', 'callback': <bound method EventBroadcaster.__broadcast_notifications__ of <fastapi_websocket_pubsub.event_broadcaster.EventBroadcaster object at 0x7fc532fb7580>>, 'notifier_id': None}
opal_server_1        | 2021-09-09T17:30:23.876083+0000 |fastapi_websocket_rpc.websocket_rpc_e...| INFO  | Client connected
opal_server_1        | 2021-09-09T17:30:23.876830+0000 |fastapi_websocket_pubsub.event_broadc...| INFO  | Starting broadcaster listener
opal_server_1        | 2021-09-09T17:30:23.894454+0000 |fastapi_websocket_rpc.websocket_rpc_e...| INFO  | Client connected
opal_server_1        | 2021-09-09T17:30:23.904230+0000 |fastapi_websocket_pubsub.event_notifier | INFO  | New subscription {'id': 'c7e4128addde41f9b41e78d6e21bb9c7', 'subscriber_id': '96aa370defc74f14ab2ace2ee874faa3', 'topic': 'policy_data', 'callback': <function RpcEventServerMethods.subscribe.<locals>.callback at 0x7fc532a8fe50>, 'notifier_id': None}
opal_client_1        | 2021-09-09T17:30:23.908319+0000 |opal_client.data.updater                | INFO  | Connected to server
opal_client_1        | 2021-09-09T17:30:23.908431+0000 |opal_client.data.updater                | INFO  | Performing data configuration, reason: Initial load
opal_server_1        | 2021-09-09T17:30:23.908219+0000 |fastapi_websocket_pubsub.event_notifier | INFO  | New subscription {'id': '17bb79917a5e4d478fbd707a1fd535e5', 'subscriber_id': '3b5a87e12519497ebce614d10932d49b', 'topic': 'policy:.', 'callback': <function RpcEventServerMethods.subscribe.<locals>.callback at 0x7fc532a8fdc0>, 'notifier_id': None}
opal_client_1        | 2021-09-09T17:30:23.908553+0000 |opal_client.data.updater                | INFO  | Getting data-sources configuration from 'http://opal_server:7002/data/config'
opal_client_1        | 2021-09-09T17:30:23.911477+0000 |opal_client.policy.updater              | INFO  | Connected to server
opal_client_1        | 2021-09-09T17:30:23.911853+0000 |opal_client.policy.updater              | INFO  | Refetching policy code (full bundle)
opal_server_1        | 2021-09-09T17:30:23.914931+0000 |opal_server.data.api                    | INFO  | Serving source configuration
opal_server_1        | 2021-09-09T17:30:23.915966+0000 |uvicorn.protocols.http.httptools_impl   | INFO  | 172.25.0.5:60762 - "GET /data/config HTTP/1.1" 200
opal_client_1        | 2021-09-09T17:30:23.917804+0000 |opal_client.data.updater                | INFO  | Triggering data update with id: ca7697c68c3f4c489bd4bc663a0eb82d
opal_client_1        | 2021-09-09T17:30:23.918044+0000 |opal_client.data.updater                | INFO  | Fetching policy data
opal_client_1        | 2021-09-09T17:30:23.918305+0000 |opal_client.data.fetcher                | INFO  | Fetching data from url: postgresql://postgres@example_db:5432/postgres
opal_server_1        | 2021-09-09T17:30:23.936952+0000 |uvicorn.protocols.http.httptools_impl   | INFO  | 172.25.0.5:60764 - "GET /policy?path=. HTTP/1.1" 200
opal_client_1        | 2021-09-09T17:30:23.936638+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Connected to PubSub server ws://opal_server:7002/ws
opal_client_1        | 2021-09-09T17:30:23.939081+0000 |opal_client.policy.updater              | INFO  | got policy bundle, commit hash: '6d849b1ce92bae05c31146470e66e07c3c41b164'
opal_client_1        | 2021-09-09T17:30:23.942061+0000 |opal_client.opa.logger                  | INFO  | Received request.    GET /v1/policies
opal_client_1        | 2021-09-09T17:30:23.942817+0000 |opal_client.opa.logger                  | INFO  | Sent response.       GET /v1/policies -> 200
opal_client_1        | 2021-09-09T17:30:23.944976+0000 |opal_client.opa.logger                  | INFO  | Received request.    PUT /v1/data
opal_client_1        | 2021-09-09T17:30:23.945914+0000 |opal_client.opa.logger                  | INFO  | Sent response.       PUT /v1/data -> 204
example_db_1         | 2021-09-09 17:30:23.948 UTC [35] ERROR:  relation "city" does not exist at character 15
example_db_1         | 2021-09-09 17:30:23.948 UTC [35] STATEMENT:  SELECT * from city;
opal_client_1        | 2021-09-09T17:30:23.948788+0000 |opal_client.opa.logger                  | INFO  | Received request.    PUT /v1/policies/rbac.rego
opal_client_1        | 2021-09-09T17:30:23.956322+0000 |opal_client.opa.logger                  | INFO  | Sent response.       PUT /v1/policies/rbac.rego -> 200
opal_client_1        | 2021-09-09T17:30:23.958569+0000 |opal_client.opa.logger                  | INFO  | Received request.    PUT /v1/policies/utils.rego
opal_client_1        | 2021-09-09T17:30:23.961442+0000 |opal_client.opa.logger                  | INFO  | Sent response.       PUT /v1/policies/utils.rego -> 200
opal_client_1        | 2021-09-09T17:30:23.962971+0000 |fastapi_websocket_pubsub.pub_sub_client | INFO  | Connected to PubSub server ws://opal_server:7002/ws
example_db_1         | 2021-09-09 17:30:24.421 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:30:24.421 UTC [35] STATEMENT:  SELECT * from city;
example_db_1         | 2021-09-09 17:30:25.058 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:30:25.058 UTC [35] STATEMENT:  SELECT * from city;
example_db_1         | 2021-09-09 17:30:27.559 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:30:27.559 UTC [35] STATEMENT:  SELECT * from city;
opal_client_1        | 2021-09-09T17:30:33.938164+0000 |opal_client.data.fetcher                |ERROR  | Timeout while fetching url: postgresql://postgres@example_db:5432/postgres
opal_client_1        | Traceback (most recent call last):
opal_client_1        |
opal_client_1        |   File "/root/.local/bin/gunicorn", line 8, in <module>
opal_client_1        |     sys.exit(run())
opal_client_1        |     │   │    └ <function run at 0x7f4ef9412940>
opal_client_1        |     │   └ <built-in function exit>
opal_client_1        |     └ <module 'sys' (built-in)>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 67, in run
opal_client_1        |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
opal_client_1        |     └ <class 'gunicorn.app.wsgiapp.WSGIApplication'>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 231, in run
opal_client_1        |     super().run()
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 72, in run
opal_client_1        |     Arbiter(self).run()
opal_client_1        |     │       └ <gunicorn.app.wsgiapp.WSGIApplication object at 0x7f4ef93ed7f0>
opal_client_1        |     └ <class 'gunicorn.arbiter.Arbiter'>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 202, in run
opal_client_1        |     self.manage_workers()
opal_client_1        |     │    └ <function Arbiter.manage_workers at 0x7f4ef8d22160>
opal_client_1        |     └ <gunicorn.arbiter.Arbiter object at 0x7f4ef93ed910>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 551, in manage_workers
opal_client_1        |     self.spawn_workers()
opal_client_1        |     │    └ <function Arbiter.spawn_workers at 0x7f4ef8d22280>
opal_client_1        |     └ <gunicorn.arbiter.Arbiter object at 0x7f4ef93ed910>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 622, in spawn_workers
opal_client_1        |     self.spawn_worker()
opal_client_1        |     │    └ <function Arbiter.spawn_worker at 0x7f4ef8d221f0>
opal_client_1        |     └ <gunicorn.arbiter.Arbiter object at 0x7f4ef93ed910>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
opal_client_1        |     worker.init_process()
opal_client_1        |     │      └ <function UvicornWorker.init_process at 0x7f4ef83fdca0>
opal_client_1        |     └ <uvicorn.workers.UvicornWorker object at 0x7f4ef83ff130>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 64, in init_process
opal_client_1        |     super(UvicornWorker, self).init_process()
opal_client_1        |           │              └ <uvicorn.workers.UvicornWorker object at 0x7f4ef83ff130>
opal_client_1        |           └ <class 'uvicorn.workers.UvicornWorker'>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 142, in init_process
opal_client_1        |     self.run()
opal_client_1        |     │    └ <function UvicornWorker.run at 0x7f4ef83fddc0>
opal_client_1        |     └ <uvicorn.workers.UvicornWorker object at 0x7f4ef83ff130>
opal_client_1        |   File "/root/.local/lib/python3.8/site-packages/uvicorn/workers.py", line 77, in run
opal_client_1        |     loop.run_until_complete(server.serve(sockets=self.sockets))
opal_client_1        |     │    │                  │      │             │    └ [<gunicorn.sock.TCPSocket object at 0x7f4ef83eef40>]
opal_client_1        |     │    │                  │      │             └ <uvicorn.workers.UvicornWorker object at 0x7f4ef83ff130>
opal_client_1        |     │    │                  │      └ <function Server.serve at 0x7f4ef8631d30>
opal_client_1        |     │    │                  └ <uvicorn.server.Server object at 0x7f4ef88f94c0>
opal_client_1        |     │    └ <method 'run_until_complete' of 'uvloop.loop.Loop' objects>
opal_client_1        |     └ <uvloop.Loop running=True closed=False debug=False>
opal_client_1        | > File "/usr/local/lib/python3.8/site-packages/opal_client-0.1.13-py3.8.egg/opal_client/data/fetcher.py", line 63, in handle_url
opal_client_1        |     response = await self._engine.handle_url(url, config=config)
opal_client_1        |                      │    │       │          │           └ {'fetcher': 'PostgresFetchProvider', 'query': 'SELECT * from city;', 'connection_params': {'password': 'postgres'}}
opal_client_1        |                      │    │       │          └ 'postgresql://postgres@example_db:5432/postgres'
opal_client_1        |                      │    │       └ <function FetchingEngine.handle_url at 0x7f4ef736f310>
opal_client_1        |                      │    └ <opal_common.fetcher.engine.fetching_engine.FetchingEngine object at 0x7f4ef88ed8e0>
opal_client_1        |                      └ <opal_client.data.fetcher.DataFetcher object at 0x7f4ef72fce20>
opal_client_1        |   File "/usr/local/lib/python3.8/site-packages/opal_common-0.1.13-py3.8.egg/opal_common/fetcher/engine/fetching_engine.py", line 104, in handle_url
opal_client_1        |     await asyncio.wait_for(wait_event.wait(), timeout)
opal_client_1        |           │       │        │          │       └ 10
opal_client_1        |           │       │        │          └ <function Event.wait at 0x7f4ef8760040>
opal_client_1        |           │       │        └ <asyncio.locks.Event object at 0x7f4ef6f09460 [unset]>
opal_client_1        |           │       └ <function wait_for at 0x7f4ef8769940>
opal_client_1        |           └ <module 'asyncio' from '/usr/local/lib/python3.8/asyncio/__init__.py'>
opal_client_1        |   File "/usr/local/lib/python3.8/asyncio/tasks.py", line 501, in wait_for
opal_client_1        |     raise exceptions.TimeoutError()
opal_client_1        |           │          └ <class 'asyncio.exceptions.TimeoutError'>
opal_client_1        |           └ <module 'asyncio.exceptions' from '/usr/local/lib/python3.8/asyncio/exceptions.py'>
opal_client_1        |
opal_client_1        | asyncio.exceptions.TimeoutError
opal_client_1        | 2021-09-09T17:30:33.949107+0000 |opal_client.data.updater                |ERROR  | Failed to fetch url postgresql://postgres@example_db:5432/postgres, got exception:
example_db_1         | 2021-09-09 17:30:35.103 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:30:35.103 UTC [35] STATEMENT:  SELECT * from city;
example_db_1         | 2021-09-09 17:30:37.384 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:30:37.384 UTC [35] STATEMENT:  SELECT * from city;
opal_server_1        | 2021-09-09T17:30:53.428806+0000 |opal_common.git.repo_watcher            | INFO  | Pulling changes from remote: 'origin'
opal_server_1        | 2021-09-09T17:30:55.551686+0000 |opal_common.git.repo_watcher            | INFO  | No new commits: HEAD is at '6d849b1ce92bae05c31146470e66e07c3c41b164'
example_db_1         | 2021-09-09 17:31:08.056 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:31:08.056 UTC [35] STATEMENT:  SELECT * from city;
opal_server_1        | 2021-09-09T17:31:25.531347+0000 |opal_common.git.repo_watcher            | INFO  | Pulling changes from remote: 'origin'
opal_server_1        | 2021-09-09T17:31:27.180727+0000 |opal_common.git.repo_watcher            | INFO  | No new commits: HEAD is at '6d849b1ce92bae05c31146470e66e07c3c41b164'
opal_client_1        | 2021-09-09T17:31:32.875584+0000 |opal_client.opa.logger                  | INFO  | Received request.    GET /v1/data
opal_client_1        | 2021-09-09T17:31:32.876693+0000 |opal_client.opa.logger                  | INFO  | Sent response.       GET /v1/data -> 200
example_db_1         | 2021-09-09 17:31:45.503 UTC [35] ERROR:  current transaction is aborted, commands ignored until end of transaction block
example_db_1         | 2021-09-09 17:31:45.503 UTC [35] STATEMENT:  SELECT * from city;
opal_server_1        | 2021-09-09T17:31:57.159742+0000 |opal_common.git.repo_watcher            | INFO  | Pulling changes from remote: 'origin'
opal_server_1        | 2021-09-09T17:31:58.816467+0000 |opal_common.git.repo_watcher            | INFO  | No new commits: HEAD is at '6d849b1ce92bae05c31146470e66e07c3c41b164'

postgres fetch request times out for some passwords due to underlying library asyncpg

Requests to a postgres data source can fail if a data source config entry's url uses the format postgresql://<user>:<password>@<host>/<db>, specifically the password field may fail to be parsed by the underlying library asyncpg. There's an issue in the asyncpg repo which helped me figure out that resolving this (if not using the connection_params field) requires changing the format to postgresql://<host>/<db>?user=<user>&password=<password> as referenced in this comment.

No code changes necessary but I think noting that bug in the docs could potentially save other folks some debugging time

database update

when I update the databases data it is not correct the values of OPA to the latest values

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.