GCP firestore connection problem about daq HOT 13 CLOSED

On Thu, Sep 13, 2018 at 11:32 PM Francesco Anselmo ***@***.***> wrote: After the last source code pull, I run into this problem when connecting to the GCP app sudo cmd/run -s Loading container configuration from /home/francesco/Code/faucetsdn/daq/local//system.conf 0.8.1: Pulling from daqf/runner Digest: sha256:ae436ab18d1c922d3a12e0ba112be8bac01e7e20297ff47bb95db542d8f98a2c Status: Image is up to date for daqf/runner:0.8.1 Configuring apparmor... sudo: apparmor_parser: command not found Starting daqf/runner:0.8.1 -s... Release version 0.8.1 Loading daq run configuration from /root/daq/local/system.conf cleanup='echo cleanup' cmdarg= gcp_cred=local/bos-daq-testing-cf03a659eff0.json network_config=/root/daq/misc/faucet.yaml site_description='Arup DAQ Testing - Francesco - Debian Stretch' value=-s - Starting Docker: docker ...done. ovsdb-server is not running ovs-vswitchd is not running - /etc/openvswitch/conf.db does not exist - Creating empty database /etc/openvswitch/conf.db - Starting ovsdb-server - system ID not configured, please use --system-id - Configuring Open vSwitch system IDs - Starting ovs-vswitchd - Enabling remote OVSDB managers Loading faux configuraiton from /root/daq/local/system.conf Launching faux ... DAQ autoclean docker kill daq-faux Removing old interface faux Adding new interface to 300... Done with faux device launch. Using python exe /usr/bin/python Executing python -s... INFO:daq:configuration map: {'single_shot': True, 'site_description': '"Arup DAQ Testing - Francesco - Debian Stretch"', 'gcp_cred': 'local/bos-daq-testing-cf03a659eff0.json'} INFO:daq:pid is 385 INFO:gcp:Loading gcp credentials from local/bos-daq-testing-cf03a659eff0.json INFO:gcp:Initialized gcp pub/sub bos-daq-testing:daq-arup INFO:gcp:Initialized gcp firestore bos-daq-testing:daq-arup Exception in thread Thread-MonitorBatchPublisher: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", line 239, in monitor return self._commit() File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", line 204, in _commit self._messages, File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/gapic/publisher_client.py", line 325, in publish return self._publish(request, retry=retry, timeout=timeout) File "/usr/local/lib/python2.7/dist-packages/google/api_core/gapic_v1/method.py", line 139, in *call* return wrapped_func(*args, **kwargs) File "/usr/local/lib/python2.7/dist-packages/google/api_core/retry.py", line 260, in retry_wrapped_func on_error=on_error, File "/usr/local/lib/python2.7/dist-packages/google/api_core/retry.py", line 195, in retry_target last_exc) File "/usr/lib/python2.7/dist-packages/six.py", line 718, in raise_from raise value RetryError: Deadline of 600.0s exceeded while calling <functools.partial object at 0x7f0685507208>, last exception: 504 Deadline Exceeded It looks like a timeout related problem, but would like to have some help debugging it. Thanks! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHjwUZ0I40t6RYD8SLCNbZh8PnuEpaks5ua01kgaJpZM4WosQa> .

It fails consistently on my Debian VM, but I need to try a different machine to tell with certainty. All other applications using a connection work (apt-get and browsing is fast speed). I'll try again when back to my computers and report. On 14 Sep 2018 at 1:22 pm, <Trevor ***@***.***>> wrote: Is it consistent, like it always fails, or only sometimes? Anything weird about your local network connection (e.g. super slow, or having other problems)? I have seen things like this before when on a super-slow connection (like on the airplane). On Thu, Sep 13, 2018 at 11:32 PM Francesco Anselmo < ***@***.*** > wrote: > After the last source code pull, I run into this problem when connecting > to the GCP app > > sudo cmd/run -s > > Loading container configuration from > /home/francesco/Code/faucetsdn/daq/local//system.conf > 0.8.1: Pulling from daqf/runner > Digest: > sha256:ae436ab18d1c922d3a12e0ba112be8bac01e7e20297ff47bb95db542d8f98a2c > Status: Image is up to date for daqf/runner:0.8.1 > Configuring apparmor... > sudo: apparmor_parser: command not found > Starting daqf/runner:0.8.1 -s... > Release version 0.8.1 > Loading daq run configuration from /root/daq/local/system.conf > cleanup='echo cleanup' > cmdarg= > gcp_cred=local/bos-daq-testing-cf03a659eff0.json > network_config=/root/daq/misc/faucet.yaml > site_description='Arup DAQ Testing - Francesco - Debian Stretch' > value=-s > > - Starting Docker: docker > ...done. > ovsdb-server is not running > ovs-vswitchd is not running > - /etc/openvswitch/conf.db does not exist > - Creating empty database /etc/openvswitch/conf.db > - Starting ovsdb-server > - system ID not configured, please use --system-id > - Configuring Open vSwitch system IDs > - Starting ovs-vswitchd > - Enabling remote OVSDB managers > Loading faux configuraiton from /root/daq/local/system.conf > Launching faux ... > DAQ autoclean docker kill daq-faux > Removing old interface faux > Adding new interface to 300... > Done with faux device launch. > Using python exe /usr/bin/python > Executing python -s... > INFO:daq:configuration map: {'single_shot': True, 'site_description': > '"Arup DAQ Testing - Francesco - Debian Stretch"', 'gcp_cred': > 'local/bos-daq-testing-cf03a659eff0.json'} > INFO:daq:pid is 385 > INFO:gcp:Loading gcp credentials from > local/bos-daq-testing-cf03a659eff0.json > INFO:gcp:Initialized gcp pub/sub bos-daq-testing:daq-arup > INFO:gcp:Initialized gcp firestore bos-daq-testing:daq-arup > Exception in thread Thread-MonitorBatchPublisher: > Traceback (most recent call last): > File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner > self.run() > File "/usr/lib/python2.7/threading.py", line 754, in run > self.__target(*self.__args, **self.__kwargs) > File > "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", > line 239, in monitor > return self._commit() > File > "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", > line 204, in _commit > self._messages, > File > "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/gapic/publisher_client.py", > line 325, in publish > return self._publish(request, retry=retry, timeout=timeout) > File > "/usr/local/lib/python2.7/dist-packages/google/api_core/gapic_v1/method.py", > line 139, in *call* > return wrapped_func(*args, **kwargs) > File > "/usr/local/lib/python2.7/dist-packages/google/api_core/retry.py", line > 260, in retry_wrapped_func > on_error=on_error, > File > "/usr/local/lib/python2.7/dist-packages/google/api_core/retry.py", line > 195, in retry_target > last_exc) > File "/usr/lib/python2.7/dist-packages/six.py", line 718, in raise_from > raise value > RetryError: Deadline of 600.0s exceeded while calling > <functools.partial object at 0x7f0685507208>, last exception: 504 Deadline > Exceeded > > It looks like a timeout related problem, but would like to have some help > debugging it. > Thanks! > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#18>, or mute the thread > < https://github.com/notifications/unsubscribe-auth/ABBHjwUZ0I40t6RYD8SLCNbZh8PnuEpaks5ua01kgaJpZM4WosQa > > . > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread < https://github.com/notifications/unsubscribe-auth/ABdnwz6aODm_jqwEZKqGLQXz8V9Skyyoks5ua5FdgaJpZM4WosQa > . — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHjwEpiXeqA0SIOQhHXXXGYEcpQws6ks5ua6C1gaJpZM4WosQa> .

On Fri, Sep 21, 2018 at 9:31 AM Francesco Anselmo ***@***.***> wrote: Sorry to open this again. Just did a git pull and rebuilt. cmd/run -s runs well and manages to show the test status on the firebase app, but cmd/run local -s stops with the following error ... INFO:daq:pid is 4291 INFO:gcp:Loading gcp credentials from local/bos-daq-testing-cf03a659eff0.json INFO:gcp:Initialized gcp pub/sub bos-daq-testing:daq-arup INFO:gcp:Initialized gcp firestore bos-daq-testing:daq-arup Traceback (most recent call last): File "daq/daq.py", line 118, in sys.exit(_execute()) File "daq/daq.py", line 103, in _execute daq_runner = runner.DAQRunner(config) File "/home/francesco/Code/daq/daq/runner.py", line 33, in *init* self.gcp = gcp.GcpManager(self.config) File "/home/francesco/Code/daq/daq/gcp.py", line 38, in *init* self._ensure_report_bucket() File "/home/francesco/Code/daq/daq/gcp.py", line 77, in _ensure_report_bucket if self._storage.lookup_bucket(bucket_name): File "/home/francesco/Code/daq/venv/lib/python3.5/site-packages/google/cloud/storage/client.py", line 245, in lookup_bucket return self.get_bucket(bucket_name) File "/home/francesco/Code/daq/venv/lib/python3.5/site-packages/google/cloud/storage/client.py", line 225, in get_bucket bucket.reload(client=self) File "/home/francesco/Code/daq/venv/lib/python3.5/site-packages/google/cloud/storage/_helpers.py", line 108, in reload _target_object=self) File "/home/francesco/Code/daq/venv/lib/python3.5/site-packages/google/cloud/_http.py", line 293, in api_request raise exceptions.from_http_response(response) google.api_core.exceptions.Forbidden: 403 GET https://www.googleapis.com/storage/v1/b/bos-daq-testing.appspot.com?projection=noAcl: ***@***.*** does not have storage.buckets.get access to bos-daq-testing.appspot.com. Done with run, exit 1 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHjzXL0A-abg7md-q-ubzFZ5chiYYoks5udRRmgaJpZM4WosQa> .

On Fri, Sep 21, 2018 at 11:21 AM Francesco Anselmo ***@***.***> wrote: Thanks, I've spent some time trying to figure this out. On the bos-daq-testing project I don't have permission to add the storage admin and pub/sub publisher roles, can you please add them for me? I also took the opportunity to follow the steps to deploy the firebase app in my personal account. I think I've managed to follow all the steps, but the app only shows "System initialized." cmd/run local -s runs now. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHj-GpanZBPU_bO45gFS04F0bdgy5nks5udS4ngaJpZM4WosQa> .

On Fri, Sep 21, 2018 at 11:22 AM Francesco Anselmo ***@***.***> wrote: https://arup-daq-testing.firebaseapp.com/ is the URL for the app deployed in my personal GCP account in case you'd like to have a look. Thanks for the help! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHj8RWgQR5ovU-pTLkA_H9wwM3Auonks5udS5mgaJpZM4WosQa> .

On Fri, Sep 21, 2018 at 11:55 AM Francesco Anselmo ***@***.***> wrote: Yes, I've a separate path set up with both GCP and Firebase — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBHj8VRKhb6pUeaKmAiCT39JCEFT9M9ks5udTYJgaJpZM4WosQa> .