lowellobservatory / ligmos Goto Github PK

I just added lig.weather.timo.virtualweatherstation as both a schema and example, but I realize that I should probably think about restructuring some of these to be more generic - that means have a lig.weather.virtualweatherstation base schema, and have an optional ".timo" postfix for the topic and probably an additional location/deployment tag in the XML to list the site too. It'd cut down on duplicates going forward, but there's no huge rush.

Tagging Mr. Freeze as a project too since it applies to cryogenic hardware especially, which is handled a bit differently already too.

Missing:

• pandas

Need to add at least 'build-essential' to the Dockerfile's install portion to make sure it's available for building packages from source! Just ran into this with the Pi build.

Right now I'm basically left guessing what died and why, but I'm pretty sure that the information on tags that failed validation is readily available ... I just have to revisit it to see where it is and how I can bring it up towards the surface. This is vitally important for checking for failures due to either a bad schema, a bad measurement in the database, or some combination of the two.

This has got to go; forgot I did this but I should change it in the next iteration of this module

Preparing Mr. Freeze, I'm left with the situation where I really want a much simpler interface to the broker connection handling logic than I have presently.

Right now the best I can do is try to utilize the boilerplate code in ligmos.workers.workerSetup.toServeMan but that's pretty clunky in practice. It'd be nice to have a much simpler interface that just does it all for me that I can import in one place, without having to follow the template I set up long ago for DataServants.

From @astrobokonon on July 9, 2018 19:21

2018-07-09 11:53:51 got an auth. exception from paramiko on alfred, unknown cause but it looks like pings to the site spiked at the same time.

Copied from original issue: LowellObservatory/DataServants#14

Seems like I'm missing at least:

xmltodict
stomp.py
xmlschema

Used to avoid SQL-style injection attacks; I'll need to uproot all of the query construction code to do this, though, so it's a bit messy. Also requires the most up-to-date version of influxdb-python, v5.2.2 or greater.

I tried to commit to a database that hadn't been created yet and I was getting a 404 response from influx; it worked locally, but I don't know if I made the database already locally and then forgot about it, or if something is actually different. I used to have checks if the table existed a while ago, but they were causing a useless CPU spike so I got rid of them.

Ultimately, this just needs to be clearly stated in the documentation, or change the database logic back to check for existance of the table before attempting to write points to it. Not sure which is best!

Need a default listener/parrot class for the amq class here, to facilitate easier debugging and on-the-fly usage of the ligmos library in general.

From @astrobokonon on July 31, 2018 15:56

When the network to the site was down, Iago threw this exception:

2018-07-30 16:55:38,975 ERROR    Could not connect to host [broker], port [port]
Traceback (most recent call last):
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/transport.py", line 724, in attempt_connection
    self.socket = get_socket(host_and_port[0], host_and_port[1], self.__timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/backwardsock26.py", line 16, in get_socket
    return socket.create_connection((host, port), timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out

Then, when I tried to control-C it to kill it, I got the following:

2018-07-30 16:56:50,981 INFO     Got SIGINT; stopping...
2018-07-30 16:56:50,982 ERROR    Could not connect to host [broker], port [port]
Traceback (most recent call last):
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/transport.py", line 724, in attempt_connection
    self.socket = get_socket(host_and_port[0], host_and_port[1], self.__timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/backwardsock26.py", line 16, in get_socket
    return socket.create_connection((host, port), timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
OSError: [Errno 22] Invalid argument
2018-07-30 16:57:02,964 INFO     Got SIGINT; stopping...
2018-07-30 16:57:02,964 ERROR    Could not connect to host [broker], port [port]
Traceback (most recent call last):
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/transport.py", line 724, in attempt_connection
    self.socket = get_socket(host_and_port[0], host_and_port[1], self.__timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/site-packages/stomp/backwardsock26.py", line 16, in get_socket
    return socket.create_connection((host, port), timeout)
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 724, in create_connection
    raise err
  File "/Users/wadsworth/miniconda3/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
OSError: [Errno 22] Invalid argument
2018-07-30 16:57:03,155 INFO     Got SIGINT; stopping...
2018-07-30 16:57:03,315 INFO     Got SIGINT; stopping...
2018-07-30 16:57:03,372 INFO     STOMP.py connection failed!
2018-07-30 16:57:03,372 INFO     PID 92336 is now out of here!

Copied from original issue: LowellObservatory/DataServants#15

For some reason, ligmos.utils.database.influxobj.alterRetention() didn't really ever work. I set things up initially, and changed it from autogen/default to "Hold6w" but when I wanted to further change it to "Hold26w" it never actually changed and I had to go in and do it by hand in the influx console. Needs experimentation, especially since me changing it by hand appeared to nuke the database. Whoops.

Not sure what happened on my end; planned a broker restart and it looks like it noticed it, but didn't actually get packets flowing again even though it said the subscriptions occurred.

Check 2018-07-05 or 2018-07-06 logs for weirdness.

There's been a big change in STOMP.py, basically every ConnectionListener needs to be updated for STOMP.py > v7. Tracking it here instead of everywhere else.

See jasonrbriggs/stomp.py#352

Missing pyserial entirely.

Pandas and numpy as incompatible, at least on a Pi:

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 44 from C header, got 40 from PyObject

Numpy should now be >= 1.20 I think?

Seriously I have like 5 different versions scattered about that all do the same thing in a slightly different way.

This includes stuff in other repositories; there's at least some in NightShift that I can remember. I'm ignoring the ones in Camelot though, since those are fine to be messy and standalone.

Seeing a lot of this sort of thing in the logs lately:

2020-08-27 02:42:29,230 ERROR    Heartbeat timeout: diff_receive=8.00024583202321, time=817049.624345889, lastrec=817041.624100057
2020-08-27 02:43:22,344 INFO     Connection died! Reestablishing...
2020-08-27 02:43:22,344 INFO     Connecting to joe.lowell.edu
2020-08-27 02:43:22,345 INFO     'StompConnection11' object has no attribute 'start'
2020-08-27 02:43:22,345 INFO     Old broker version?
2020-08-27 02:43:22,347 INFO     Connection established. Hooray!
2020-08-27 02:43:22,347 INFO     Subscribing to:
2020-08-27 02:43:22,347 INFO     [big_list_of_topics]

Have to check whether a STOMP.py thing has changed, or if I'm really getting an old broker connection type for some reason. I could have sworn that ActiveMQ 5.15.x was previously connecting via StompConnection12?

Clarify everything and get sphinx output up and running again

Right now, in ligmos.utils.amq.schemaDicter() I'm doing this:

That means I can get into trouble with topic/schema names that have a name delineated with periods, that have the last portion starting with the letter 'v'. I should probably fix that because it's going to silently confuse me or someone else someday pretty soon!

Need to make this a proper package to better handle dependencies!