lowellobservatory / mrfreeze Goto Github PK

Need to make sure that all device queries are wrapped in a try...except block, to take care of the instances where the serial device's reply is clobbered/interrupted by something else heading that way thru the MOXA box. Despite all of my hopes this will happen because someone

• Sends a command in the "old" way using Peter's scripts
• Tries to talk to the device for some other reason
• I screw up and clobber myself

Getting caught by a LOIS log line that I'm not ignoring, but should:

2021-02-22 10:21:53,636 INFO     Unknown log line!
2021-02-22 10:21:53,636 INFO     Lois Log Module Initialized
2021-02-22 10:21:53,636 INFO     {'measurement': 'LMI_arc-loisgettemp', 'tags': {'Device': 'arc-loisgettemp'}, 'fields': {}}
2021-02-22 10:21:53,644 INFO     {"error":"unable to parse 'LMI_arc-loisgettemp,Device=arc-loisgettemp': missing fields"}
2021-02-22 10:21:53,645 INFO     INFLUXDB ERROR. Check above for more details :(

Now that Mr. Freeze is minimally up and running, I should migrate all the temperature data that was collected in the original DCT database into the Mr. Freeze database. I could just drop the DCT database info, but I really should move it over so I at least have notes on how to migrate the data properly in my environment.

As things stand right now, I'm facing an existential question.

"Iago" has been the DCT broker consumer/database storage process for quite some time now. Iago has been handling the instrument temperatures as reported by LOIS, and contains all the logic for parsing those ugly LOIS log files.

Mr. Freeze is rapidly reaching a deploy-able state; as such, I'd like to get all the temperature/telemetry stuff under his umbrella.

That means doing ONE of the following:

• Removing the temperature parsing (from parserLOlogs) and putting it into Mr. Freeze

• Making something (perhaps Abu) repackage the values into a new broker topic

• Something else??

Ideally, this would be fairly transparent and all happen thru the broker. BUT, that means that whomever/whichever client issues the commands needs to know about the active modules that Mr. Freeze has, either because they're all hardcoded or because they're advertised. If it's the former, I wonder just why I have a configuration file in the first place. Seems like it should be the latter - Mr. Freeze/Nora should advertise their active modules once every XX seconds and anyone can listen/discover them.

Should I allow arbitrary injection of whole new modules? Seems possible, but might not be a good idea. The command-issuer necessarily must have some knowledge of what the format should be, but the more I think about this the more I realize that having a hardcoded configuration file is a bit of a breaking

Do what the title says; could either initially schedule them all to happen immediately, and then re-schedule, or call the query commands manually once before the schedule starts. Would greatly speed up debugging/testing, and I think it's a natural expectation of what would happen when the program starts up.

In the current incarnation, all the topics are gathered up in one place at the initial startup as defined in the configuration file. That's fine!

What's less-fine, though is that the listener is set as well, and it'll parse anything/everything regardless of the value of the 'enabled' key in the device configuration.

That means in practice, the listener/parser for things like the LOIS status topics are always on. I don't know yet if that's the preferred behavior. It's probably fine? But I wanted to make a note here to remind myself of this in the future.

Clarify everything and get sphinx output up and running again

Make life easier going forward for dependency management!

Mr. Freeze needs a strict set of commands. This is the place where they will be decided. I suppose this could be thought of as a sort of API.

It's probably a good idea to have flat log files as backup for Mr. Freeze, and ideally per-instrument like we've been doing for a while now.

Off the top of my head, it seems like this should be done using the log file manager and appropriate log levels. But, I don't know how to do that.

Seems like I should do a scan of the configuration file, and set up a log output file per instrument that I find? Once that's done I could probably define a series of log objects that get rotated, etc., but there are many details to figure out still.

This is probably something that will get folded back up the chain to ligmos/DataServants, but it's needed here first.

Either find a way to use the ligmos base listener class, or make the queue concept here another listener that anyone else can use. I'm leaning towards the latter, but I'd still like to think about this more. At least I've gone from one full custom listener per site/instance to just one overall, so maybe two overall with this one tossed in isn't too bad. But this is the remaining blocker before I can dive headfirst into LowellObservatory/ligmos#19.

For a Sunpower GenII cooler, the STATE command returns 265 bytes:

2021-08-13 03:46:11,108 INFO     Good write: b'STATE\r'
2021-08-13 03:46:11,858 INFO     265 bytes recieved in response
2021-08-13 03:46:11,858 INFO     b'STATE\r\nMODE    = 002.00\r\nTSTATM  = 000.00\r\nTSTAT   = 000.00\r\nSSTOPM  = 001.00\r\nSSTOP   = 000.00\r\nPID     = 002.00\r\nLOCK    = 000.00\r\nMAX     = 110.00\r\nMIN     = 050.00\r\nPWOUT   = 000.00\r\nTTARGET = 065.00\r\nTBAND   = 000.50\r\nTEMP KP = 050.00000\r\nTEMP KI = 001.00000\r\n'

For timeout values less than 0.75 sec, I was getting less than 265 bytes - seemed like 0.5 seconds would get ~208 bytes, causing this query to fail because some of it was left on the socket buffer 😞

Nora is hard coded to look for certain config sections like 'broker-dct' and I should either give up the hope of multi databases and/or broker or fix it properly. Not sure which is the better path, but it did just bite me today deploying Mr Freeze out to a non-DCT site.

Need to teach Mr. Freeze about the Varian AGC-100 that we have on the DCT pump cart, which is hooked in to the line pressure. Seems simple enough according to the datasheet but I don't have one on hand to test with except the one at the site.

It's hard to figure out a way to do this in a generic fashion, but it's probably a good idea to start storing the temperature rates of change automatically because doing it in Grafana works only if you don't do any groupby time() stuff. Observe the following two plots, exactly the same except one of them is using the groupby time($__interval) option in the query:

The groupby() query is getting hung up on the query interval, and I can't really get around that that the grafana level from what I've seen so far.