sotamat / sotacat Goto Github PK

Per Brian: I think I need to debug this as it is likely in the Xamarin code.
Duplicate of #3?

If you launch SOTAMAT in SOTACAT mode, you can use the SOTAMAT band/mode selector and the frequency selector to change the band/mode/frequency on the KX radio. Most of the time this works, but frequently making a new selection in SOTAMAT will not update the frequency on the radio, and all subsequent changes will be blocked at the radio. After about 1 minute of hanging the entire sequence of commands that had been buffered during the hang will all play out onto the KX radio.

I do not know the cause of the issue: is the hang on the SOTAMAT side or the SOTACAT side? Instrumenting with debugging console logs could help on the SOTACAT side to see when REST API calls are made and when radio commands are issued.

This requires serial communication with the radio at regular intervals. Due to the RTOS and FT8 timing issues we need some way to mutex and or buffer serial communication with the radio. If the HTML page is polling the radio via serial at the same time that the user does a click-to-pounce REST call, multi-step UART commands would get messed up. The ESP-IDF has a nice RTOS priority model similar to VAX VMS (absolute priority levels, and then shared time within a priority level).

It could be a common occurrence in the field:

1. Use the SOTAcat in conjunction with the radio for a while.
2. QRT, pack up, unplug SOTAcat from radio, but forget to unplug TRS connector also from SOTAcat.
3. SOTAcat remains active, not noticing it's no longer connected to radio
4. Battery drains

Need to ping the radio over the uart from time to time to see it it's still connected. Otherwise, go into deep sleep.

SOTACAT modules based on the Seeed Studio ESP32C3 MCU causes KX2/KX3 radios to lock up if the SOTACAT is connected to the radio when booting.
It appears that the ESP32C3 is emitting a bunch of boot messages into the KX via the UART and that these random characters get the KX into a locked state that can only be fixed by a KX power cycle.

• If the ESP32C3 is booted first, and then attached to the KX radio there is no problem.
• The problem does not seem to happen on Wemos LOLIN32 Lite based ESP32 boards
• I tried using the PlatformIO / Espressif "Run MenuConfig" tool and tried changing "Bootloader Config -> Bootloader Log verbosity -> None" and other variations and this did not solve the issue.
• There are two active UARTs in SOTACAT: the one connected to the USB connector which are sent to the host computer when debugging, and the UART connected to the KX radio. I expect boot messages to go to the USB UART and not the KX UART, especially since the KX UART is not configured in the Firmware to send anything until after booting. So I'm confused what is happening.

If turning off bootloader messages doesn't solve the issue, then I am confused why the KX is locked up. We may want to attach a serial port sniffer between the radio and the KX.

Perhaps deep-sleep is not useful when connected to certain SSIDs, like the home network, where it may be presumed the SOTAcat is plugged in to charge.

We offer this in advance of truly detecting charging state via hardware input.
Meanwhile, it's useful for debugging.

Brian reports:
I'm seeing a bug in SOTAMAT when launched from SOTACAT. It no longer is matching a SOTAMAT frequency / mode pair to the radio's current state. Likely a problem in SOTAMAT but seems like a new issue.

Brian states:
Another regent regression is that the SOTACAT and KX will stop talking to each other after a long period. I think this happens after the SOTACAT reboots due to the prior stack overflow issue/crash because the Blue light comes on solid (meaning no connection to the radio during auto-baud-rate negotiation). The SOTACAT will try forever to connect and fails to do so. But something else is odd: if there is no connection to the radio after booting the device should go to deep sleep after 1 minute. This sleep code works if you turn on your SOTACAT and lay it on your desk without a radio attached. But after a crash reboot the behavior changes. It might be that the light gets left on after a crash. I don’t know. Regardless, the only way to restore operation after this is to turn the radio off and back on again. Unplugging SOTACAT and re-plugging doesn’t help. So something is also happening on the radio side such that it refuses to listen to the serial port at some point.
Possible dup of #3?

When multiple SOTAcats are powered on we can't currently tell which one we're connecting to over WiFi. At the moment this is more of a developer issue, but once they take over the world and we start having name collisions on summits it could become a user issue.

I suggest we have a factory/user settable serial number saved so each SOTAcat has a different default SSID, e.g. SOTAcat-1234.

An extension of this could be making the broadcast SSID completely customizable, but we may still want some default option that includes a serial-number-like value added to a human-readable SSID.

Brian says:
Figuring out how to get Debug and Release builds in PlatformIO should be easy but I don’t know how yet. Set up the code so that LOGI (informational logs) don’t get emitted in release mode.

Enhancement.

Currently it's hard-coded to 1 hour; for testing during certain time periods it's nice to have it be longer, and in the field it may be nice to have it be shorter to only see very recent spots.

Maybe a drop down with a list of options:

15m, 30m, 1h, 2h, 3h

which just changes the duration on the web queries.

Brian says:
I would like a web UI that allows me to at least “read” the state the radio is in (Frequency, Mode, power level, etc.). And then the question is what would be a good UI for “write” and control of the radio from a cell phone beyond click-to-pounce? For example: record / play CQ audio memories # 1 and # 2? View transmit power level? Toggle transmit power from maximum to 0.0 watts and back (for testing) so we don’t have to spin the dial forever? Quick enable/disable of noise reduction? A “smart Tune” that steps up or down by 3KHz and tunes, and then returns to the operating frequency so as to not “step” on conversations? Etc. Be creative.

See commit af8f125

Presently, 1.2.5 of mdns is not compiling. Look out for future compatible versions.

The boot sequence of a newly flashed SOTACAT keeps trying to connect to a KX radio before it allows formation of the WiFi network. This means you can't setup the wifi settings of a newly flashed SOTACAT if it isn't connected to an Elecraft Radio - hangs on boot.

Due to the RTOS and FT8 timing issues we need some way to mutex and or buffer serial communication with the radio. If the HTML page is polling the radio via serial at the same time that the user does a click-to-pounce REST call, multi-step UART commands would get messed up. The ESP-IDF has a nice RTOS priority model similar to VAX VMS (absolute priority levels, and then shared time within a priority level).

Splitting this issue from #8, which will now simply be about the web UI to show radio is connected.

Brian says:
Rather than, or in addition to, OTA updates, there are tools to allow installing firmware directly from a Chrome web browser (Chrome has WebUART ability! Firefox does not). This is a very simple way to allow field updates without all the storage requirements of OTA (which has to store the current and upgraded firmware). While I love this Browser based flashing tool, I can’t seem to get it working even when I flash the different “pages” that PlatformIO builds. https://espressif.github.io/esptool-js/ so we are close but not there. Also, I would want a single unified “package” to flash the entire device, but the ESP Tool is set up to only flash the sub-packages individually. Yuck.

See also #13

Brian says:
OTA updates would be wonderful. However, we are nearly at the 2MB limit and OTA requires 3X the storage. We can double our memory because PlatformIO thinks our board only has 2MB of storage when in fact it has 4MB. We need to twiddle a setting.

Today SOTACAT displays a list of recent Spots for SOTA or POTA.

A feature request would be to see a list of upcoming Alerts (in particular SOTA Alerts) sorted by their time into the future...

We'll need users to be able to set the WiFi credentials for the network they want SOTAcat to connect to. This might be a home network or a phone/hotspot. Since the SOTAcat will serve an AP when it doesn't find a host WiFi to connect to, the user should be able to set their home WiFi credentials via the captive portal.

We may want another issue to track for general user-defined preferences management, and implement this on top of that once it's in place.

Consider implementing more REST API points described in
https://github.com/ki6syd/minimalist-amateur-xcvr/tree/main/api

Per Brian, we may choose to create different entry points since the document is currently MAX-3B specific (for example, it has RxBandwidth setting but not an independent Mode setting, and for a KX radio we want Mode and would rarely want to change RxBandwidth).

Presently, we search for a home wifi in order to act in client mode. After some timeout of non-finding, we revert to acting as an AP beacon.

For some OSes that's great - they connect to SOTAcat and retain their external internet access for calls to figure spots, etc.
On Android, this bi-network situation is somehow precluded. A work-around, though, is to broadcast a hotspot from the phone, and have the SOTAcat connect to that.

That is very convenient, but perhaps not most convenient for debugging (or even data-usage fees), where preferential access to home network might be best.

Here I propose two SSID/password combinations to be searched in order, and specified in a way that is user-describable and saved across sessions in NVRAM.\

Comment?

In the SOTACAT web site you can see a table of recent spots for SOTA or POTA. Those tables show the mode of the spot (SSB, CW, FM, etc.).

This is a feature request to allow users to filter out (hide) any recent spots that have certain modes that the user doesn't want to see. For example, some SSB users might not want to see recent CW spots.

Right now there is prep code, handler code, cleanup code, and a variety of flags controlling FT8 transmission.
Compounding this, we have incoming competing requests to initiate (or re-initiate) FT8 transmission, or cancel it.
As well there is the salient resource: the radio.

See handler-ft8.cpp

Using a Google Pixel 7 running Android 14, the SOTAcat webpage does not load external data about sota and pota spots.
Connection via 192.168.4.1 in Chrome browser works just fine, and the webpage loads. Controls from the webpage back to the phone work as designed. But the chart of spots never populates.

Brian reports that until he added a null gateway to wifi.cpp

    IP4_ADDR(&info.gw, 0, 0, 0, 0); // Zero gateway address

this didn't work on any android or ios device.

Perhaps there's something else that is malfunctioning now that we've moved to C++ and initialization of the wifi_config is incompletely specified in wifi_init_ap()

In the SOTACAT web site you can see a table of recent spots for SOTA or POTA. Those tables show the spot Peak or Park.

This is a feature request for the ability to capture the user's current location from GPS (if the user provides permission to do so) and then compute the distance between the user and the Peak / Park shown in the recent spots table. We can get the Latitude/Longitude information from the same SOTA/POTA servers that give us the recent spots list.

We can then display the distance in the SOTACAT tables, and potentially allow sorting by that column.

As a user, for the currently displayed spots (respecting any filters the user may have enabled, as filtering is added), provide a scanning mode whereby the SOTAcat automatically advances through the spots, dwelling on each frequency/mode for a user-defined amount of time. I would expect this would be implemented on the SOTAcat web UI using features already available to select frequency and mode from spots. I don't think any SOTAcat changes outside of the web UI / Javascript are required.

This would be provided as a button which normally says "Scan" or "Start Scanning" when not in scan mode. Once clicked, the text on the button might change to "Stop" or "Stop Scanning." Once a scan is started, the SOTAcat (Web UI JS) would command the radio to go to the frequency and mode of the first spot in the table. After dwell time seconds, the next entry in the table would be selected automatically, and so on. The user can stop scanning at any time by pressing the scan button again.

Special consideration would need to be made for getting new spots while scanning. As a starting point, we could update after the entire visible list has been scanned and scanning is about to re-start at the top.

I have put together a (mem- and code- cheap) morse code blinker.
I had initially thought it might be useful to convey messages to us users.
Before I check anything in, I would like input.

Broad questions:

1. Would it be better to blink one led in morse (e.g. blue flashes dit/dah) or to have the two colored leds alternate (e.g. blue dit, red dah)? The visual difference in color may be much easier to discern.

2. What messages are most appropriate? For example:
   CQ WIFI = "looking for wifi client"
   QSL WIFI = "found wifi client"
   QRV .. QRV .. QRV = "all ready from setup"
   E = "generally working, ready, waiting, ...." [most often omitted]
   QRT .. QRT .. QRT = "going to sleep"

3. what might we code for problems/errors?

Comment? Good idea, bad idea? What would you like to see?

Justin K5EM says that the default WiFi power is much higher than required. Lower power reduces WiFi noise into the KX radio, slightly lengthens battery life span.

From Justin:

Note that with the (much) lower wifi power I was still able to maintain a connection over 50ft away through 2 walls. Anything lower than 44 didn't work at all though 🤷 the AP didn't show up.

The default power is 20dBm, reducing to 44 brings it down to around 11dBm, so 9dB output power reduction. The wifi range at full power was longer than I wanted to walk down my street. The other benefit of reducing the power is that it all but eliminates the "wifi buzz and pops" that I hear on the KX3. I heard them at home with no antenna connected, but with my home antenna connected they're in the noise. However they were very noticeable in the field with a real antenna. I suspect they're showing up from the fast current pulses from the WiFi transmissions

Average current draw is ~85mA. Around 100mA with both LEDs on. Peaks over 200mA with the default code (I don't have a fast current probe, but the meter is switching scales at 200mA on wifi TX events). I reduced the TX power from 80 to 44 in my code and it drops the peaks substantially (long term average power only goes down about 2% since the wifi is so short duration on average).

The "ESP Web Tools" allow users to install firmware directly from a web page in Chrome, Edge, or Opera browsers. This is integrated into the https://sotamat.com/sotacat page for non-developer SOTACAT users.

As part of our Release build process, the manifest.json file should automatically get its build number updated so that it matches the merged *.bin file that is produced. The manifest is loaded from the github page by the web tools, and the binary is loaded from the sotamat.com site.

At present there are error messages omitted when the ACC port is about to be accessed directly by the uart, and the port is not locked per RadioPortLock.
Perhaps for clarity, they should be migrated upward to the programmer's API level, to functions like get_from_kx, etc.

Brian states:
Recent changes to the code seem to have caused a memory leak. Actually I think it is a stack overflow since the ESP32 ESP-IDF framework does not have a dynamically adjusting stack and instead you have to declare your stack size for each thread. Predicting what that stack size needs to be is hard and error prone. The error manifests as a random reboot after sitting idle for a while my Windows PC browser has the sotacat.local page open (which refreshes the battery every minute). In fact, the browser will call the battery voltage REST and the battery Percentage REST sometimes at the same moment, and I suspect the WiFi code gets an overflow.

Attempting to send an FT8 message with SOTAMat 1.2.1 (67) on iOS, the operation fails and the following is what shows up in the logs:

V (62084) sc:hdl_ft8.: request buffer[93] = "messageText=SM%20KE6MT%2F720F&timeNow=1709679838759&rfFrequency=14074000&audioFrequency=1553"
V (62104) sc:kx_cmds.: trace: get_kx_state()
V (62104) sc:kx_cmds.: trace: get_from_kx()
V (62104) sc:kx_cmds.: trace: uart_get_command()
E (62114) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62134) sc:kx_cmds.: command 'FT;' returned 'FT0' after 10.871 ms
I (62134) sc:kx_cmds.: kx command 'FT' returns 0
V (62134) sc:kx_cmds.: trace: get_from_kx()
V (62134) sc:kx_cmds.: trace: uart_get_command()
E (62144) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62174) sc:kx_cmds.: command 'FA;' returned 'FA00007235000' after 24.008 ms
I (62174) sc:kx_cmds.: kx command 'FA' returns 7235000
V (62174) sc:kx_cmds.: trace: get_from_kx_menu_item()
V (62184) sc:kx_cmds.: put_to_kx(MN) attempting value 58
V (62184) sc:kx_cmds.: trace: get_from_kx()
V (62194) sc:kx_cmds.: trace: uart_get_command()
E (62194) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62234) sc:kx_cmds.: command 'MN;' returned 'MN058' after 27.811 ms
I (62234) sc:kx_cmds.: kx command 'MN' returns 58
I (62234) sc:kx_cmds.: command 'MN' successful; value = 58
V (62244) sc:kx_cmds.: trace: get_from_kx()
V (62244) sc:kx_cmds.: trace: uart_get_command()
E (62244) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62264) sc:kx_cmds.: command 'MP;' returned 'MP000' after 12.300 ms
I (62264) sc:kx_cmds.: kx command 'MP' returns 0
V (62274) sc:kx_cmds.: put_to_kx(MN) attempting value 255
V (62274) sc:kx_cmds.: trace: get_from_kx()
V (62284) sc:kx_cmds.: trace: uart_get_command()
E (62284) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62304) sc:kx_cmds.: command 'MN;' returned 'MN255' after 10.014 ms
I (62304) sc:kx_cmds.: kx command 'MN' returns 255
I (62304) sc:kx_cmds.: command 'MN' successful; value = 255
V (62314) sc:kx_cmds.: trace: get_from_kx()
V (62314) sc:kx_cmds.: trace: uart_get_command()
E (62324) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (62334) sc:kx_cmds.: command 'MD;' returned 'MD1' after 9.651 ms
I (62334) sc:kx_cmds.: kx command 'MD' returns 1
V (62334) sc:kx_cmds.: put_to_kx(MD) attempting value 3
V (62344) sc:kx_cmds.: trace: get_from_kx()
V (62344) sc:kx_cmds.: trace: uart_get_command()
E (62354) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (63084) sc:kx_cmds.: command 'MD;' returned 'MD3' after 719.129 ms
I (63084) sc:kx_cmds.: kx command 'MD' returns 3
I (63084) sc:kx_cmds.: command 'MD' successful; value = 3
V (63084) sc:kx_cmds.: trace: get_from_kx()
V (63094) sc:kx_cmds.: trace: uart_get_command()
E (63094) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (63114) sc:kx_cmds.: command 'AP;' returned 'AP0' after 8.975 ms
I (63114) sc:kx_cmds.: kx command 'AP' returns 0
V (63114) sc:kx_cmds.: put_to_kx(MD) attempting value 1
V (63124) sc:kx_cmds.: trace: get_from_kx()
V (63124) sc:kx_cmds.: trace: uart_get_command()
E (63134) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
V (63414) sc:hdl_batt: battery voltage: 4.095 V
V (63414) sc:idletask: heap: 289044 (used 90784, free 198260) [bytes]
V (63414) sc:idletask: blinks 3799289
I (63694) sc:kx_cmds.: command 'MD;' returned 'MD1' after 552.656 ms
I (63694) sc:kx_cmds.: kx command 'MD' returns 1
I (63694) sc:kx_cmds.: command 'MD' successful; value = 1
V (63694) sc:kx_cmds.: put_to_kx(FR) attempting value 0
V (63704) sc:kx_cmds.: trace: get_from_kx()
V (63704) sc:kx_cmds.: trace: uart_get_command()
E (63714) sc:kx_cmds.: RADIO PORT NOT LOCKED! (coding error in caller)
I (63784) sc:idletask: powering off due to inactivity

This is with the current HEAD - c93569122ad46560c32ef911205dff167f8802a1

As a user, I would like to be able to filter messages in the spots table by mode (CW, SSB, Digital). Similar to, but simpler than how pota.app has filtering of all spots by mode/band/etc.

Most important is danger of the hitting factory reset button without secondary confirmation, clearing wifi credentials.

Beyond that, I'd say the whole Settings page is ugly and could use a bunch of work to make it

1. more aesthetic, and
2. implement a variety of checks like
   a) are you sure you want to factory reset? and
   b) checks to ensure ssids are either empty or >= 2 and <= MAX_WIFI_SSID_SIZE chars and passwords are either empty or >= 8 and <= MAX_WIFI_PASS_SIZE chars. There are some rudiments of the latter already in the HTML, but no validation.

Bug: On iOS the spots "Time-range" selector does nothing and does not persist.

Also, the formatting is not aesthetically pleasing.

Also, the drop-down has crazy options that I think should get purged: such as 10 seconds. The list only refreshes every 1 minute. I would remove the 10 second and 1 minute options and add a 12 hour option (for the purpose of finding S2S mistakes at the end of the day by reviewing history).

This is less likely an issue on these scenarios:

1. when the sotacat joins an existing home wifi network
2. when the sotacat creates its own access point (discoverable at 192.168.4.1), but not availing split networking on android (sigh)
3. when the sotacat joins a hotspot on apple devices, where it's discoverable as sotacat.local

The chief issue is

• when sotacat joins a hotspot on android devices, in order to provide split networking, but doesn't advertise it's client IP

There's no good way, currently, of determining the IP addr of the sotacat as it is presented on the hotspot. Therefore there's no reliable address to which to connect. Fortunately it does have a lifetime, so if observed in the serial logs while connected to PC, it could be a basis for subsequent connection, but that's really terrible.

Proposal here is to scroll the IP address of the SOTAcat on the VFOb display until a connection from client (e.g. webserver request) is made.