In attempting to automate the switching of the postprocess, device and sync modules to newly created stable branches I came across an issue caused by for some reason certain repositories not having upstream branch tracking info set. This meant I could not pull in all branches and switch to the newly created stable branches.

The solution I found to work was the following:
https://github.com/openenergymonitor/emonpi/compare/923e05deeb9ea5945b5493c665f21a20f6be182c..master

I needed to run:

git branch --set-upstream-to=origin/$branch

and then:

git pull

without the branch specific pull, it was then possible to run:

git checkout stable 

@borpin what do you think of this solution? Its a shame to pollute every emonpiupdate log with:

Branch stable set up to track remote branch stable from origin.

but it does seem to work well.

Looking for somewhere to put the git command to allow a git describe to respond with the expected output, and I realised that only the current branch is pulled on update.

Why not just a git pull as that 'fetches' (everything) and 'merges' the current branch.

Either way, can git fetch --all --prune be inserted before this conditional as it simply updates git with the commit history rather than changing anything.

See https://community.openenergymonitor.org/t/permanent-raspberry-pi-booting-after-failed-emonpi-update/10963?u=pb66

https://github.com/openenergymonitor/emonpi/blob/2.9.1/firmware/src/src.ino#L269-L271

if ((now - last_rf_rest) > RF_RESET_PERIOD) {
      rf12_initialize(nodeID, RF_freq, networkGroup);                             // Periodically reset RFM69CW to keep it alive :-(
    }

This code appears to run every iteration of the main loop after the first 60s because last_rf_rest is declared as 0 during set up and never gets updated.

Move this to the backup install script.

I note from the release notes for the latest SD Card build, there is reference to a number of changes that do not seem to be included in the build doc.

(copied from change log)

• Symlink fstab in emonpi repo to /etc/fstab to allow updating
  

• Reduce garbage in /var/log/messages but to Raspbian bug
  

• Fix log rotate to includue all log files to ensure /var/log does not fill up
  

• Fix node-RED flows to survive update cycle (??? not sure if this is in the NodeRed Guide)
  

• Trim SD card (allow 60mB of unallocated partition) to fit on all (or majority) of 4GB SD cards, shop pre-built SD cards included with emonPi / emonBase will be 8GB with ~/data partition expanded accordingly.
  

• Generate new SSH keys
  

Some of these are really useful in general so would be good to capture :)

I recently installed emonSD-24Jul20, and one of my ansible jobs is complaining about a recursive symlink in /opt. If I remove it it gets recreated, so it appears something is creating it at either install or upgrade. The problem is that /opt/openenergymonitor/emonhub/src/src is symlinked to /opt/openenergymonitor/emonhub/src, which causes anything walking the filesystem to end up in a recursive loop.

> ls -l /opt/openenergymonitor/emonhub/src/
total 72
-rw-r--r-- 1 pi   pi     922 Aug 19  2020 Cargo.py
-rw-r--r-- 1 pi   pi    2821 Jun 26 23:23 emonhub_buffer.py
-rw-r--r-- 1 pi   pi     913 Jun 26 23:23 emonhub_coder.py
-rw-r--r-- 1 pi   pi   23667 Jun 26 23:23 emonhub_interfacer.py
-rwxr-xr-x 1 pi   pi   12802 Jun 26 23:23 emonhub.py*
-rw-r--r-- 1 pi   pi    4412 Jun 26 23:23 emonhub_setup.py
drwxr-xr-x 2 pi   pi    4096 Jun 26 23:23 goodwe/
drwxr-xr-x 3 pi   pi    4096 Jun 26 23:23 interfacers/
drwxr-xr-x 2 pi   pi    4096 Jun 26 23:23 smalibrary/
lrwxrwxrwx 1 root root    34 Jun 29 20:39 src -> /opt/openenergymonitor/emonhub/src/

I haven't been able to find what might be creating the symlink. The only obvious link commands are in emonnhub/install.sh and emonpi/update.sh, and those are both linking to the src directory so shouldn't be creating a new link in the src directory.

/var/log/ is filling up I suspect due to the changes in the emonhub.log

It seems the emonhub messages are ending up in both the emonhub and the syslog. Log file fills up very quickly!

https://community.openenergymonitor.org/t/emonpi-stopped-updating-inputs-var-log-full/10416

@glynhudson

Suggest that there should be a dist-upgrade here to really get the image up to date before we start.

new install of emonSD-21Jul21.img onto emonpi
error:
Warning: require_once(/home/pi/demandshaper/lib/misc.php): failed to open stream: No such file or directory in /opt/emoncms/modules/demandshaper/demandshaper-module/demandshaper_controller.php on line 31

Fatal error: require_once(): Failed opening required '/home/pi/demandshaper/lib/misc.php' (include_path='.:/usr/share/php') in /opt/emoncms/modules/demandshaper/demandshaper-module/demandshaper_controller.php on line 31

https://github.com/openenergymonitor/emonpi/blob/master/firmware/compiled/readme.md

@glynhudson

The line .update cannot be correct. as a minimum in needs a cd to move to the correct folder.

As per most configurations, instead of editing a configuration file that may well be modified on upgrade, use drop in file of settings in the '/etc/mosquitto/conf.d` folder.

Just created an EmonSD and booted it on a vanilla RPi B+ - no emonbase installed.

service-runner fails to start.

emonpiupdate.log

These should be with the relevant sub script.

If I do not select Apache for being installed, I would not expect an apt-get apache command to be run.

Is this line required, removing could provide for better non pi environment support:
https://github.com/openenergymonitor/emonpi/blob/master/service-runner#L25

RasPi3 wifi reports:

$ iwconfig
wlan0     IEEE 802.11bgn  ESSID:"Moelwyn"  
          Mode:Managed  Frequency:2.447 GHz  Access Point: 00:11:09:0D:1F:CD   
          Bit Rate=54 Mb/s   Tx-Power=1496 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Encryption key:off
          Power Management:on
          Link Quality=41/70  Signal level=-69 dBm
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

Edimax dongle on Pi2 reports

$ iwconfig
wlan0     IEEE 802.11bgn  ESSID:"Crumble"  Nickname:"<WIFI@REALTEK>"
          Mode:Managed  Frequency:2.447 GHz  Access Point: 7C:4C:A5:2E:7C:ED   
          Bit Rate:72.2 Mb/s   Sensitivity:0/0  
          Retry:off   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=100/100  Signal level=90/100  Noise level=0/100
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

Currently with the above o/p the LCD would display -69% for the Wifi strength on the pi3 and 90% from pi2 as the LCD script currently reads the Signal Level with

        if wlan0ip:
            # wlan link status
            with open('/proc/net/wireless', 'r') as f:
                wireless = f.readlines()
            signals = [x.split()[3] for x in wireless if x.strip().startswith('wlan0')]
            if signals:
                signallevel = signals[0].partition('.')[0]
        r.set("wlan:signallevel", signallevel)

We need to detect a negative number and then instead display signal quality as a percentage e.g Link Quality=41/70 = 41/70*100 = 59%

like this page's info import to the emonpi?
any one can help?

Use systemctl for services.

sudo systemctl restart redis.service

There will be instances where the Mosquitto server is not desirable when there is already an MQTT server in use.

This line is intended to prevent duplicate daemon.log entries. My standard EmonPi does not have this line in the rsyslog.conf

In src.ino, the variable now is set to the result of millis(). The Arduino documentation for this function states:

This number will overflow (go back to zero), after approximately 50 days.

I think this has an effect on the RF reset timeout and sample checks:

if ((now - last_rf_rest) > RF_RESET_PERIOD)

if ((now - last_sample) > TIME_BETWEEN_READINGS)

I think that after now overflows, the checks above will always result in false until now again reaches the time of the last last_rf_rest and last_sample times before the overflow, i.e. around once every 50 days. That means that from power-on, the board will reset the RF module and make samples every RF_RESET_PERIOD and TIME_BETWEEN_READINGS milliseconds, until ~50 days have past, then only make samples and reset the RF module every 50 days.

Maybe I've misunderstood how this works in the C/AVR/Arduino world, but I thought I'd point it out in case this is a bug.

A potential fix could simply be to put this before the two checks above:

  // Check if now has overflowed.
  if (now < last_sample || now < last_rf_rest) {
    // Assume now has overflowed, so reset variables that are checked against now.
    last_rf_rest = 0;
    last_sample = 0;
  }

/opt/openenergymonitor/emonpi/factoryreset is out of date with respect to the rest of the OS. It tries to access /home/pi/emonpi and /home/pi/data which don't exist, and then leaves the system in a broken state. In my case, Wi-Fi searching no longer worked in the web setup.

Suggest either fixing or removal.

Suggest the first clone should be of a new EmonScripts repo to pull in the required scripts

Rationale: Make this less emonpi specific.

In emonpi/docs/SD-card-build.md this line fails:

sudo sh -c 'echo "extension=mosquitto.so" > /etc/php5/apache2/conf.d/20-mosquitto.ini'

because apache isn't installed yet.

I wonder if section 5 (Installing MQTT) should follow after section 7 (Installing emonCMS) which is where apache is installed?

I've also noticed that section 6 is missing - probably should have been associated with this heading:

Install emonHub

EDIT: while I'm here... there's also an extraneous double quote (”) on the end of this line:

sudo apt-get install -y python-pip python-serial python-configobj”

see https://community.openenergymonitor.org/t/permanent-raspberry-pi-booting-after-failed-emonpi-update/10963/5?u=pb66

Is the travis build still usefull ?

Could the current user be used instead of pi or else pass in a parameter for the user?

Thinking is making it non-rpi specific

As node-red was removed from the image , consider removing the old .node-red modules.
If you install node red-on the emon pi with the installation script node red crashed on startup as there are still old nodes that has not been loaded in the flows.
See below -

Flows stopped due to missing node types.
emoncms-server
wunderground

Just downloaded and flashed emonSD-07Nov16

Boots up fine but logging in via SSH is not working:

$ ssh pi@emonpi
pi@emonpi's password: emonpi2016
Permission denied, please try again.

The current logrotate mechanism does not prevent /var/log filling up.

I propose a new mechanism that diverts the log messages from emonhub that currently end up in both daemon.log and syslog to its own log file and then rotate hourly. The rotated logs will be concatenated together to provide a daily log, rotated and retained for X days.

There are discussions ongoing on employing Log2Ram in the future, but this will work in the meantime.

This setup will leave all existing package rotation schemas alone.

1. Purge and reinstall rsyslog to restore default configuration
2. Setup rsyslog to extract emonhub messages and not put them in daemon.log or syslog 10-emonhub.conf below.
3. Delete excessive daemon.log sudo truncate -s 0 /var/log/daemon.log
4. Restart the rsyslog service sudo systemctl restart rsyslog.service
5. Delete conf files installed by emoncms

sudo rm /etc/logrotate.conf
sudo rm /etc/cron.daily/logrotate
sudo rm /etc/cron.hourly/logrotate

6. Reinstall logrotate to restore default setup

sudo apt-get purge logrotate
sudo apt-get install logrotate

7. Install logrotate file to handle emonhub.log /etc/logrotate.d/emonhub
8. Make logrotate run hourly

sudo mv /etc/cron.daily/logrotate /etc/cron.hourly/

/etc/rsyslog.d/10-emonhub.conf

template(name="EmonhubMsg" type="string" string="%msg:::drop-last-lf%\n")

if ( $programname == 'emonhub' or $programname == 'emonhub.py' ) then {
    action(type="omfile" file="/var/log/emonhub/emonhub.log" template="EmonhubMsg")
    stop
}

/etc/logrotate.d/emonhub

/var/log/emonhub/emonhub.log {
        rotate 6
        daily
        size 2M
        maxsize 2M
        missingok
        copytruncate
        notifempty
        postrotate
             mv /var/log/emonhub/emonhub.log.1 /home/pi/data/
             cat /home/pi/data/emonhub.log.1 >> /home/pi/data/emonhub.log
             rm /home/pi/data/emonhub.log.1
        endscript
}

/home/pi/data/emonhub.log {
        su pi pi
        rotate 10
        daily
        compress
        missingok
        notifempty
}

The download link doesn't work or is private, from the website https://guide.openenergymonitor.org/technical/emonpi/ to https://github.com/openenergymonitor/emonpi/Docs/emonSD-pre-built-SD-card-Download-&-Change-Log I found a community post with the image: https://community.openenergymonitor.org/t/emonsd-24jul20-release/15170

Hi Developers

I made a typo in the emonhub.conf from the GUI. It puzzled me a lot since using ssh I was looking at a different file. I followed a thread which suggested me to update firmware....

I would very much like to get back to my starting point v2.83 bit it is missing in your archive. Is it possible for you to add it.

Thanks in advance
Jens

add the purge command to the update script please as discussed here Unexpected output from git describe

Review for EmonScripts as well

There appears to be no versioning on the emonpi repository.

The impact of this is that there is no way to confirm what state the emonpi part of an EmonPi is in.

Solution is to create an annotated tag and generate a release that can be picked up by the EmonCMS administration page.

Is the postprocess module intended to be included in the stock SD image? It's not currently in the listed default modules, though I have it on my Nov 2016 image, possibly as a result of having run the emoncms update script, which appears to install that module.

Seeking clarification before I add this to the default module list in my Chef-version of the build process. 😄

Hello,

I am using the store built emon pi device with a VE direct cable to a 712 battery monitor. That was working fine.

Then I plugged in a norteck Zwave/Zigbee stick for use with OPEN HAB. Now it seems like the USB ports rotate every day or so and emonpi loses the VE Direct feed.

Here is a screen shot showing the missing data from the last week:

Whenever this happens, I have to edit the emonhub config to point to the correct usb port. /dev/ttyUSB2 was the winner today but /dev/ttyUSB0 was the winner yesterday.

Why don't the devices stay the same? I mean they aren't being unplugged because the emonpi is in an access controlled area.

Log the install process to aid debugging either within the script (how?) or with the external command.

Following on from a discussion https://community.openenergymonitor.org/t/help-fatal-error-after-emonpi-update/9681 where a user updated then had issues with their system, I suggest that users are encouraged to backup their instance before performing updates.

Even better would be a mechanism to do a backup automatically before an update is attempted. This could perhaps be to a zip file that is then downloaded to the local machine.

I downloaded emonSD-21Jul21 imaged and created a SDcard to install in Rpi zero w 2. At the time of created imaged I did modify hostname, SSH and Wifi settings, using [CTRL][SHIFT] [X] in Raspberry Pi Imager.
The Pi zero 2 booted fine and I can log in via SSH and see the webpage but the emoncms webpage is asking for a login/password which is not mentioned in user guide https://guide.openenergymonitor.org/setup/local/ .
That is first problem, second problem is the command to reset password does not work,
$ php /opt/emoncms/modules/usefulscripts/resetpassword.php
I looked and see it there but it responds with
' -bash: resetpassword: command not found '

Latest build and fully updated.

My emonPi updates on every reboot, it seems the "firstbootupdate" script is initializing an update even when it is not required. When I looked into it and tested the separate "or" parts of the if statement in this line

https://github.com/openenergymonitor/emonpi/blob/master/firstbootupdate#L13

I found the log file is non-zero size as expected since it is not the first boot, but the $button value is 0 at rest and 1 when pressed, so the button action actually works as a "update bypass" if held while rebooting not a "forced update".

By editing this line to

if [ ! -s /home/pi/data/emonpiupdate.log ] || [ $button == 1 ]; then

it works as expected.

I can see there was a recent change to this code that inverted the action based on the $button value, was this intended? has the switch been physically changed in production?

I purchased and installed an emonPi to read power consumption, PV production, etc.

It was working just fine for many years to suddenly stop logging one day without any warning.

I would like to re-install the latest emonPI monitor software and restore current data and configuration files to carry over all the reports and config that are in the current environment.

In the admin I can see there is still an attempt to read and scroll back in the reports I can see all data prior to the Feed stop .

Is there anybody out there that did the same thing before that is willing to give me some pointers. The backup-restore function appears to overwrite everything and I am not sure that is the correct thing to do.

Thanking in advance.

One of the Wifi hotspots near me has a single quote in it's name.

It is basically:
"<name>'s wifi"

Due to the single quote I get Javascript errors when trying to save wifi settings.

Hello,
i've follow the tuto : WIFI Hotspot
https://github.com/openenergymonitor/emonpi/tree/master/wifiAP

my emonSD works well on my PI3
but in AP mode i don't have any graph

but in station mode, i have graph
cf : https://community.openenergymonitor.org/t/emonsd-07nov16-on-rasberrypi-no-graph/5708

is it the same for you ?
or have you graph in AP mode
here is the img link could be long it's on frendh server
http://dl.free.fr/m9XQPlioP

see https://community.openenergymonitor.org/t/has-var-log-filling-up-been-fixed/10985/5?u=pb66

I ran the factory rest script on the latest SD card image as part of debugging a feeds issue and after the system had been cleaned and the Pi had rebooted I was still able to login using the previously setup authorised key.

Hi,

Seems like when i use 64gb SD card with the 30oct18 image for raspberry it runs fine until i run: sudo emonSDexpand

Then it will not start up correctly seems Data partition is gone - maybe more

/Kasper

Hello, my emonPi device currently has an error that cannot connect to the network as follows.

The first is about the configuration information I'm using:

• The OS I use for emonPi is version: emonSD-24Jul20 (Stable)
• The types of wifi I connect to include 2.4GHz and 5.0GHz
• My current router is very normal and I have no device blocking

My current problem is as follows:

I have reinstalled the operating system with the latest version as described above. I have seen the installation instructions of emonPi on its website and I follow the steps as above.

After booting, the LCD screen of emonPi shows the IP address 192.168.42.1 and I connected to the wifi network of emonPi and connected to my Wifi network. After doing the above, the device reboots on the LCD screen and after reboot it shows "Ethernet not connected" on the LCD screen. I tried other times the same with wired LAN, 2.4GHz - 5.0GHz wireless network and even after every try I reinstalled emonPi's operating system. However, the same results are all not connected to the network. I use emonCMS, Fing to scan and find emonPi IP in my network, I can't see it, and I also use PuTTy to connect SSH with emonPi but it doesn't work.

I tried connecting my monitor to emonPi via the HDMI port of rasberry3 and I entered the command ifconfig and iwconfig both showed no connection to the network.

During the device reboot after connecting me to wifi for it through 192.168.42.1, I pinged the device, the result showed that the device can only ping in a very short time after that's impossible to ping completely, the failure rate is 98%.

Don't know what this error is and how to fix it. Thanks.

Discussion https://community.openenergymonitor.org/t/new-emonsd-release-emonsd-30oct18-raspbian-stretch-pi3b-compatible/9005/59 refers.

The SD-card-build.md document currently reads:

The emon-pi variant of emonHub locates the config file in the RW partition /home/pi/data/emonhub.conf, this config file includes the default emonpi MQTT authentication details. Symlink the latest default config file:

	sudo rm ~/data/emonhub.conf
	sudo ln -s /home/pi/emonhub/conf/emonpi.default.emonhub.conf /home/pi/data/emonhub.conf

	sudo service emonhub start

The emonhub/install process the user is instructed to launch just prior to this step moves the emonpi.default.emonhub.conf file to the /home/pi/data directory, causing:

1. the ln -s to link to a non-existent file
2. emonhub to fail to start
3. much sadness 😭

The above quoted section can either be removed completely from the build doc or comments can be added that this is an optional step if not running emonhub/install. I vote for removing completely. 😄