Syntax error reported in pv_opt.py about pv_opt HOT 23 CLOSED

from pv_opt.

This is a weird one. Spurious code had got duplicated out of place - probably git related. Fixed in 3.4.1

from pv_opt.

Thanks for sorting that out so fast! I removed the automation and reinstalled/downloaded the latest version 3.4.3, restarted AppDaemon and get a different error now....

2023-12-21 12:19:21.675067 INFO pv_opt: Checking config:
2023-12-21 12:19:21.689109 INFO pv_opt: -----------------------
2023-12-21 12:19:21.705747 WARNING pv_opt: forced_charge = True True: system default. Not in YAML.
2023-12-21 12:19:21.717969 WARNING pv_opt: forced_discharge = True True: system default. Not in YAML.
2023-12-21 12:19:21.727570 WARNING pv_opt: read_only = True True: system default. Not in YAML.
2023-12-21 12:19:21.740293 WARNING pv_opt: allow_cyclic = False False: system default. Not in YAML.
2023-12-21 12:19:21.753859 WARNING pv_opt: optimise_frequency_minutes = 10 10: system default. Not in YAML.
2023-12-21 12:19:21.770879 WARNING pv_opt: slot_threshold_p = 1.0 1.0: system default. Not in YAML.
2023-12-21 12:19:21.783239 WARNING pv_opt: day_of_week_weighting = 0.5 0.5: system default. Not in YAML.
2023-12-21 12:19:21.788466 WARNING pv_opt: pass_threshold_p = 4.0 4.0: system default. Not in YAML.
2023-12-21 12:19:21.800459 WARNING pv_opt: octopus_auto = True True: system default. Not in YAML.
2023-12-21 12:19:21.807217 WARNING pv_opt: battery_capacity_wh = 10000 10000: system default. Not in YAML.
2023-12-21 12:19:21.812659 WARNING pv_opt: inverter_efficiency_percent = 97 97: system default. Not in YAML.
2023-12-21 12:19:21.820220 WARNING pv_opt: charger_efficiency_percent = 91 91: system default. Not in YAML.
2023-12-21 12:19:21.828807 WARNING pv_opt: charger_power_watts = 3000 3000: system default. Not in YAML.
2023-12-21 12:19:21.834249 WARNING pv_opt: inverter_power_watts = 3600 3600: system default. Not in YAML.
2023-12-21 12:19:21.844975 WARNING pv_opt: inverter_loss_watts = 100 100: system default. Not in YAML.
2023-12-21 12:19:21.864409 WARNING pv_opt: solar_forecast = Solcast Solcast: system default. Not in YAML.
2023-12-21 12:19:21.882539 WARNING pv_opt: id_solcast_today = sensor.solcast_pv_forecast_forecast_today 2.36665: system default. Not in YAML.
2023-12-21 12:19:21.898519 WARNING pv_opt: id_solcast_tomorrow = sensor.solcast_pv_forecast_forecast_tomorrow 1.23335: system default. Not in YAML.
2023-12-21 12:19:21.904050 WARNING pv_opt: consumption_history_days = 7 7: system default. Not in YAML.
2023-12-21 12:19:21.908951 WARNING pv_opt: consumption_margin = 10 10: system default. Not in YAML.
2023-12-21 12:19:21.914868 WARNING pv_opt: consumption_grouping = mean mean: system default. Not in YAML.
2023-12-21 12:19:21.921318 WARNING pv_opt: forced_power_group_tolerance = 100 100: system default. Not in YAML.
2023-12-21 12:19:21.933773 WARNING pv_opt: maximum_dod_percent = number.solis_battery_minimum_soc 15.0: system default. Not in YAML.
2023-12-21 12:19:21.942235 WARNING pv_opt: id_battery_soc = sensor.solis_battery_soc 88.0: system default. Not in YAML.
2023-12-21 12:19:21.955034 WARNING pv_opt: id_consumption = ['sensor.solis_house_load', 'sensor.solis_bypass_load'] 800.0: system default. Not in YAML.
2023-12-21 12:19:21.967882 WARNING pv_opt: id_consumption_today = sensor.solis_house_load_today 14.3: system default. Not in YAML.
2023-12-21 12:19:21.974392 WARNING pv_opt: id_grid_import_today = sensor.solis_grid_import_today 20.4: system default. Not in YAML.
2023-12-21 12:19:21.987331 WARNING pv_opt: id_grid_export_today = sensor.solis_grid_export_today 0.0: system default. Not in YAML.
2023-12-21 12:19:22.003585 WARNING pv_opt: id_grid_import_power = sensor.solis_grid_import_power 40.0: system default. Not in YAML.
2023-12-21 12:19:22.015674 WARNING pv_opt: id_grid_export_power = sensor.solis_grid_export_power 0.0: system default. Not in YAML.
2023-12-21 12:19:22.020292 WARNING pv_opt: id_battery_charge_power = sensor.solis_battery_input_energy 0.0: system default. Not in YAML.
2023-12-21 12:19:22.030825 WARNING pv_opt: id_inverter_ac_power = sensor.solis_active_power 750.0: system default. Not in YAML.
2023-12-21 12:19:22.041799 WARNING pv_opt: supports_hold_soc = True True: system default. Not in YAML.
2023-12-21 12:19:22.047999 WARNING pv_opt: update_cycle_seconds = 15 15: system default. Not in YAML.
2023-12-21 12:19:22.052555 WARNING pv_opt: battery_voltage = sensor.solis_battery_voltage 50.4: system default. Not in YAML.
2023-12-21 12:19:22.062033 WARNING pv_opt: id_timed_charge_start_hours = number.solis_timed_charge_start_hours 8.0: system default. Not in YAML.
2023-12-21 12:19:22.075129 WARNING pv_opt: id_timed_charge_start_minutes = number.solis_timed_charge_start_minutes 0.0: system default. Not in YAML.
2023-12-21 12:19:22.080238 WARNING pv_opt: id_timed_charge_end_hours = number.solis_timed_charge_end_hours 8.0: system default. Not in YAML.
2023-12-21 12:19:22.090436 WARNING pv_opt: id_timed_charge_end_minutes = number.solis_timed_charge_end_minutes 30.0: system default. Not in YAML.
2023-12-21 12:19:22.100932 WARNING pv_opt: id_timed_charge_current = number.solis_timed_charge_current 100.0: system default. Not in YAML.
2023-12-21 12:19:22.114510 WARNING pv_opt: id_timed_discharge_start_hours = number.solis_timed_discharge_start_hours 0.0: system default. Not in YAML.
2023-12-21 12:19:22.129170 WARNING pv_opt: id_timed_discharge_start_minutes = number.solis_timed_discharge_start_minutes 0.0: system default. Not in YAML.
2023-12-21 12:19:22.136232 WARNING pv_opt: id_timed_discharge_end_hours = number.solis_timed_discharge_end_hours 0.0: system default. Not in YAML.
2023-12-21 12:19:22.144457 WARNING pv_opt: id_timed_discharge_end_minutes = number.solis_timed_discharge_end_minutes 0.0: system default. Not in YAML.
2023-12-21 12:19:22.154182 WARNING pv_opt: id_timed_discharge_current = number.solis_timed_discharge_current 100.0: system default. Not in YAML.
2023-12-21 12:19:22.164463 WARNING pv_opt: id_timed_charge_discharge_button = button.solis_update_charge_discharge_times 2023-12-21T07:38:25.006983+00:00: system default. Not in YAML.
2023-12-21 12:19:22.176141 WARNING pv_opt: id_inverter_mode = select.solis_energy_storage_control_switch Timed Charge/Discharge: system default. Not in YAML.
2023-12-21 12:19:22.184480 WARNING pv_opt: id_backup_mode_soc = number.solis_backup_mode_soc 40.0: system default. Not in YAML.
2023-12-21 12:19:22.191088 INFO pv_opt:
2023-12-21 12:19:22.197981 INFO pv_opt: Exposing config to Home Assistant:
2023-12-21 12:19:22.206095 INFO pv_opt: ----------------------------------
2023-12-21 12:19:22.289384 INFO pv_opt:
2023-12-21 12:19:22.296769 INFO pv_opt: Config Item HA Entity Current State
2023-12-21 12:19:22.302159 INFO pv_opt: ----------- --------- -------------
2023-12-21 12:19:22.310521 INFO pv_opt: forced_charge switch.pvopt_forced_charge on
2023-12-21 12:19:22.322155 INFO pv_opt: forced_discharge switch.pvopt_forced_discharge off
2023-12-21 12:19:22.334587 INFO pv_opt: read_only switch.pvopt_read_only on
2023-12-21 12:19:22.339894 INFO pv_opt: allow_cyclic switch.pvopt_allow_cyclic off
2023-12-21 12:19:22.350355 INFO pv_opt: optimise_frequency_minutes number.pvopt_optimise_frequency_minutes 10
2023-12-21 12:19:22.359397 INFO pv_opt: slot_threshold_p number.pvopt_slot_threshold_p 1.0
2023-12-21 12:19:22.367334 INFO pv_opt: day_of_week_weighting number.pvopt_day_of_week_weighting 0.5
2023-12-21 12:19:22.379286 INFO pv_opt: pass_threshold_p number.pvopt_pass_threshold_p 4.0
2023-12-21 12:19:22.393275 INFO pv_opt: battery_capacity_wh number.pvopt_battery_capacity_wh 10000
2023-12-21 12:19:22.400147 INFO pv_opt: inverter_efficiency_percent number.pvopt_inverter_efficiency_percent 97
2023-12-21 12:19:22.411354 INFO pv_opt: charger_efficiency_percent number.pvopt_charger_efficiency_percent 91
2023-12-21 12:19:22.417734 INFO pv_opt: charger_power_watts number.pvopt_charger_power_watts 3000
2023-12-21 12:19:22.427614 INFO pv_opt: inverter_power_watts number.pvopt_inverter_power_watts 3600
2023-12-21 12:19:22.437726 INFO pv_opt: inverter_loss_watts number.pvopt_inverter_loss_watts 100
2023-12-21 12:19:22.445971 INFO pv_opt: solar_forecast select.pvopt_solar_forecast Solcast
2023-12-21 12:19:22.454327 INFO pv_opt: consumption_history_days number.pvopt_consumption_history_days 7
2023-12-21 12:19:22.462294 INFO pv_opt: consumption_margin number.pvopt_consumption_margin 10
2023-12-21 12:19:22.473727 INFO pv_opt: consumption_grouping select.pvopt_consumption_grouping mean
2023-12-21 12:19:22.483451 INFO pv_opt: forced_power_group_tolerance number.pvopt_forced_power_group_tolerance 100
2023-12-21 12:19:41.679266 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7f9405f6cf40>) took too long (10 seconds), cancelling the task...
2023-12-21 12:19:41.682185 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 12:19:41.682844 WARNING pv_opt: Unexpected error in worker for App pv_opt:
2023-12-21 12:19:41.683359 WARNING pv_opt: Worker Ags: {}
2023-12-21 12:19:41.683866 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 12:19:41.720280 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 273, in initialize
self._estimate_capacity()
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 320, in _estimate_capacity
self.hass2df(entity_id=self.config["id_battery_soc"], days=7)
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 235, in hass2df
df = pd.DataFrame(hist[0]).set_index("last_updated")["state"]
~~~~^^^
TypeError: 'NoneType' object is not subscriptable

2023-12-21 12:19:41.720604 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 12:19:41.728708 INFO AppDaemon: App initialization complete

Thanks again for your help.

from pv_opt.

Very odd, after a restart of HA, I get different errors...

2023-12-21 13:54:26.517059 INFO pv_opt: Checking config:
2023-12-21 13:54:26.530029 INFO pv_opt: -----------------------
2023-12-21 13:54:26.542731 WARNING pv_opt: forced_charge = True True: system default. Not in YAML.
2023-12-21 13:54:26.559050 WARNING pv_opt: forced_discharge = True True: system default. Not in YAML.
2023-12-21 13:54:26.563911 WARNING pv_opt: read_only = True True: system default. Not in YAML.
2023-12-21 13:54:26.571249 WARNING pv_opt: allow_cyclic = False False: system default. Not in YAML.
2023-12-21 13:54:26.581179 WARNING pv_opt: optimise_frequency_minutes = 10 10: system default. Not in YAML.
2023-12-21 13:54:26.591444 WARNING pv_opt: slot_threshold_p = 1.0 1.0: system default. Not in YAML.
2023-12-21 13:54:26.595978 WARNING pv_opt: day_of_week_weighting = 0.5 0.5: system default. Not in YAML.
2023-12-21 13:54:26.607846 WARNING pv_opt: pass_threshold_p = 4.0 4.0: system default. Not in YAML.
2023-12-21 13:54:26.614109 WARNING pv_opt: octopus_auto = True True: system default. Not in YAML.
2023-12-21 13:54:26.625275 WARNING pv_opt: battery_capacity_wh = 10000 10000: system default. Not in YAML.
2023-12-21 13:54:26.632352 WARNING pv_opt: inverter_efficiency_percent = 97 97: system default. Not in YAML.
2023-12-21 13:54:26.639833 WARNING pv_opt: charger_efficiency_percent = 91 91: system default. Not in YAML.
2023-12-21 13:54:26.649228 WARNING pv_opt: charger_power_watts = 3000 3000: system default. Not in YAML.
2023-12-21 13:54:26.656819 WARNING pv_opt: inverter_power_watts = 3600 3600: system default. Not in YAML.
2023-12-21 13:54:26.671211 WARNING pv_opt: inverter_loss_watts = 100 100: system default. Not in YAML.
2023-12-21 13:54:26.695111 WARNING pv_opt: solar_forecast = Solcast Solcast: system default. Not in YAML.
2023-12-21 13:54:26.714577 WARNING pv_opt: id_solcast_today = sensor.solcast_pv_forecast_forecast_today 2.1508: system default. Not in YAML.
2023-12-21 13:54:26.736303 WARNING pv_opt: id_solcast_tomorrow = sensor.solcast_pv_forecast_forecast_tomorrow 1.22705: system default. Not in YAML.
2023-12-21 13:54:26.753196 WARNING pv_opt: consumption_history_days = 7 7: system default. Not in YAML.
2023-12-21 13:54:26.765889 WARNING pv_opt: consumption_margin = 10 10: system default. Not in YAML.
2023-12-21 13:54:26.784740 WARNING pv_opt: consumption_grouping = mean mean: system default. Not in YAML.
2023-12-21 13:54:26.806815 WARNING pv_opt: forced_power_group_tolerance = 100 100: system default. Not in YAML.
2023-12-21 13:54:26.817524 WARNING pv_opt: maximum_dod_percent = number.solis_battery_minimum_soc 15.0: system default. Not in YAML.
2023-12-21 13:54:26.835908 WARNING pv_opt: id_battery_soc = sensor.solis_battery_soc 88.0: system default. Not in YAML.
2023-12-21 13:54:26.850121 WARNING pv_opt: id_consumption = ['sensor.solis_house_load', 'sensor.solis_bypass_load'] 906.0: system default. Not in YAML.
2023-12-21 13:54:26.866967 WARNING pv_opt: id_consumption_today = sensor.solis_house_load_today 16.1: system default. Not in YAML.
2023-12-21 13:54:26.876552 WARNING pv_opt: id_grid_import_today = sensor.solis_grid_import_today 21.1: system default. Not in YAML.
2023-12-21 13:54:26.900706 WARNING pv_opt: id_grid_export_today = sensor.solis_grid_export_today 0.0: system default. Not in YAML.
2023-12-21 13:54:26.918935 WARNING pv_opt: id_grid_import_power = sensor.solis_grid_import_power 151.0: system default. Not in YAML.
2023-12-21 13:54:26.934837 WARNING pv_opt: id_grid_export_power = sensor.solis_grid_export_power 0.0: system default. Not in YAML.
2023-12-21 13:54:26.949174 WARNING pv_opt: id_battery_charge_power = sensor.solis_battery_input_energy 0.0: system default. Not in YAML.
2023-12-21 13:54:26.960071 WARNING pv_opt: id_inverter_ac_power = sensor.solis_active_power 730.0: system default. Not in YAML.
2023-12-21 13:54:26.971890 WARNING pv_opt: supports_hold_soc = True True: system default. Not in YAML.
2023-12-21 13:54:26.978206 WARNING pv_opt: update_cycle_seconds = 15 15: system default. Not in YAML.
2023-12-21 13:54:26.991253 WARNING pv_opt: battery_voltage = sensor.solis_battery_voltage 50.4: system default. Not in YAML.
2023-12-21 13:54:27.012032 WARNING pv_opt: id_timed_charge_start_hours = number.solis_timed_charge_start_hours 8.0: system default. Not in YAML.
2023-12-21 13:54:27.026686 WARNING pv_opt: id_timed_charge_start_minutes = number.solis_timed_charge_start_minutes 0.0: system default. Not in YAML.
2023-12-21 13:54:27.040425 WARNING pv_opt: id_timed_charge_end_hours = number.solis_timed_charge_end_hours 8.0: system default. Not in YAML.
2023-12-21 13:54:27.048281 WARNING pv_opt: id_timed_charge_end_minutes = number.solis_timed_charge_end_minutes 30.0: system default. Not in YAML.
2023-12-21 13:54:27.068005 WARNING pv_opt: id_timed_charge_current = number.solis_timed_charge_current 100.0: system default. Not in YAML.
2023-12-21 13:54:27.084755 WARNING pv_opt: id_timed_discharge_start_hours = number.solis_timed_discharge_start_hours 0.0: system default. Not in YAML.
2023-12-21 13:54:27.104774 WARNING pv_opt: id_timed_discharge_start_minutes = number.solis_timed_discharge_start_minutes 0.0: system default. Not in YAML.
2023-12-21 13:54:27.112800 WARNING pv_opt: id_timed_discharge_end_hours = number.solis_timed_discharge_end_hours 0.0: system default. Not in YAML.
2023-12-21 13:54:27.130587 WARNING pv_opt: id_timed_discharge_end_minutes = number.solis_timed_discharge_end_minutes 0.0: system default. Not in YAML.
2023-12-21 13:54:27.147373 WARNING pv_opt: id_timed_discharge_current = number.solis_timed_discharge_current 100.0: system default. Not in YAML.
2023-12-21 13:54:27.158493 WARNING pv_opt: id_timed_charge_discharge_button = button.solis_update_charge_discharge_times 2023-12-21T07:38:25.006983+00:00: system default. Not in YAML.
2023-12-21 13:54:27.176436 WARNING pv_opt: id_inverter_mode = select.solis_energy_storage_control_switch Timed Charge/Discharge: system default. Not in YAML.
2023-12-21 13:54:27.189898 WARNING pv_opt: id_backup_mode_soc = number.solis_backup_mode_soc 40.0: system default. Not in YAML.
2023-12-21 13:54:27.195830 INFO pv_opt:
2023-12-21 13:54:27.201079 INFO pv_opt: Exposing config to Home Assistant:
2023-12-21 13:54:27.206916 INFO pv_opt: ----------------------------------
2023-12-21 13:54:27.281623 INFO pv_opt:
2023-12-21 13:54:27.291240 INFO pv_opt: Config Item HA Entity Current State
2023-12-21 13:54:27.313426 INFO pv_opt: ----------- --------- -------------
2023-12-21 13:54:27.350792 INFO pv_opt: forced_charge switch.pvopt_forced_charge on
2023-12-21 13:54:27.355839 INFO pv_opt: forced_discharge switch.pvopt_forced_discharge off
2023-12-21 13:54:27.371172 INFO pv_opt: read_only switch.pvopt_read_only on
2023-12-21 13:54:27.390299 INFO pv_opt: allow_cyclic switch.pvopt_allow_cyclic off
2023-12-21 13:54:27.407831 INFO pv_opt: optimise_frequency_minutes number.pvopt_optimise_frequency_minutes 10
2023-12-21 13:54:27.422801 INFO pv_opt: slot_threshold_p number.pvopt_slot_threshold_p 1.0
2023-12-21 13:54:27.430531 INFO pv_opt: day_of_week_weighting number.pvopt_day_of_week_weighting 0.5
2023-12-21 13:54:27.441754 INFO pv_opt: pass_threshold_p number.pvopt_pass_threshold_p 4.0
2023-12-21 13:54:27.450528 INFO pv_opt: battery_capacity_wh number.pvopt_battery_capacity_wh 10000
2023-12-21 13:54:27.461309 INFO pv_opt: inverter_efficiency_percent number.pvopt_inverter_efficiency_percent 97
2023-12-21 13:54:27.484274 INFO pv_opt: charger_efficiency_percent number.pvopt_charger_efficiency_percent 91
2023-12-21 13:54:27.493443 INFO pv_opt: charger_power_watts number.pvopt_charger_power_watts 3000
2023-12-21 13:54:27.501488 INFO pv_opt: inverter_power_watts number.pvopt_inverter_power_watts 3600
2023-12-21 13:54:27.509511 INFO pv_opt: inverter_loss_watts number.pvopt_inverter_loss_watts 100
2023-12-21 13:54:27.523707 INFO pv_opt: solar_forecast select.pvopt_solar_forecast Solcast
2023-12-21 13:54:27.528103 INFO pv_opt: consumption_history_days number.pvopt_consumption_history_days 7
2023-12-21 13:54:27.536155 INFO pv_opt: consumption_margin number.pvopt_consumption_margin 10
2023-12-21 13:54:27.549795 INFO pv_opt: consumption_grouping select.pvopt_consumption_grouping mean
2023-12-21 13:54:27.556906 INFO pv_opt: forced_power_group_tolerance number.pvopt_forced_power_group_tolerance 100
2023-12-21 13:54:37.574691 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7f9405230940>) took too long (10 seconds), cancelling the task...
2023-12-21 13:54:37.580598 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 13:54:37.582959 WARNING pv_opt: Unexpected error in worker for App pv_opt:
2023-12-21 13:54:37.584153 WARNING pv_opt: Worker Ags: {}
2023-12-21 13:54:37.584894 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 13:54:37.595113 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 273, in initialize
self._estimate_capacity()
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 307, in _estimate_capacity
self.hass2df(
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 235, in hass2df
df = pd.DataFrame(hist[0]).set_index("last_updated")["state"]
~~~~^^^
TypeError: 'NoneType' object is not subscriptable

2023-12-21 13:54:37.600624 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 13:54:37.641892 WARNING AppDaemon: Excessive time spent in utility loop: 11369.0ms, 11369.0ms in check_app_updates(), 0.0ms in other

from pv_opt.

I've disabled the function that was causing this error as it's not essential. It looks like it wasn't getting the necessary data from HA so when I re-enable it I will add some logging and error tracing to test this.

Try 3.4.4 and let me know

from pv_opt.

Ok thanks, just loaded 3.4.4 and get the following...

...
2023-12-21 15:29:04.689729 INFO pv_opt: No upcoming Octopus Saving Events detected or joined:
2023-12-21 15:29:04.694286 INFO pv_opt: Starting Opimisation with discharge disabled
2023-12-21 15:29:04.701436 INFO pv_opt: ---------------------------------------------------
2023-12-21 15:29:04.899951 INFO pv_opt: Solcast forecast loaded OK
2023-12-21 15:29:04.905877 INFO pv_opt: Getting expected consumption data
2023-12-21 15:29:14.924988 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7fa048e60940>) took too long (10 seconds), cancelling the task...
2023-12-21 15:29:14.928274 ERROR pv_opt: Unable to get historical consumption from sensor.solis_house_load. 'NoneType' object is not subscriptable
2023-12-21 15:29:21.468465 INFO pv_opt: Initial SOC: 87.0
2023-12-21 15:29:21.476165 INFO pv_opt: Calculating Base flows
2023-12-21 15:29:21.486354 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 15:29:21.491351 WARNING pv_opt: Unexpected error running initialize() for pv_opt
2023-12-21 15:29:21.492425 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 15:29:21.547681 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3791, in get_loc
return self._engine.get_loc(casted_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'consumption'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 294, in initialize
self.optimise()
File "/usr/lib/python3.11/site-packages/appdaemon/adbase.py", line 35, in f_app_lock
return f(*args, **kw)
^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 1282, in optimise
self.base = self.pv_system.flows(
^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pvpy.py", line 458, in flows
consumption = static_flows[cols["consumption"]]
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/frame.py", line 3893, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3798, in get_loc
raise KeyError(key) from err
KeyError: 'consumption'

2023-12-21 15:29:21.554135 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 15:29:21.556754 INFO AppDaemon: App initialization complete

from pv_opt.

from pv_opt.

Yes, and checked the sensor and looks ok to me.

from pv_opt.

from pv_opt.

Brilliant, and thanks again for your help!

I can't seem to get AppDaemon logging to work at all. Added the config as per your instructions, created the share/log directories but still nothing. Also note when AppDaemon starts it doesn't show INFO AppDaemon: Added log: PV_Opt. I know appdaemon.config is being read and using secrets ok. Checked permissions on the folders (root with drwxr-xr-x). Resorted to asking ChatGRP and everything seems correct, just not writing any files.

from pv_opt.

from pv_opt.

OK - try 3.4.5-alpha. You'll need to enable pre-releases in HACS to see it. I'm wondering if this and your logging issues might be related?

from pv_opt.

Loaded 3.4.5.... I will also reinstall AppDaemon & MQTT and try again later to see if that improves the situation.

023-12-21 20:11:44.719205 INFO pv_opt: Exposing config to Home Assistant:
2023-12-21 20:11:44.721499 INFO pv_opt: ----------------------------------
2023-12-21 20:11:44.821291 INFO pv_opt:
2023-12-21 20:11:44.835804 INFO pv_opt: Config Item HA Entity Current State
2023-12-21 20:11:44.844503 INFO pv_opt: ----------- --------- -------------
2023-12-21 20:11:44.853338 INFO pv_opt: forced_charge switch.pvopt_forced_charge on
2023-12-21 20:11:44.867053 INFO pv_opt: forced_discharge switch.pvopt_forced_discharge off
2023-12-21 20:11:44.884608 INFO pv_opt: read_only switch.pvopt_read_only on
2023-12-21 20:11:44.896332 INFO pv_opt: allow_cyclic switch.pvopt_allow_cyclic off
2023-12-21 20:11:44.909395 INFO pv_opt: optimise_frequency_minutes number.pvopt_optimise_frequency_minutes 10
2023-12-21 20:11:44.919161 INFO pv_opt: slot_threshold_p number.pvopt_slot_threshold_p 1.0
2023-12-21 20:11:44.935128 INFO pv_opt: day_of_week_weighting number.pvopt_day_of_week_weighting 0.5
2023-12-21 20:11:44.943968 INFO pv_opt: pass_threshold_p number.pvopt_pass_threshold_p 4.0
2023-12-21 20:11:44.954876 INFO pv_opt: battery_capacity_wh number.pvopt_battery_capacity_wh 10000
2023-12-21 20:11:44.961044 INFO pv_opt: inverter_efficiency_percent number.pvopt_inverter_efficiency_percent 97
2023-12-21 20:11:44.980092 INFO pv_opt: charger_efficiency_percent number.pvopt_charger_efficiency_percent 91
2023-12-21 20:11:44.992809 INFO pv_opt: charger_power_watts number.pvopt_charger_power_watts 3000
2023-12-21 20:11:45.005762 INFO pv_opt: inverter_power_watts number.pvopt_inverter_power_watts 3600
2023-12-21 20:11:45.017238 INFO pv_opt: inverter_loss_watts number.pvopt_inverter_loss_watts 100
2023-12-21 20:11:45.030485 INFO pv_opt: solar_forecast select.pvopt_solar_forecast Solcast
2023-12-21 20:11:45.043515 INFO pv_opt: consumption_history_days number.pvopt_consumption_history_days 7
2023-12-21 20:11:45.054010 INFO pv_opt: consumption_margin number.pvopt_consumption_margin 10
2023-12-21 20:11:45.066297 INFO pv_opt: consumption_grouping select.pvopt_consumption_grouping mean
2023-12-21 20:11:45.081905 INFO pv_opt: forced_power_group_tolerance number.pvopt_forced_power_group_tolerance 100
2023-12-21 20:11:55.097613 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7f23c50ee040>) took too long (10 seconds), cancelling the task...
2023-12-21 20:11:55.103111 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 20:11:55.104264 WARNING pv_opt: Unexpected error in worker for App pv_opt:
2023-12-21 20:11:55.104965 WARNING pv_opt: Worker Ags: {}
2023-12-21 20:11:55.107079 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 20:11:55.112034 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 273, in initialize
self._estimate_capacity()
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 307, in _estimate_capacity
self.hass2df(
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 235, in hass2df
df = pd.DataFrame(hist[0]).set_index("last_updated")["state"]
~~~~^^^
TypeError: 'NoneType' object is not subscriptable

2023-12-21 20:11:55.118805 WARNING pv_opt: ------------------------------------------------------------
2023-12-21 20:11:55.156563 WARNING AppDaemon: Excessive time spent in utility loop: 13552.0ms, 13551.0ms in check_app_updates(), 1.0ms in other

from pv_opt.

from pv_opt.

Hello, I've spent hours trying to get AppDaemon to log, still not working.
Everything is upto date.

Reinstalled AppDaemon, MQTT - runs HelloWorld to screen ok, but refuses to log with appdaemon.yaml log parameters and no idea why. It is using appdaemon.yaml as the secrets.yaml is loaded. I deinstalled PV_opt restarted multiple times, and ran HelloWorld app. This logs to file if I explicitly tell it in hello.py.

secrets: /homeassistant/secrets.yaml
appdaemon:
latitude: 1.0
longitude: 1.0
elevation: 10
time_zone: Europe/London
thread_duration_warning_threshold: 45
app_dir: /homeassistant/appdaemon/apps
plugins:
HASS:
type: hass
app_init_delay: 20
MQTT:
type: mqtt
namespace: mqtt
verbose: True
client_host: core-mosquitto
client_port: 1883
client_id: localad
event_name: MQTT_MESSAGE
client_topics: NONE
client_user: !secret mqtt-username
client_password: !secret mqtt-password
logs:
main_log:
filename: /config/appdaemon/logs/main.log
date_format: "%H:%M:%S"
error_log:
filename: /config/appdaemon/logs/error.log
date_format: "%H:%M:%S"
pv_opt_log:
name: PV_Opt
filename: /share/logs/pv_opt.log
date_format: "%H:%M:%S"
format: "{asctime} {levelname:>8s}: {message}"

http:
url: http://127.0.0.1:5050
admin:
api:
hadashboard:

Installed PV_OPT 3.4.4 and then Alpha 3.4.5.

Output from 3.4.4:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service base-addon-banner: starting

Add-on: AppDaemon
Python Apps and Dashboard using AppDaemon 4.x for Home Assistant

Add-on version: 0.16.0
You are running the latest version of this add-on.
System: Home Assistant OS 11.2 (amd64 / qemux86-64)
Home Assistant Core: 2023.12.3
Home Assistant Supervisor: 2023.12.0

Please, share the above information when looking for help
or support in, e.g., GitHub, forums or the Discord chat.

s6-rc: info: service base-addon-banner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service base-addon-log-level: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service base-addon-log-level successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-appdaemon: starting
Looking in indexes: https://pypi.org/simple, https://wheels.home-assistant.io/musllinux-index/
Collecting numpy==1.25
Downloading https://wheels.home-assistant.io/musllinux-index/numpy-1.25.0-cp311-cp311-musllinux_1_2_x86_64.whl (17.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.9/17.9 MB 7.5 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-1.25.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Looking in indexes: https://pypi.org/simple, https://wheels.home-assistant.io/musllinux-index/
Collecting pandas
Downloading https://wheels.home-assistant.io/musllinux-index/pandas-2.1.4-cp311-cp311-musllinux_1_2_x86_64.whl (13.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.8/13.8 MB 7.4 MB/s eta 0:00:00
Requirement already satisfied: numpy<2,>=1.23.2 in /usr/lib/python3.11/site-packages (from pandas) (1.25.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/lib/python3.11/site-packages (from pandas) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3.11/site-packages (from pandas) (2023.3.post1)
Collecting tzdata>=2022.1 (from pandas)
Downloading https://wheels.home-assistant.io/musllinux-index/tzdata-2023.3-py2.py3-none-any.whl (341 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 6.8 MB/s eta 0:00:00
Requirement already satisfied: six>=1.5 in /usr/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)
Installing collected packages: tzdata, pandas
Successfully installed pandas-2.1.4 tzdata-2023.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
s6-rc: info: service init-appdaemon successfully started
s6-rc: info: service appdaemon: starting
s6-rc: info: service appdaemon successfully started
s6-rc: info: service legacy-services: starting
[13:46:13] INFO: Starting AppDaemon...
s6-rc: info: service legacy-services successfully started
...
2023-12-22 13:46:29.255404 INFO pv_opt:
2023-12-22 13:46:29.257836 INFO pv_opt: Exposing config to Home Assistant:
2023-12-22 13:46:29.261043 INFO pv_opt: ----------------------------------
2023-12-22 13:46:29.316260 INFO pv_opt:
2023-12-22 13:46:29.319417 INFO pv_opt: Config Item HA Entity Current State
2023-12-22 13:46:29.322388 INFO pv_opt: ----------- --------- -------------
2023-12-22 13:46:29.325000 INFO pv_opt: forced_charge switch.pvopt_forced_charge on
2023-12-22 13:46:29.329683 INFO pv_opt: forced_discharge switch.pvopt_forced_discharge on
2023-12-22 13:46:29.335449 INFO pv_opt: read_only switch.pvopt_read_only on
2023-12-22 13:46:29.339649 INFO pv_opt: allow_cyclic switch.pvopt_allow_cyclic off
2023-12-22 13:46:29.344431 INFO pv_opt: optimise_frequency_minutes number.pvopt_optimise_frequency_minutes 10
2023-12-22 13:46:29.348596 INFO pv_opt: slot_threshold_p number.pvopt_slot_threshold_p 1.0
2023-12-22 13:46:29.352522 INFO pv_opt: day_of_week_weighting number.pvopt_day_of_week_weighting 0.5
2023-12-22 13:46:29.356320 INFO pv_opt: pass_threshold_p number.pvopt_pass_threshold_p 4.0
2023-12-22 13:46:29.364329 INFO pv_opt: battery_capacity_wh number.pvopt_battery_capacity_wh 10000
2023-12-22 13:46:29.368895 INFO pv_opt: inverter_efficiency_percent number.pvopt_inverter_efficiency_percent 97
2023-12-22 13:46:29.374871 INFO pv_opt: charger_efficiency_percent number.pvopt_charger_efficiency_percent 91
2023-12-22 13:46:29.378845 INFO pv_opt: charger_power_watts number.pvopt_charger_power_watts 3000
2023-12-22 13:46:29.382491 INFO pv_opt: inverter_power_watts number.pvopt_inverter_power_watts 3600
2023-12-22 13:46:29.391063 INFO pv_opt: inverter_loss_watts number.pvopt_inverter_loss_watts 100
2023-12-22 13:46:29.396202 INFO pv_opt: solar_forecast select.pvopt_solar_forecast Solcast
2023-12-22 13:46:29.407932 INFO pv_opt: consumption_history_days number.pvopt_consumption_history_days 7
2023-12-22 13:46:29.413366 INFO pv_opt: consumption_margin number.pvopt_consumption_margin 10
2023-12-22 13:46:29.418037 INFO pv_opt: consumption_grouping select.pvopt_consumption_grouping mean
2023-12-22 13:46:29.422152 INFO pv_opt: forced_power_group_tolerance number.pvopt_forced_power_group_tolerance 100
2023-12-22 13:46:29.444068 INFO pv_opt:
2023-12-22 13:46:29.452518 INFO pv_opt: Loading Contract:
2023-12-22 13:46:29.475678 INFO pv_opt: -----------------
2023-12-22 13:46:29.486319 INFO pv_opt: Trying to auto detect Octopus tariffs:
2023-12-22 13:46:29.493281 INFO pv_opt: Found import entity event.octopus_energy_electricity_17m1043118_1012890530550_current_day_rates
2023-12-22 13:46:29.495486 INFO pv_opt: Found export entity event.octopus_energy_electricity_17m1043118_1050002432148_export_current_day_rates
2023-12-22 13:46:30.094707 INFO pv_opt: Contract tariffs loaded OK
2023-12-22 13:46:30.097654 INFO pv_opt: Import: E-1R-AGILE-FLEX-22-11-25-A
2023-12-22 13:46:30.099612 INFO pv_opt: Export: E-1R-OUTGOING-FIX-12M-19-05-13-A
2023-12-22 13:46:30.101561 INFO pv_opt: AGILE tariff detected. Rates will update at 16:00 daily
2023-12-22 13:46:30.103455 INFO pv_opt:
2023-12-22 13:46:30.117151 INFO pv_opt: Previous message repeated 1 times
2023-12-22 13:46:30.118237 INFO pv_opt: Found Octopus Savings Events entity: event.octopus_energy_a_f1f23c9b_octoplus_saving_session_events
2023-12-22 13:46:30.122373 INFO pv_opt:
2023-12-22 13:46:30.124609 INFO pv_opt: No upcoming Octopus Saving Events detected or joined:
2023-12-22 13:46:30.127266 INFO pv_opt: Finished loading contract
2023-12-22 13:46:31.773559 INFO pv_opt:
2023-12-22 13:46:31.775719 INFO pv_opt: Running initial Optimisation:
2023-12-22 13:46:31.777799 INFO pv_opt:
2023-12-22 13:46:31.789064 INFO pv_opt: Previous message repeated 1 times
2023-12-22 13:46:31.789916 INFO pv_opt: Found Octopus Savings Events entity: event.octopus_energy_a_f1f23c9b_octoplus_saving_session_events
2023-12-22 13:46:31.793560 INFO pv_opt:
2023-12-22 13:46:31.795713 INFO pv_opt: No upcoming Octopus Saving Events detected or joined:
2023-12-22 13:46:31.797445 INFO pv_opt: Starting Opimisation with discharge enabled
2023-12-22 13:46:31.799151 INFO pv_opt: --------------------------------------------------
2023-12-22 13:46:31.882572 INFO pv_opt: Solcast forecast loaded OK
2023-12-22 13:46:31.886296 INFO pv_opt: Getting expected consumption data
2023-12-22 13:46:41.896722 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7f8b8e031140>) took too long (10 seconds), cancelling the task...
2023-12-22 13:46:41.903288 ERROR pv_opt: Unable to get historical consumption from sensor.solis_house_load. 'NoneType' object is not subscriptable
2023-12-22 13:46:45.692098 INFO pv_opt: Initial SOC: 96.0
2023-12-22 13:46:45.697808 INFO pv_opt: Calculating Base flows
2023-12-22 13:46:45.708218 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 13:46:45.709497 WARNING pv_opt: Unexpected error running initialize() for pv_opt
2023-12-22 13:46:45.710034 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 13:46:45.755932 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3791, in get_loc
return self._engine.get_loc(casted_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'consumption'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 294, in initialize
self.optimise()
File "/usr/lib/python3.11/site-packages/appdaemon/adbase.py", line 35, in f_app_lock
return f(*args, **kw)
^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 1282, in optimise
self.base = self.pv_system.flows(
^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pvpy.py", line 458, in flows
consumption = static_flows[cols["consumption"]]
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/frame.py", line 3893, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3798, in get_loc
raise KeyError(key) from err
KeyError: 'consumption'

2023-12-22 13:46:45.765169 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 13:46:45.766491 INFO AppDaemon: App initialization complete

Output from 3.4.5:
...
2023-12-22 14:15:09.643479 INFO AppDaemon: Added log: Error
2023-12-22 14:15:09.644795 INFO AppDaemon: Added log: Access
2023-12-22 14:15:09.646200 INFO AppDaemon: Added log: Diag
2023-12-22 14:15:12.917068 INFO AppDaemon: Loading Plugin HASS using class HassPlugin from module hassplugin
2023-12-22 14:15:14.299411 INFO HASS: HASS Plugin Initializing
2023-12-22 14:15:14.300436 WARNING HASS: ha_url not found in HASS configuration - module not initialized
2023-12-22 14:15:14.302853 INFO HASS: HASS Plugin initialization complete
2023-12-22 14:15:14.303887 INFO AppDaemon: Loading Plugin MQTT using class MqttPlugin from module mqttplugin
2023-12-22 14:15:14.525077 INFO MQTT: MQTT Plugin Initializing
2023-12-22 14:15:14.534471 INFO MQTT: Using 'localad/status' as Will Topic
2023-12-22 14:15:14.535904 INFO MQTT: Using 'localad/status' as Birth Topic
2023-12-22 14:15:14.537367 INFO AppDaemon: HTTP is disabled
2023-12-22 14:15:14.643760 INFO HASS: Connected to Home Assistant 2023.12.3
2023-12-22 14:15:14.799436 INFO AppDaemon: App 'hello_world' added
2023-12-22 14:15:14.810623 INFO AppDaemon: App 'pv_opt' added
2023-12-22 14:15:14.822280 INFO AppDaemon: Found 2 active apps
2023-12-22 14:15:14.823817 INFO AppDaemon: Found 0 inactive apps
2023-12-22 14:15:14.826290 INFO AppDaemon: Found 2 global libraries
2023-12-22 14:15:14.830604 INFO AppDaemon: Starting Apps with 2 workers and 2 pins
2023-12-22 14:15:15.107557 INFO MQTT: Connected to Broker at URL core-mosquitto:1883
2023-12-22 14:15:15.134301 INFO AppDaemon: Got initial state from namespace mqtt
2023-12-22 14:15:15.135573 INFO MQTT: MQTT Plugin initialization complete
2023-12-22 14:15:15.196811 INFO HASS: Evaluating startup conditions
2023-12-22 14:15:15.290813 INFO HASS: Startup condition met: hass state=RUNNING
2023-12-22 14:15:15.291945 INFO HASS: All startup conditions met
2023-12-22 14:15:15.495964 INFO AppDaemon: Got initial state from namespace default
2023-12-22 14:15:16.839902 INFO AppDaemon: Scheduler running in realtime
2023-12-22 14:15:16.842546 INFO AppDaemon: Adding /homeassistant/appdaemon/apps to module import path
2023-12-22 14:15:16.845012 INFO AppDaemon: Adding /homeassistant/appdaemon/apps/pv_opt to module import path
2023-12-22 14:15:16.846454 INFO AppDaemon: Adding /homeassistant/appdaemon/apps/pv_opt/config to module import path
2023-12-22 14:15:16.853839 INFO AppDaemon: Loading App Module: /homeassistant/appdaemon/apps/hello.py
2023-12-22 14:15:16.968930 WARNING AppDaemon: No app description found for: /homeassistant/appdaemon/apps/pv_opt/solis.py - ignoring
2023-12-22 14:15:16.971708 INFO AppDaemon: Loading App Module: /homeassistant/appdaemon/apps/pv_opt/pv_opt.py
2023-12-22 14:15:16.979564 WARNING Error: ------------------------------------------------------------
2023-12-22 14:15:16.980663 WARNING Error: Unexpected error loading module: /homeassistant/appdaemon/apps/pv_opt/pv_opt.py:
2023-12-22 14:15:16.981871 WARNING Error: ------------------------------------------------------------
2023-12-22 14:15:16.991734 WARNING Error: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 1000, in check_app_updates
await utils.run_in_executor(self, self.read_app, mod["name"], mod["reload"])
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 783, in read_app
self.modules[module_name] = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 936, in exec_module
File "", line 1074, in get_code
File "", line 1004, in source_to_code
File "", line 241, in _call_with_frames_removed
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 239
self.log(f">>> {hist})
^
SyntaxError: unterminated string literal (detected at line 239)

2023-12-22 14:15:17.003187 WARNING Error: ------------------------------------------------------------
2023-12-22 14:15:17.004129 WARNING AppDaemon: Removing associated apps:
2023-12-22 14:15:17.004987 WARNING AppDaemon: pv_opt
2023-12-22 14:15:17.005957 INFO AppDaemon: Loading Global Module: /homeassistant/appdaemon/apps/pv_opt/pvpy.py
2023-12-22 14:15:19.773944 WARNING AppDaemon: Unable to find app solis in dependencies for pv_opt
2023-12-22 14:15:19.775608 WARNING AppDaemon: Ignoring app pv_opt
2023-12-22 14:15:19.776956 INFO AppDaemon: Loading app hello_world using class HelloWorld from module hello
2023-12-22 14:15:19.782597 INFO AppDaemon: Loading app pv_opt using class PVOpt from module pv_opt
2023-12-22 14:15:19.792856 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 14:15:19.795064 WARNING pv_opt: Unexpected error initializing app: pv_opt:
2023-12-22 14:15:19.798180 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 14:15:19.802105 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 1035, in check_app_updates
await self.init_object(app)
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 323, in init_object
modname = await utils.run_in_executor(self, import, app_args["module"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 239
self.log(f">>> {hist})
^
SyntaxError: unterminated string literal (detected at line 239)

2023-12-22 14:15:19.809198 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 14:15:19.811399 INFO AppDaemon: Calling initialize() for hello_world
2023-12-22 14:15:20.226807 INFO hello_world: Hello from AppDaemon
2023-12-22 14:15:20.237127 INFO hello_world: You are now ready to run Apps!
2023-12-22 14:15:20.250206 WARNING AppDaemon: Unable to find module pv_opt - initialize() skipped
2023-12-22 14:15:20.252548 INFO AppDaemon: App initialization complete

from pv_opt.

Fixed in 3.4.5-beta.

from pv_opt.

ok, getting a bit further now...

2023-12-21 23:47:52.111627+00:00 0.0
2023-12-22 00:21:48.831425+00:00 0.0
2023-12-22 13:16:26.437224+00:00 0.0
2023-12-22 14:00:15.426772+00:00 0.0
Name: state, dtype: object
2023-12-22 15:48:54.580409 INFO pv_opt:
2023-12-22 15:48:54.589988 INFO pv_opt: Running initial Optimisation:
2023-12-22 15:48:54.601006 INFO pv_opt:
2023-12-22 15:48:54.657011 INFO pv_opt: Previous message repeated 1 times
2023-12-22 15:48:54.661888 INFO pv_opt: Found Octopus Savings Events entity: event.octopus_energy_a_f1f23c9b_octoplus_saving_session_events
2023-12-22 15:48:54.676302 INFO pv_opt:
2023-12-22 15:48:54.689303 INFO pv_opt: No upcoming Octopus Saving Events detected or joined:
2023-12-22 15:48:54.691860 INFO pv_opt: Starting Opimisation with discharge enabled
2023-12-22 15:48:54.693705 INFO pv_opt: --------------------------------------------------
2023-12-22 15:48:54.710210 INFO pv_opt: Getting Solcast data
2023-12-22 15:48:54.812367 INFO pv_opt: Solcast forecast loaded OK
2023-12-22 15:48:54.822429 INFO pv_opt: Getting expected consumption data
2023-12-22 15:48:54.829857 INFO pv_opt: >>> Getting 7 history for sensor.solis_house_load
2023-12-22 15:48:54.845770 INFO pv_opt: >>> Entity exits: True
2023-12-22 15:49:04.883990 WARNING pv_opt: Coroutine (<coroutine object Hass.get_history at 0x7f28e1a9cc40>) took too long (10 seconds), cancelling the task...
2023-12-22 15:49:04.888581 INFO pv_opt: >>> None
2023-12-22 15:49:04.900003 ERROR pv_opt: Unable to get historical consumption from sensor.solis_house_load. 'NoneType' object is not subscriptable
2023-12-22 15:49:04.919677 INFO pv_opt: >>> Getting 1 history for sensor.solis_battery_soc
2023-12-22 15:49:04.922910 INFO pv_opt: >>> Entity exits: True
2023-12-22 15:49:08.552243 INFO pv_opt: >>> [[{'entity_id': 'sensor.solis_battery_soc', 'state': '87', 'attributes': {'unit_of_measurement': '%', 'device_class': 'battery', 'friendly_name': 'Solis Battery SOC'}, 'last_changed': '2023-
.....
22T14:00:15.422263+00:00'}, {'entity_id': 'sensor.solis_battery_soc', 'state': '95', 'attributes': {'unit_of_measurement': '%', 'device_class': 'battery', 'friendly_name': 'Solis Battery SOC'}, 'last_changed': '2023-12-22T15:40:24.697935+00:00', 'last_updated': '2023-12-22T15:40:24.697935+00:00'}]]
2023-12-22 15:49:08.614070 INFO pv_opt: >>> last_updated
2023-12-21 15:49:04+00:00 87
2023-12-21 16:03:23.829776+00:00 86
2023-12-21 16:08:59.504421+00:00 85
2023-12-21 16:15:49.704407+00:00 84
2023-12-21 16:21:59.704229+00:00 83
...
2023-12-22 13:16:26.427414+00:00 97
2023-12-22 13:24:21.887546+00:00 96
2023-12-22 13:59:56.604727+00:00 unknown
2023-12-22 14:00:15.422263+00:00 96
2023-12-22 15:40:24.697935+00:00 95
Name: state, Length: 218, dtype: object
2023-12-22 15:49:08.657432 INFO pv_opt: >>> last_updated
2023-12-21 15:49:04+00:00 87
2023-12-21 16:03:23.829776+00:00 86
2023-12-21 16:08:59.504421+00:00 85
2023-12-21 16:15:49.704407+00:00 84
2023-12-21 16:21:59.704229+00:00 83
..
2023-12-22 11:02:24.380767+00:00 97
2023-12-22 13:16:26.427414+00:00 97
2023-12-22 13:24:21.887546+00:00 96
2023-12-22 14:00:15.422263+00:00 96
2023-12-22 15:40:24.697935+00:00 95
Name: state, Length: 209, dtype: object
2023-12-22 15:49:08.717058 INFO pv_opt: Initial SOC: 95.5
2023-12-22 15:49:08.722782 INFO pv_opt: Calculating Base flows
2023-12-22 15:49:08.762175 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 15:49:08.771139 WARNING pv_opt: Unexpected error running initialize() for pv_opt
2023-12-22 15:49:08.772163 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 15:49:08.891102 WARNING pv_opt: Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3791, in get_loc
return self._engine.get_loc(casted_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "index.pyx", line 152, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 181, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'consumption'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/appdaemon/app_management.py", line 162, in initialize_app
await utils.run_in_executor(self, init)
File "/usr/lib/python3.11/site-packages/appdaemon/utils.py", line 304, in run_in_executor
response = future.result()
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 305, in initialize
self.optimise()
File "/usr/lib/python3.11/site-packages/appdaemon/adbase.py", line 35, in f_app_lock
return f(*args, **kw)
^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pv_opt.py", line 1293, in optimise
self.base = self.pv_system.flows(
^^^^^^^^^^^^^^^^^^^^^
File "/homeassistant/appdaemon/apps/pv_opt/pvpy.py", line 458, in flows
consumption = static_flows[cols["consumption"]]
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/frame.py", line 3893, in getitem
indexer = self.columns.get_loc(key)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3798, in get_loc
raise KeyError(key) from err
KeyError: 'consumption'

2023-12-22 15:49:08.914489 WARNING pv_opt: ------------------------------------------------------------
2023-12-22 15:49:08.915360 INFO AppDaemon: App initialization complete

from pv_opt.

The 'unable to get historic data' message relating to solis_house_load again. It's definitely there...

From Dev tools:

from pv_opt.

from pv_opt.

No it's not. That is in the SOC data which it gets succesfully. I'm wondering if there are just too many data points in your history. Can you try adding this to config.yaml after device_name: solis:

consumption_history_days: 2

I have another workaround which is to use the cumulative daily data instead of the instantaneous load. It's arguably a better approach anyway - I'll release another version using that but it will need a bit of testing first.

from pv_opt.

Try 3.4.5-beta-02 which loads consumption from the daily total not the instantaneous power

from pv_opt.

Brilliant, it's now working! Thankyou so much for this. I've put it into Readonly mode to work out what wizzardy you have going on. Probably a mindset change on my behalf, as usually I simply set the inverter to start battery charging (usually last thing at night) until it's 100%. That's my winter mode, but for spring/summer, I'd setup a good, medium, bad solar forecast and adjusted the charing hours according. Very much looking forward to seeing how this very intelligent approach operates.

Truly appreciate your fast responses. Thank you.

(I may have more questions, hope you don't mind!)

from pv_opt.

from pv_opt.