green garbage schedule does not work about fkf-garbage-collection HOT 32 CLOSED

I fixed the debugging information and added more details.

Please upgrade your component, enable debugging, restart HASS and check the logs.

from fkf-garbage-collection.

Could you please provide the publicplace and a fake but existing housenr to be able to reproduce and investigate the issue.

from fkf-garbage-collection.

Sure, thank you.
Lajosház utca
2

from fkf-garbage-collection.

It works for me: querying today, Friday, it gives me "next_green_days":5 which makes Wednesday which matches what https://www.fkf.hu/kerti-zoldhulladek-korzetek-XVII-kerulet shows.

(ha_venv3_12) pi@raspberrypi:~/homeassistant_test $date
Fri Mar 22 09:23:31 CET 2024
(ha_venv3_12) pi@raspberrypi:~/homeassistant_test $cat sensors/fkf_test.yaml
platform: fkf_garbage_collection
name: 'fkf_test'
zipcode: '1171'
publicplace: 'Lajosház utca'
housenr: '2'
calendar: false
calendar_lang: hu
green: true
greencolor: 'lila'
(ha_venv3_12) pi@raspberrypi:~/homeassistant_test $curl -H @hass_header.txt http://192.168.1.156:9000/api/states/sensor.fkf_test
{"entity_id":"sensor.fkf_test","state":"0","attributes":{"items":7,"in0":0,"day0":"Friday","date0":"2024.03.22","garbage0":"communal","in1":5,"day1":"Wednesday","date1":"2024.03.27","garbage1":"green","in2":7,"day2":"Friday","date2":"2024.03.29","garbage2":"communal","in3":14,"day3":"Friday","date3":"2024.04.05","garbage3":"communal","in4":18,"day4":"Tuesday","date4":"2024.04.09","garbage4":"selective","in5":21,"day5":"Friday","date5":"2024.04.12","garbage5":"communal","in6":28,"day6":"Friday","date6":"2024.04.19","garbage6":"communal","current":"current","next_communal_days":0,"next_green_days":5,"next_selective_days":18,"calendar_lang":"hu","provider":"Data provided by fkf.hu","icon":"mdi:trash-can-outline","friendly_name":"fkf_test"},"last_changed":"2024-03-22T08:17:19.356108+00:00","last_updated":"2024-03-22T08:17:19.356108+00:00","context":{"id":"01HSJKRQZWFD6SXBGDR0VMPKQX","parent_id":null,"user_id":null}}(ha_venv3_12) pi@raspberrypi:~/homeassistant_test $

from fkf-garbage-collection.

Ok. I am testing with the same setting as you.
In my config the "housenr" as different, and the "calendar" and "calendar_lang" was missing, but they are optional.
Is there a way to force to reload the sensor? After a HA restart it take always atleast 1 hour to show values.

from fkf-garbage-collection.

The 1 hour has passed, any results on your tests?

from fkf-garbage-collection.

Yes, it was update, exactly one hour later.
But no luck. Only the normal and selective garbage collectionbis displayed

from fkf-garbage-collection.

Can you check the homeassistant.log for possible errors on fkf component?

from fkf-garbage-collection.

I did earlier also, i have not mentioned, but i could not find anything, except it is a custom component and not tested with HA, blabla as all other, and later that it is loading slower, more the 60 sec, etc

from fkf-garbage-collection.

Could you please enable logging for this component in configuration.yaml and restart hass

logger:
  logs:
    custom_components.fkf_garbage_collection.sensor: debug

from fkf-garbage-collection.

Ok. I did. I will be busy in the next couple of hours, but tonight i will send you the result.
Thank you for your support!

from fkf-garbage-collection.

This is what i have found, but nothing else😒

2024-03-22 14:04:48.243 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration fkf_garbage_collection which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
...
2024-03-22 14:05:54.215 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform fkf_garbage_collection is taking over 10 seconds.
...
2024-03-22 14:06:05.724 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error for fetching green schedule fromhttps://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
...
2024-03-22 14:06:14.225 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error to fkf.hu
2024-03-22 14:06:14.225 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Fetch info for 1171/Lajosh谩z---utca/2:
...
2024-03-22 15:06:17.709 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosh谩z---utca: 0: communal
2024-03-22 15:06:17.710 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosh谩z---utca: 7: communal
ETC................

from fkf-garbage-collection.

Seems an intermittent connection issue.

from fkf-garbage-collection.

I have noticed also the "Connection error to fkf.hu" error, but it could connect the fkf site, and communal garbage and selective garbage dates are available. Only the green has issues and it is constant.

from fkf-garbage-collection.

In today's release I added retries for fetching green garbage schedule.

Please upgrade your component, enable debugging, restart HASS and check the logs.

Please get back to me with results.

from fkf-garbage-collection.

Last night i have installed the upgrade and restarted the HA, but i had not time to test it. This morning i noticed that the green is still not working. I will grep the log today and send it to you.

from fkf-garbage-collection.

This is what I have found in the log:

2024-03-26 08:17:10.254 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-26 08:17:10.267 WARNING (MainThread) [homeassistant.util.async_] Detected blocking call to sleep inside the event loop by custom integration 'fkf_garbage_collection' at custom_components/fkf_garbage_collection/sensor.py, line 178: time.sleep(10), please report it to the author of the 'fkf_garbage_collection' custom integration
2024-03-26 08:17:20.269 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-26 08:17:20.283 WARNING (MainThread) [homeassistant.util.async_] Detected blocking call to sleep inside the event loop by custom integration 'fkf_garbage_collection' at custom_components/fkf_garbage_collection/sensor.py, line 178: time.sleep(10), please report it to the author of the 'fkf_garbage_collection' custom integration

+2 more times: attempt 2 and 3

from fkf-garbage-collection.

Pushed a new release to address the blocking event loop issue.

Please upgrade your component, enable debugging, restart HASS and check the logs. I do not have connection errors so I can't test that part. But apparently you have so please get back to me with test results.

from fkf-garbage-collection.

2024-03-26 10:06:15.389 WARNING (MainThread) [py.warnings] /config/custom_components/fkf_garbage_collection/sensor.py:187: SyntaxWarning: invalid escape sequence '\ '
matchre = re.compile('[A-ZÁÉÖŐÜÚŰ]*\ *')

2024-03-26 10:06:15.391 WARNING (MainThread) [py.warnings] /config/custom_components/fkf_garbage_collection/sensor.py:260: SyntaxWarning: invalid escape sequence '\s'
s1 = re.sub("\s{2,}"," ",s)
...
2024-03-26 10:07:19.054 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-26 10:07:19.054 ERROR (MainThread) [homeassistant.components.sensor] fkf_garbage_collection: Error on device update!
Traceback (most recent call last):
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 170, in async_get_fkfdata
async with _session.get(url, timeout=HTTP_TIMEOUT) as response:
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1194, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
with timer:
File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in exit
raise asyncio.TimeoutError from None
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 682, in _async_add_entity
await entity.async_device_update(warning=False)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1262, in async_device_update
await self.async_update()
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 422, in async_update
fkfdata = await async_get_fkfdata(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 181, in async_get_fkfdata
await hass.async_add_executor_job(_sleep, 10)
^^^^
NameError: name 'hass' is not defined. Did you mean: 'self.hass'?

from fkf-garbage-collection.

Yet another release fixing things.

Please let me know if there are still exceptions on connection issues.

from fkf-garbage-collection.

Sure! Thank you. I am checking

from fkf-garbage-collection.

at the start it is the same:
2024-03-26 10:48:01.670 WARNING (MainThread) [py.warnings] /config/custom_components/fkf_garbage_collection/sensor.py:187: SyntaxWarning: invalid escape sequence '\ '
matchre = re.compile('[A-ZÁÉÖŐÜÚŰ]*\ *')

2024-03-26 10:48:01.672 WARNING (MainThread) [py.warnings] /config/custom_components/fkf_garbage_collection/sensor.py:260: SyntaxWarning: invalid escape sequence '\s'
s1 = re.sub("\s{2,}"," ",s)

But the second exception is solved, but still no data:
2024-03-26 10:49:03.022 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-26 10:49:03.023 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
... attempt 2 and 3 are the same ...
2024-03-26 10:49:28.673 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 3: communal
2024-03-26 10:49:28.674 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 10: communal
2024-03-26 10:49:28.674 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 14: selective
2024-03-26 10:49:28.674 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 17: communal
2024-03-26 10:49:28.675 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 24: communal
2024-03-26 10:49:28.675 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 31: communal
2024-03-26 10:49:28.683 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 3: communal
2024-03-26 10:49:28.684 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 10: communal
2024-03-26 10:49:28.684 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 14: selective
2024-03-26 10:49:28.684 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 17: communal
2024-03-26 10:49:28.685 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 24: communal
2024-03-26 10:49:28.685 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 31: communal

from fkf-garbage-collection.

At least now it works. Thanks for testing it.
I'd recommend reviewing your internet connectivity as the requests does not return http code 200 (OK).

from fkf-garbage-collection.

:o
What should I check? It seems to be working. "curl -XGET 'https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet'" response is valid: "......"

from fkf-garbage-collection.

You could try to add after line 177 a line like (watch out for indentation):

_LOGGER.debug("response code: " + str(response.status))

and restart HASS.

I might add more debugging information in the next release.

from fkf-garbage-collection.

It looks like timeout issue, so the response object remains empty:

2024-03-26 23:21:43.056 WARNING (SyncWorker_3) [homeassistant.loader] We found a custom integration fkf_garbage_collection which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-03-26 23:23:09.130 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Connection error on fetch attempt 1 for green schedule from https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-26 23:23:09.130 ERROR (MainThread) [homeassistant.components.sensor] fkf_garbage_collection: Error on device update!
Traceback (most recent call last):
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 170, in async_get_fkfdata
async with _session.get(url, timeout=HTTP_TIMEOUT) as response:
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 1194, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 605, in _request
await resp.start(conn)
File "/usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 961, in start
with self._timer:
File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in exit
raise asyncio.TimeoutError from None
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 682, in _async_add_entity
await entity.async_device_update(warning=False)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1262, in async_device_update
await self.async_update()
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 423, in async_update
fkfdata = await async_get_fkfdata(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/fkf_garbage_collection/sensor.py", line 178, in async_get_fkfdata
_LOGGER.debug("response code: " + str(response.status))
^^^^^^^^
UnboundLocalError: cannot access local variable 'response' where it is not associated with a value
2024-03-26 23:23:09.945 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform fkf_garbage_collection is taking over 10 seconds.

from fkf-garbage-collection.

It does not show the error:
2024-03-27 15:19:10.790 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Fetch attempt 1: for green schedule https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
... and then the next fkf related logs are these:
2024-03-27 15:19:31.137 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 2: communal
2024-03-27 15:19:31.137 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Lajosház---utca: 9: communal

from fkf-garbage-collection.

Added more debugging info to the latest release. Hopefully that will reveal what connection issue you have.

from fkf-garbage-collection.

it looks like the 'err' is null:
2024-03-28 23:16:45.751 DEBUG (MainThread) [custom_components.fkf_garbage_collection.sensor] Fetch attempt 1 failed for green schedule https://www.fkf.hu/kerti-zoldhulladek-korzetek-xvii-kerulet
2024-03-28 23:16:45.752 ERROR (MainThread) [custom_components.fkf_garbage_collection.sensor]

from fkf-garbage-collection.

I made connection error exception handling generic. And also tested it this time before release :)

from fkf-garbage-collection.

Hi amaximus!
Thanks to the new logging. I could noticed that the problem is with timeout. During the startup process it tried to load the page 3 times, but all of them it timed out. I modified the HTTP_TIMEOUT variable from 5 (secunds) to 60, and now it works.
Maybe I have too many addons and intagration and the cpu usage is very high during the startup, and thats why the 5s is not enough in my case.
Thank you very much for the support and for this component.
I whish you a Happy Easter!

from fkf-garbage-collection.

Thanks for the confirmation. I'll increase the timeout in the next release.
Happy Easter!

from fkf-garbage-collection.