Very unstable system after migrating to Arduino-ESP32 3.0.01 about arduino-esp32 HOT 17 OPEN

If you want to remove some specific components of the libs you can use Arduino as a component or recompile the libs using the Lib Builder

from arduino-esp32.

If you want to remove some specific components of the libs you can use Arduino as a component or recompile the libs using the Lib Builder

Arduino is used as an ESP-IDF component. Arduino libraries are alright, what I suspect is RainMaker and Skainet which are baked into ESP-IDF, I am not sure how to remove them from the project.

from arduino-esp32.

@xwtk - a few questions:

1- What is the partition table used within Arduino 2.0.x and 3.0.x? NVS has its own partition and maybe there is no enough space after moving to IDF 5.1.
2- What are all the network services, resources and peripherals that your project uses?
3- Arduino as Component allow the user to change the sdkconfig of the project to better suit its needs. How differente is it from the one used in Arduino 2.0.x and 3.0.x?

from arduino-esp32.

@xwtk - a few questions:

1- What is the partition table used within Arduino 2.0.x and 3.0.x? NVS has its own partition and maybe there is no enough space after moving to IDF 5.1. 2- What are all the network services, resources and peripherals that your project uses? 3- Arduino as Component allow the user to change the sdkconfig of the project to better suit its needs. How differente is it from the one used in Arduino 2.0.x and 3.0.x?

Same partition table, however it was flashed while on 2.0.x, afterwards upgrade to 3.0.x has been done only via OTA. No changes to the partition table or the project components except those added by ESP-IDF 5.1.4. Regarding network, WiFi, Asyncwebserver (HTTP), mDNS (broadcast).

from arduino-esp32.

Same partition table, however it was flashed while on 2.0.x, afterwards upgrade to 3.0.x has been done only via OTA.

As far as I remember, OTA only update the firmware. Therefore the partition information and NVS information are kept the same.

if I mount the NVS once and unmount, whenever I try to do any change I get nvs_init failed with error "OTHER" and subsequently nvs_open failed with error "NOT_INITALIZED".

Nothing was changed in Arduino regarding NVS. Given that the application doesn't use Preferences, this must be something related to IDF change from 4.x to 5.1.

I suggest checking IDF documentation.

While webserver and mDNS is running, a lot of the times ssl fails to allocate memory (all requests are done in HTTPS with some exceptions), note that this was NOT the case with Arduino-ESP32 2.x.x. As well as seems like heap memory got a hit, while previously I had 90-100K free, I now have 60-70K with systems running, on paper that shouldn't be a problem but seems like it is

You can disable PPP in IDF Menuconfig of your Arduino as IDF component project. That shall increase available HEAP.

from arduino-esp32.

Same partition table, however it was flashed while on 2.0.x, afterwards upgrade to 3.0.x has been done only via OTA.

As far as I remember, OTA only update the firmware. Therefore the partition information and NVS information are kept the same.

if I mount the NVS once and unmount, whenever I try to do any change I get nvs_init failed with error "OTHER" and subsequently nvs_open failed with error "NOT_INITALIZED".

Nothing was changed in Arduino regarding NVS. Given that the application doesn't use Preferences, this must be something related to IDF change from 4.x to 5.1.

I suggest checking IDF documentation.

While webserver and mDNS is running, a lot of the times ssl fails to allocate memory (all requests are done in HTTPS with some exceptions), note that this was NOT the case with Arduino-ESP32 2.x.x. As well as seems like heap memory got a hit, while previously I had 90-100K free, I now have 60-70K with systems running, on paper that shouldn't be a problem but seems like it is

You can disable PPP in IDF Menuconfig of your Arduino as IDF component project. That shall increase available HEAP.

Correct, OTA doesn't change anything but the app firmware. NVS interaction is done via Preferences library which is modified to accommodate my needs, the modifications are explained in the initial post, I suspect the problem here comes from heap too, not sure though. Can you guide me where can I disable PPP in menuconfig? Thank you.

from arduino-esp32.

not sure if this is same / similar problem to mine with boards manager 3.0.1
i am using painlessmesh, with Arduino 2.3.2, after upgrading to boards manager (ESP32) to 3.0.1 none of my nodes would compile.
downgraded to boards manager 2.0.17 and everything works again, contacted Rui Santos who says this is the best way to go for now but in his opinion painlessmesh is one of a few libs which are affected, does anyone know if painlessmesh will be updates to work with boards manager 3.0.1 please
cheers
Dave

from arduino-esp32.

not sure if this is same / similar problem to mine with boards manager 3.0.1 i am using painlessmesh, with Arduino 2.3.2, after upgrading to boards manager (ESP32) to 3.0.1 none of my nodes would compile. downgraded to boards manager 2.0.17 and everything works again, contacted Rui Santos who says this is the best way to go for now but in his opinion painlessmesh is one of a few libs which are affected, does anyone know if painlessmesh will be updates to work with boards manager 3.0.1 please cheers Dave

Hey Dave, I am not using Arduino, rather native ESP-IDF with Arduino-ESP32 core as a component.

from arduino-esp32.

ok, thanks but it does show that 3.0.1 is causing problems on different platforms, commonality is saying something methinks

from arduino-esp32.

Yes definitely. I also ran the same project with almost the same configuration (only changes done by IDF itself) on both ESP-IDF 4.4.6 (Arduino-ESP32 2.x.x) and ESP-IDF 5.1.4 (Arduino-ESP32 3.0.1), I have monitored and saw that 20-35kB of heap memory is lost! That is a hell of memory to be used while no new feature has been enabled. This is definitely caused by the garbage ESP-IDF Component Registry, the compiler adds literally as many components as it can, and nowhere is it written how to exclude them! Additionally PPP has always been disabled in the project, I double-checked to be sure. This is absolutely unacceptable, how did 3.0.0 and 3.0.1 get even marked as stable releases??

from arduino-esp32.

Can you guide me where can I disable PPP in menuconfig? Thank you.

@xwtk - did you manage to disable it in your project?

idf.py menuconfig
Go in the Menu->Component config->LWIP
seach for Enable PPP suport, press space bar on top of this option to uncheck it.
press ESC, press S to save sdkconfig file (rename it or define a new folder for it), press ENTER, ESC, ESC and then build your project using the new sdkconfig modified file.

from arduino-esp32.

This is absolutely unacceptable, how did 3.0.0 and 3.0.1 get even marked as stable releases??

It is stable. As a new Version, it has breaking changes.

from arduino-esp32.

Can you guide me where can I disable PPP in menuconfig? Thank you.

@xwtk - did you manage to disable it in your project?

idf.py menuconfig Go in the Menu->Component config->LWIP seach for Enable PPP suport, press space bar on top of this option to uncheck it. press ESC, press S to save sdkconfig file (rename it or define a new folder for it), press ENTER, ESC, ESC and then build your project using the new sdkconfig modified file.

As mentioned before, I checked and it was never enabled in the first place.

from arduino-esp32.

This is absolutely unacceptable, how did 3.0.0 and 3.0.1 get even marked as stable releases??

It is stable. As a new Version, it has breaking changes.

Sure of course. But this isn't a breaking change, eating 20-35kB heap for nothing isn't normal. It is definitely a bunch of new components from the new ESP-IDF 5, but that should be something that can be removed in the first place.

from arduino-esp32.

@xwtk everything that comes with the component is listed here: https://github.com/espressif/arduino-esp32/blob/master/idf_component.yml#L47-L94

You can comment out the components that you do not need. You also need to understand that not everything you see comes from us, IDF5 has many changes that increase ram/flash usage, but using it as component you could turn some off, like WPA3 and others.

from arduino-esp32.

@xwtk everything that comes with the component is listed here: https://github.com/espressif/arduino-esp32/blob/master/idf_component.yml#L47-L94

You can comment out the components that you do not need. You also need to understand that not everything you see comes from us, IDF5 has many changes that increase ram/flash usage, but using it as component you could turn some off, like WPA3 and others.

Alright perfect, let me comment out unused modules and see if it changes anything.

from arduino-esp32.

It increased heap a bit, but issues still remain.

from arduino-esp32.