Code Monkey home page Code Monkey logo

esp32_spiffs_example's Introduction

Example of using SPIFFS with modified esp_idf spiffs VFS driver

esp-idf has support for the spiffs file system, but the directory support is not implemented.

This example uses the modified spiffs VFS driver which enables the directory support.


The original esp-idf spiffs driver is modified in a way that directories are enabled.

To enable the new spiffs driver, spiffs directory from esp-idf components directory is copied to the project's components directory.

Only the esp_spiffs.c file was modified to enable directory support.

The standard mkdir() and rmdir() functions are added to the spiffs VFS driver.

The modified version of mkspiffs is also provided which supports building spiffs file system image with directories.


Features

  • directory handling enabled
  • example of directory list functions
  • example of file copy functions

When using file related functions which has filename argument, prefix /spiffs/ has to be added to the file name.


How to build

Configure your esp32 build environment as for other esp-idf examples

Clone the repository

git clone https://github.com/loboris/ESP32_spiffs_example.git

Execute menuconfig and configure your Serial flash config and other settings. Included sdkconfig.defaults sets some defaults to be used.

Navigate to SPIffs Example Configuration and set SPIFFS options.

Select if you want to use wifi (recommended) to get the time from NTP server and set your WiFi SSID and password.

make menuconfig

Make and flash the example.

make all && make flash


Prepare SPIFFS image


Build mkspiffs

Before the image can be prepared, the mkspiffs executable must be created.
The sdkconfig.h is used in build process!
Before building, copy the sdkconfig.h from build/include directory to components/mkspiffs/include.
Change the working directory to components/mkspiffs/include.
Execute:

make clean
make

The simple build_mkspiffs script is provided which executes all the necessary steps automatically, just execute

./build_mkspiffs

Important: whenever you change spiffs configuration using menuconfig you must rebuild mkspiffs


It is not required to prepare the image, as the spiffs will be automatically formated on first use, but it might be convenient.


SFPIFFS image can be prepared on host and flashed to ESP32.

Copy the files to be included on spiffs into components/spiffs_image/image/ directory. Subdirectories can also be added.

Execute:

make makefs

to create spiffs image in build directory without flashing to ESP32

Execute:

make flashfs

to create spiffs image in build directory and flash it to ESP32



Example functions

  • get the time from NTP server and set the system time (if WiFi is enabled)
  • register spiffs as VFS file system; if the fs is not formated (1st start) it will be formated and mounted
  • perform some file system tests
    • write text to file
    • read the file back
    • make directory
    • copy file from root to the new directory
    • remove file from new directory
    • remove the new directory
    • list files in root directory and subdirectories
    • perform timing test
    • perform multitask test (if enabled in menuconfig)


Example output:

I (4221) [SPIFFS example]: ==== STARTING SPIFFS TEST ====

=======================
==== Write to file ====
=======================
  file: "/spiffs/test.txt"
  351 bytes written

========================
==== Read from file ====
========================
  file: "/spiffs/test.txt"
  351 bytes read [
ESP32 spiffs write to file, line 1
ESP32 spiffs write to file, line 2
ESP32 spiffs write to file, line 3
ESP32 spiffs write to file, line 4
ESP32 spiffs write to file, line 5
ESP32 spiffs write to file, line 6
ESP32 spiffs write to file, line 7
ESP32 spiffs write to file, line 8
ESP32 spiffs write to file, line 9
ESP32 spiffs write to file, line 10

]

=================================================
==== Read from file included in sfiffs image ====
=================================================
  file: "/spiffs/spiffs.info"
  405 bytes read [
INTRODUCTION

Spiffs is a file system intended for SPI NOR flash devices on embedded targets.
Spiffs is designed with following characteristics in mind:

  * Small (embedded) targets, sparse RAM without heap
  * Only big areas of data (blocks) can be erased
  * An erase will reset all bits in block to ones
  * Writing pulls one to zeroes
  * Zeroes can only be pulled to ones by erase
  * Wear leveling

]

=============================
==== List root directory ====
=============================

List of Directory [/spiffs/]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
d         -  07/06/2018 10:59  images
f       405  07/06/2018 10:59  spiffs.info
f       351  07/06/2018 10:59  test.txt
-----------------------------------
        756 in 2 file(s)
-----------------------------------
SPIFFS: free 758 KB of 934 KB
-----------------------------------

============================
==== Make new directory ====
============================
  dir: "/spiffs/newdir"
  Directory created
  Copy file from root to new directory...
  List the new directory

List of Directory [/spiffs/newdir]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
f       351  07/06/2018 10:59  test.txt.copy
-----------------------------------
        351 in 1 file(s)
-----------------------------------
SPIFFS: free 757 KB of 934 KB
-----------------------------------

  List root directory, the "newdir" should be listed

List of Directory [/spiffs/]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
d         -  07/06/2018 10:59  images
f       405  07/06/2018 10:59  spiffs.info
f       351  07/06/2018 10:59  test.txt
d         -  07/06/2018 10:59  newdir
-----------------------------------
        756 in 2 file(s)
-----------------------------------
SPIFFS: free 757 KB of 934 KB
-----------------------------------

  Try to remove non empty directory...
  Error removing directory (90) Directory not empty
  Removing file from new directory...
  Removing directory...
  List root directory, the "newdir" should be gone

List of Directory [/spiffs/]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
d         -  07/06/2018 10:59  images
f       405  07/06/2018 10:59  spiffs.info
f       351  07/06/2018 10:59  test.txt
-----------------------------------
        756 in 2 file(s)
-----------------------------------
SPIFFS: free 758 KB of 934 KB
-----------------------------------


================================================
==== List content of the directory "images" ====
==== which is included on spiffs image      ====
================================================

List of Directory [/spiffs/images]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
d         -  07/06/2018 10:59  test
f     39310  07/06/2018 10:59  test1.jpg
f     50538  07/06/2018 10:59  test2.jpg
f     47438  07/06/2018 10:59  test4.jpg
-----------------------------------
     137286 in 3 file(s)
-----------------------------------
SPIFFS: free 758 KB of 934 KB
-----------------------------------

==============================================================
==== Get the timings of spiffs operations
==== Operation:
====   Open file for writting, append 1 byte, close file
====   Open file for readinging, read 8 bytes, close file
==== 2 ms sleep between operations
==== 1000 operations will be executed
==============================================================
I (9028) Test: Started.
I (9498) Test: 100 reads/writes
I (9918) Test: 200 reads/writes
I (10346) Test: 300 reads/writes
I (10779) Test: 400 reads/writes
I (11209) Test: 500 reads/writes
I (11645) Test: 600 reads/writes
I (12079) Test: 700 reads/writes
I (12511) Test: 800 reads/writes
I (12945) Test: 900 reads/writes
I (13376) Test: 1000 reads/writes
W (13376) Test: Min write time: 2523 us
W (13376) Test: Max write time: 2987 us
W (13377) Test: Min read  time: 327 us
W (13382) Test: Max read  time: 462 us
W (13386) Test: Total run time: 4348 ms
I (13391) Test: Finished


List of Directory [/spiffs/]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
d         -  07/06/2018 10:59  images
f       405  07/06/2018 10:59  spiffs.info
f       351  07/06/2018 10:59  test.txt
f      1001  07/06/2018 10:59  testfil3.txt
-----------------------------------
       1757 in 3 file(s)
-----------------------------------
SPIFFS: free 756 KB of 934 KB
-----------------------------------


====================================================
STARTING MULTITASK TEST (3 tasks created)
Operation:
  Open file for writting, append 1 byte, close file
  Open file for readinging, read 8 bytes, close file
2 ms sleep between operations
Each task will perform 1000 operations
Expected run time 40~100 seconds
====================================================
I (14970) [TASK_1]: Started.
I (15170) [TASK_2]: Started.
I (15350) [TASK_1]: 100 reads/writes
I (15370) [TASK_3]: Started.
I (15840) [TASK_2]: 100 reads/writes
I (16220) [TASK_1]: 200 reads/writes
I (16340) [TASK_3]: 100 reads/writes
I (16722) [TASK_2]: 200 reads/writes
I (17149) [TASK_1]: 300 reads/writes
I (17295) [TASK_3]: 200 reads/writes
I (17644) [TASK_2]: 300 reads/writes
I (18055) [TASK_1]: 400 reads/writes
I (18295) [TASK_3]: 300 reads/writes
I (18573) [TASK_2]: 400 reads/writes
I (18999) [TASK_1]: 500 reads/writes
I (19308) [TASK_3]: 400 reads/writes
I (20023) [TASK_2]: 500 reads/writes
I (21294) [TASK_1]: 600 reads/writes
I (21900) [TASK_3]: 500 reads/writes
I (22153) [TASK_2]: 600 reads/writes
I (22577) [TASK_1]: 700 reads/writes
I (22829) [TASK_3]: 600 reads/writes
I (23086) [TASK_2]: 700 reads/writes
I (23541) [TASK_1]: 800 reads/writes
I (23731) [TASK_3]: 700 reads/writes
I (24002) [TASK_2]: 800 reads/writes
I (24417) [TASK_1]: 900 reads/writes
I (24699) [TASK_3]: 800 reads/writes
I (24932) [TASK_2]: 900 reads/writes
I (25308) [TASK_1]: 1000 reads/writes
I (25308) [TASK_1]: Finished
I (25669) [TASK_3]: 900 reads/writes
I (26083) [TASK_2]: 1000 reads/writes
I (26083) [TASK_2]: Finished
I (26875) [TASK_3]: 1000 reads/writes
W (26875) [TASK_3]: Min write time: 2553 us
W (26875) [TASK_3]: Max write time: 154609 us
W (26878) [TASK_3]: Min read  time: 337 us
W (26883) [TASK_3]: Max read  time: 144128 us
W (26888) [TASK_3]: Total run time: 57729 ms
I (26893) [TASK_3]: Finished


List of Directory [/spiffs/]
-----------------------------------
T  Size      Date/Time         Name
-----------------------------------
f      1001  07/06/2018 11:00  testfil2.txt
f       405  07/06/2018 10:59  spiffs.info
f      1001  07/06/2018 11:00  testfil3.txt
d         -  07/06/2018 10:59  images
f      1001  07/06/2018 11:00  testfil1.txt
f       351  07/06/2018 10:59  test.txt
-----------------------------------
       3759 in 5 file(s)
-----------------------------------
SPIFFS: free 754 KB of 934 KB
-----------------------------------


Total multitask test run time: 60 seconds

I (28870) [SPIFFS example]: ==== SPIFFS TEST FINISHED ====

esp32_spiffs_example's People

Contributors

loboris avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

esp32_spiffs_example's Issues

DIR dir; /*!< VFS DIR struct */

i could not resolve DIR, d_ino, d_name, d_type, meta, DT_DIR, DT_REG, SPIFFS_CHECK_PROGRESS . which header file contains these structs please?

WRITE_SPIFFS_BASE_ADDR is hardcoded to 0x110000

Why is the base address of the spiffs partition hardcoded?

I found it in components/spiffs_image/spiffs_param.mk, and I don't know if spiffs_image is a component directly from somewhere else or if you made this component, but it only flashes SPIFFS to a specific address, making other partition tables not function correctly.

editing that file directly seems to fix the issue

If enable psram ,can not vfs_spiffs_register

I found that if enable psram,the project will stop because the vfs_spiffs_register fail.

`rst:0x1 (POWERON_RESET),boot:0x3e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0010,len:4
load:0x3fff0014,len:5384
load:0x40078000,len:0
ho 12 tail 0 room 4
load:0x40078000,len:13112
entry 0x40078fd0
I (47) boot: ESP-IDF v3.0-dev-526-g29d48ec3-dirty 2nd stage bootloader
I (47) boot: compile time 10:41:34
I (64) boot: Enabling RNG early entropy source...
I (67) boot: SPI Speed : 40MHz
I (80) boot: SPI Mode : DIO
I (92) boot: SPI Flash Size : 4MB
I (104) boot: Partition Table:
I (116) boot: ## Label Usage Type ST Offset Length
I (138) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (162) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (185) boot: 2 factory factory app 00 00 00010000 00100000
I (208) boot: 3 storage Unknown data 01 82 00180000 00100000
I (232) boot: End of partition table
I (245) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x1491c ( 84252) map
I (362) esp_image: segment 1: paddr=0x00024944 vaddr=0x3ffb0000 size=0x030c4 ( 12484) load
I (378) esp_image: segment 2: paddr=0x00027a10 vaddr=0x40080000 size=0x00400 ( 1024) load
0x40080000: _iram_start at ??:?

I (384) esp_image: segment 3: paddr=0x00027e18 vaddr=0x40080400 size=0x081f8 ( 33272) load
I (453) esp_image: segment 4: paddr=0x00030018 vaddr=0x400d0018 size=0x5ecc8 (388296) map
0x400d0018: _stext at ??:?

I (867) esp_image: segment 5: paddr=0x0008ece8 vaddr=0x400885f8 size=0x0f50c ( 62732) load
0x400885f8: vTaskDelete at /Users/sky/esp/esp-idf/components/freertos/./tasks.c:3458

I (948) esp_image: segment 6: paddr=0x0009e1fc vaddr=0x400c0000 size=0x00000 ( 0) load
I (992) boot: Loaded app from partition at offset 0x10000
I (992) boot: Disabling RNG early entropy source...
I (998) spiram: SPI RAM mode: flash 40m sram 40m
I (1010) spiram: PSRAM initialized, cache is in low/high (2-core) mode.
I (1032) cpu_start: Pro cpu up.
I (1044) cpu_start: Starting app cpu, entry point is 0x400811f0
0x400811f0: call_start_cpu1 at /Users/sky/esp/esp-idf/components/esp32/./cpu_start.c:218

I (0) cpu_start: App cpu up.
I (3811) spiram: SPI SRAM memory test OK
I (3813) heap_init: Initializing. RAM available for dynamic allocation:
I (3813) heap_init: At 3FFAE2A0 len 00001D60 (7 KiB): DRAM
I (3832) heap_init: At 3FFB8160 len 00027EA0 (159 KiB): DRAM
I (3851) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (3871) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (3891) heap_init: At 40097B04 len 000084FC (33 KiB): IRAM
I (3910) cpu_start: Pro cpu start user code
I (3972) cpu_start: Starting scheduler on PRO CPU.
I (2916) cpu_start: Starting scheduler on APP CPU.
psraw_task!
I (2916) WM8978: init
I (2916) WM8978: reset succeed
I (2926) [SPIFFS]: Registering SPIFFS file system
I (2936) [SPIFFS]: Mounting SPIFFS files system
I (2936) [SPIFFS]: Start address: 0x180000; Size 1024 KB
I (2936) [SPIFFS]: Work buffer: 2048 B
I (2946) [SPIFFS]: FDS buffer: 384 B
I (2946) [SPIFFS]: Cache size: 2048 B
ets Jun 8 2016 00:22:57

rst:0x7 (TG0WDT_SYS_RESET),boot:0x3e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0010,len:4
load:0x3fff0014,len:5384
load:0x40078000,len:0
ho 12 tail 0 room 4
load:0x40078000,len:13112
entry 0x40078fd0
W (95) boot: PRO CPU has been reset by WDT.
W (95) boot: WDT reset info: PRO CPU PC=0x40084313
0x40084313: _xt_user_exc at xtensa_vectors.o:?

W (95) boot: WDT reset info: APP CPU PC=0x40089811
0x40089811: _xt_context_save at ??:?

I (114) boot: ESP-IDF v3.0-dev-526-g29d48ec3-dirty 2nd stage bootloader
I (136) boot: compile time 10:41:34
I (163) boot: Enabling RNG early entropy source...
I (166) boot: SPI Speed : 40MHz
I (179) boot: SPI Mode : DIO
I (192) boot: SPI Flash Size : 4MB
I (204) boot: Partition Table:
I (216) boot: ## Label Usage Type ST Offset Length
I (238) boot: 0 nvs WiFi data 01 02 00009000 00006000
I (262) boot: 1 phy_init RF data 01 01 0000f000 00001000
I (285) boot: 2 factory factory app 00 00 00010000 00100000
I (308) boot: 3 storage Unknown data 01 82 00180000 00100000
I (332) boot: End of partition table
I (345) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x1491c ( 84252) map
I (462) esp_image: segment 1: paddr=0x00024944 vaddr=0x3ffb0000 size=0x030c4 ( 12484) load
I (478) esp_image: segment 2: paddr=0x00027a10 vaddr=0x40080000 size=0x00400 ( 1024) load
0x40080000: _iram_start at ??:?
`

error when compiling

hi,

I found this error when compiling:

ESP32_spiffs_example-master/components/spiffs/./spiffs_vfs.c:836:9: error: unknown field 'fd_offset' specified in initializer
.fd_offset = 0,

BR,
Loris

How to create the image with Windows?

Hello,
sorry for the stupid question, but I did not figure out, how to create an image and how it is uploaded to the ESP32?
Maybe sombody can help to get into the right direction.
Commands like "make makefs" do not work under Windows.
For the ESP8266 there is a plugin for the arduino IDE, but that does not work for the ESP32.

Sincerely
Boris

Compiling Error

Hi ,

WARNING: Missing submodule ESP32_spiffs_example/components/spiffs/spiffs...
Attempting 'git submodule update --init ESP32_spiffs_example/components/spiffs/spiffs' in esp-idf root directory...
error: pathspec 'ESP32_spiffs_example/components/spiffs/spiffs' did not match any file(s) known to git.

I try to compile the example but it gives me the error. There is a module under spiffs/spiffs as well but it just detect it is not there, not sure why.

root is not handled properly

please try out (and check the returned values) - all the calls should succeed, IMHO:

int res;
DIR *dir;
res = stat("/spiffs");
res = stat("/spiffs/");
dir = opendir("/spiffs");
dir = opendir("/spiffs/");

One problem is in esp-idf that does not pass the call to VFS if there is no ending slash.
Another problem is in your folder implementation that doesn't seem to handle root folder ("/" or "/.") well.

The secret sauce in 'make makefs' and 'make flashfs'

Hi
Your project works great and according to your instructions.
However I can't see how to adapt it to my project.
Both makefs and the flashfs targets do not exist in the standard IDF Makefile
and I could not find where they are defined (but works great:-)
Could you please shed some light on how to adapt it to the standard make ?
Thanks

Error in Instructions to make mkspiffs

In the instructions for make a mkspiffs you tell:
"Change the working directory to components/mkspiffs/include."

The Corrent must be:
"Change the working directory to components/mkspiffs/."

Regards.

Why not pushing this to IDF github ?

Hi, this is nice work,
why not pushing this to idf github ? so it can be part of it - people using esp32-arduino could also get benefit of this great feature

.

My fault, sorry. Wish I could remove this issue.

rmdir is missing

rmdir() is missing. Can be linked to unlink, it seems to work but is it proper solution?

readdir generate stack overflow

Hello Loboris,

I open an issue regarding esp32 and spiffs since I think you could help me. I have opened a thread on esp32.com forum:

https://esp32.com/viewtopic.php?f=13&t=6486

where I explain the problem of using readdir on spiffs if there a "lot of" (i.e. 100 files) and also shared a very simple project to demonstrate the issue

https://github.com/davctv/test_spiffs_crash

I would like to know your opinion about that and if it is a bug of spiffs, vfs of esp32 or what else. This is quite impacting our project since we could you a lot of file (around 1000).

Thanks

compilation trouble

I configured with make menuconfig then executed make flash
it was compiling then I got

...
AR libmdns.a
CC micro-ecc/uECC.o
AR libmicro-ecc.a
AR libmkspiffs.a
usage: mkdir [-pv] [-m mode] directory ...
make[1]: *** [build] Error 64

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.