From 60a82f1f581738c20d05dff7e20130f720772135 Mon Sep 17 00:00:00 2001 From: Rod Montrose Date: Sat, 6 Sep 2025 18:38:59 -0600 Subject: [PATCH] Improved FAT32 support for ESP32 and allows Files to be used with SDCard that are mounted externally. - Replaced esp_vfs_fat_spiflash_mount with esp_vfs_fat_spiflash_mount_rw_wl and esp_vfs_fat_spiflash_unmount with esp_vfs_fat_spiflash_unmount_rw_wl to enabled RW access to file system and support wear leveling of the SPI flash. - Removed hard coded SECTOR_SIZE used in Fat32 size calculations and use the sector size from the Fat structure. This allows to correctly handle various sector sizes (512, 1024, 2048 or 4096) instead of just the hardcoded 512 and 4096 and report correct capacity and amount used. - For use with SD cards, these are mounted separately from the current File code. (this could be added to modFile.c if needed.) Added MODDEF_FILE_SDCARD if this is set to a 1, then the File internal mounting/unmounting is not used). - Throws JavaScript xsUnknownError exception if there is an error when mounting the media. Before it went to modLog that didn't appear to do anything. - If SPIFFS is used, the directory create and delete functions are bypassed in case the user software calls it. It will still create files with the directory name and path seporator in the root directory. --- documentation/files/files.md | 36 +++++--------- modules/files/file/esp32/modFile.c | 76 +++++++++++++++--------------- 2 files changed, 49 insertions(+), 63 deletions(-) diff --git a/documentation/files/files.md b/documentation/files/files.md index 138aaecd17..bfef85537f 100644 --- a/documentation/files/files.md +++ b/documentation/files/files.md @@ -366,19 +366,27 @@ On ESP32, the SPIFFS file system is mounted at a specified path and all files/di The `File` class implements an optional FAT32 file system for the ESP32. Unlike SPIFFS, FAT32 file systems are not flat: they have directory structures and long filenames (up to 255 characters). -If the FAT32 file system has not been initialized then it is formatted when first used. As with SPIFFS, the `File` class only instantiates the FAT32 file system when necessary and it is automatically closed when not in use. +If the FAT32 file system has not been initialized, then it is formatted when first used. As with SPIFFS, the `File` class only instantiates the FAT32 file system when necessary, and it is automatically closed when not in use. -To enable the FAT32 file system, set the `fat32` [manifest define](https://github.com/Moddable-OpenSource/moddable/blob/public/documentation/tools/defines.md) to `1`: +There are several settings for the FAT32 file system in the [manifest define](https://github.com/Moddable-OpenSource/moddable/blob/public/documentation/tools/defines.md) . ```JSON "defines": { "file":{ - "fat32": 1 + "fat32": 1, + "root": "\"/sdcard\"", + "partition": "#userdata", + "sdcard": 0 } } ``` -The storage partition used by the default Moddable SDK build for ESP32 does not reserve a partition for FAT32. Therefore, it is necessary to use a different partition file in projects that use FAT32. To do that, set the `PARTITIONS_FILE` variable in the `build` section of the project manifest: +- You need to set `fat32` to to enable the FAT32 settings. Set this to 0 or remove it for SPIFFS file system. +- By default, the FAT32 file system is mounted at `/mod`. To change the default root, set the `root` define in the manifest. Do not include a trailing `/` in the root name. +- The default name for the FAT32 partition is `storage`. To use a different name, set the `partition` defined in the manifest. +- If the file system is not on the internal ESP32 Flash, it will need to be managed externally to the File's internal software. For example, this is required for SD card support, Set the `sdcard` define to 1 to bypass using the File class's software to mount, unmount and format the file system. You will need to manage the file system externally from the `File` class. + +The storage partition used by the default Moddable SDK build for ESP32 does not reserve a partition for FAT32. Therefore, it is necessary to use a different partition file in projects that use FAT32. To do that, set the `PARTITIONS_FILE` variable in the `build` section of the project manifest to point at an ESP32 partition file: ```JSON "build": { @@ -397,26 +405,6 @@ xs, 0x40, 1, 0x310000, 0x040000, settings, data, 1, 0x350000, 0x010000, storage, data, fat, 0x360000, 0x090000, ``` -The default name for the FAT32 partition is `storage`. To use a different name, set the `partition` define in the manifest: - -```JSON -"defines": { - "file":{ - "partition": "#userdata" - } -} -``` - -By default, the FAT32 file system is mounted at `/mod`. To change the default root, set the `root` define in the manifest: - -```JSON -"defines": { - "file":{ - "root": "/myroot/" - } -} -``` - #### littlefs The [littlefs](https://github.com/littlefs-project/littlefs) file system is "a little fail-safe filesystem designed for microcontrollers." It provides a high reliability, hierarchical file system in a small code footprint (about 60 KB) using minimal memory (well under 1 KB) with a high degree of configurability. littlefs also supports long file names (up to 255 characters) and formats a new partition very quickly. diff --git a/modules/files/file/esp32/modFile.c b/modules/files/file/esp32/modFile.c index 9b3300eeed..b46317fd46 100644 --- a/modules/files/file/esp32/modFile.c +++ b/modules/files/file/esp32/modFile.c @@ -47,11 +47,6 @@ #else #define MAX_FILENAME_LENGTH 12 #endif - #ifdef CONFIG_WL_SECTOR_SIZE - #define SECTOR_SIZE CONFIG_WL_SECTOR_SIZE - #else - #define SECTOR_SIZE 512 - #endif #else #include "esp_spiffs.h" #include "spiffs_config.h" @@ -72,14 +67,14 @@ typedef struct { char path[1]; } iteratorRecord, *iter; -static int startFS(void); +static int startFS(xsMachine *the); static void stopFS(void); static FILE *getFile(xsMachine *the) { FILE *result = xsmcGetHostData(xsThis); if (!result) - xsUnknownError("closed"); + xsUnknownError("closed file"); return result; } @@ -100,7 +95,7 @@ void xs_File(xsMachine *the) uint8_t write = (argc < 2) ? 0 : xsmcToBoolean(xsArg(1)); char *path = xsmcToString(xsArg(0)); - startFS(); + startFS(the); file = fopen(path, write ? "rb+" : "rb"); if (NULL == file) { @@ -197,7 +192,7 @@ void xs_file_write(xsMachine *the) } result = fflush(file); if (0 != result) - xsUnknownError("file flush failed"); + xsUnknownError("file flush failed on write"); } void xs_file_close(xsMachine *the) @@ -235,7 +230,7 @@ void xs_file_delete(xsMachine *the) int32_t result; char *path = xsmcToString(xsArg(0)); - startFS(); + startFS(the); result = unlink(path); @@ -250,7 +245,7 @@ void xs_file_exists(xsMachine *the) int32_t result; char *path = xsmcToString(xsArg(0)); - startFS(); + startFS(the); result = stat(path, &buf); @@ -269,21 +264,21 @@ void xs_file_rename(xsMachine *the) path = xsmcToString(xsArg(0)); if ('/' != toPath[0]) { if (c_strchr(toPath + 1, '/')) - xsUnknownError("invalid to"); + xsUnknownError("Invalid to on rename"); char *slash = c_strrchr(path, '/'); if (!slash) - xsUnknownError("invalid from"); + xsUnknownError("Invalid from on rename"); size_t pathLength = slash - path + 1; if (pathLength >= (c_strlen(path) + sizeof(toPath))) - xsUnknownError("path too long"); + xsUnknownError("Path too long on rename"); c_strcpy(toPath, path); xsmcToStringBuffer(xsArg(1), toPath + pathLength, sizeof(toPath) - pathLength); } - startFS(); + startFS(the); result = rename(path, toPath); @@ -326,7 +321,7 @@ void xs_File_Iterator(xsMachine *the) #endif d->rootPathLen = i; - startFS(); + startFS(the); if (NULL == (d->dir = opendir(d->path))) { c_free(d); @@ -367,7 +362,7 @@ void xs_file_iterator_next(xsMachine *the) if (DT_REG == de->d_type) { c_strcpy(d->path + d->rootPathLen, de->d_name); if (-1 == stat(d->path, &buf)) - xsUnknownError("stat error"); + xsUnknownError("stat error on iterator next"); xsmcSetInteger(xsVar(0), buf.st_size); xsmcSet(xsResult, xsID_length, xsVar(0)); } @@ -385,7 +380,7 @@ void xs_file_system_info(xsMachine *the) { xsResult = xsmcNewObject(); - startFS(); + startFS(the); size_t total = 0, used = 0; esp_err_t ret; @@ -394,10 +389,10 @@ void xs_file_system_info(xsMachine *the) FATFS *fs; DWORD fre_clust, fre_sect, tot_sect; ret = f_getfree("0:", &fre_clust, &fs); - tot_sect = (fs->n_fatent - 2) * fs->csize; - fre_sect = fre_clust * fs->csize; - total = tot_sect * SECTOR_SIZE; - used = (tot_sect - fre_sect) * SECTOR_SIZE; + tot_sect = (fs->n_fatent - 2) * fs->csize; // First two FAT clusters are reserved + fre_sect = fre_clust * fs->csize; // Free clusters + total = tot_sect * fs->ssize; // Sector size (512, 1024, 2048 or 4096) + used = (tot_sect - fre_sect) * fs->ssize; #else ret = esp_spiffs_info(NULL, &total, &used); #endif @@ -405,12 +400,12 @@ void xs_file_system_info(xsMachine *the) stopFS(); if (ret != ESP_OK) - xsUnknownError("system info failed"); + xsUnknownError("System info failed"); xsmcVars(1); - xsmcSetInteger(xsVar(0), total); + xsmcSetInteger(xsVar(0), (uint64_t)total); xsmcSet(xsResult, xsID_total, xsVar(0)); - xsmcSetInteger(xsVar(0), used); + xsmcSetInteger(xsVar(0), (uint64_t)used); xsmcSet(xsResult, xsID_used, xsVar(0)); } @@ -420,12 +415,13 @@ static uint16_t gUseCount; static wl_handle_t wl_handle; #endif -int startFS(void) +int startFS(xsMachine *the) { esp_err_t ret; if (0 != gUseCount++) return 0; - +// If this is an SD card filesystem, then don't do the internal Flash partition filesystem +#if MODDEF_FILE_SDCARD != 1 #if MODDEF_FILE_FAT32 esp_vfs_fat_mount_config_t conf = { .format_if_mount_failed = true, @@ -433,7 +429,7 @@ int startFS(void) .allocation_unit_size = 512 }; - ret = esp_vfs_fat_spiflash_mount(MODDEF_FILE_ROOT, MODDEF_FILE_PARTITION, &conf, &wl_handle); + ret = esp_vfs_fat_spiflash_mount_rw_wl(MODDEF_FILE_ROOT, MODDEF_FILE_PARTITION, &conf, &wl_handle); #else //@@ these behaviors could be controlled by DEFINE properties esp_vfs_spiffs_conf_t conf = { @@ -451,20 +447,21 @@ int startFS(void) if (ret != ESP_OK) { if (ret == ESP_FAIL) { - modLog("Failed to mount or format filesystem"); + xsUnknownError("Failed to mount or format filesystem"); } else if (ret == ESP_ERR_NO_MEM) { - modLog("Failed to allocate memory for filesystem"); + xsUnknownError("Failed to allocate memory for filesystem"); } else if (ret == ESP_ERR_NOT_FOUND) { - modLog("Failed to find partition"); + xsUnknownError("Failed to find partition"); } else if (ret == ESP_ERR_INVALID_STATE) { - modLog("Failed because filesystem was already registered"); + xsUnknownError("Failed because filesystem was already registered"); } else { - modLog("Failed to initialize filesystem"); + xsUnknownError("Failed to initialize filesystem"); } gUseCount--; } +#endif // if file sdcard is set to 1, the don't do the internal Flash partition filesystem return 0; } @@ -472,13 +469,14 @@ void stopFS(void) { if (0 != --gUseCount) return; - +#if MODDEF_FILE_SDCARD != 1 #if MODDEF_FILE_FAT32 - esp_vfs_fat_spiflash_unmount(MODDEF_FILE_ROOT, wl_handle); + esp_vfs_fat_spiflash_unmount_rw_wl(MODDEF_FILE_ROOT, wl_handle); wl_handle = 0; #else esp_vfs_spiffs_unregister(NULL); #endif +#endif // if file sdcard is set to 1, the don't do the internal Flash partition filesystem } @@ -486,10 +484,10 @@ void xs_directory_create(xsMachine *the) { char *path = xsmcToString(xsArg(0)); - startFS(); + startFS(the); int result = mkdir(path, 0775); if (result && (EEXIST != errno)) - xsUnknownError("failed"); + xsUnknownError("Directory create failed"); stopFS(); } @@ -497,9 +495,9 @@ void xs_directory_delete(xsMachine *the) { char *path = xsmcToString(xsArg(0)); - startFS(); + startFS(the); int result = rmdir(path); if (result && (ENOENT != errno)) - xsUnknownError("failed"); + xsUnknownError("Directory delete failed"); stopFS(); }