feat(fs): Add filesystem validation test suite and related configurat… #20
Conversation
👋 Hello JakubAndrysek, we appreciate your contribution to this project! 📘 Please review the project's Contributions Guide for key guidelines on code, documentation, testing, and more. 🖊️ Please also make sure you have read and signed the Contributor License Agreement for this project. Click to see more instructions ...
Review and merge process you can expect ...
|
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive filesystem validation test suite for ESP32, testing three filesystem implementations (SPIFFS, FFat, and LittleFS) through a unified test interface.
Key Changes:
- Comprehensive test coverage for filesystem operations including basic I/O, directory operations, binary data handling, seek operations, and edge cases
- Unified test abstraction layer using
IFileSysteminterface andWrappedFStemplate wrapper to test all three filesystem types with the same test suite - Custom partition table configured to allocate space for all three filesystems plus NVS and coredump partitions
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/validation/fs/fs.ino | Main test implementation with 760 lines covering 16 test cases across filesystem operations |
| tests/validation/fs/test_fs.py | Python test runner using pytest-embedded framework to execute Unity test output |
| tests/validation/fs/partitions.csv | Partition table defining storage allocation for NVS, app, and all three filesystem types |
| tests/validation/fs/ci.yml | CI configuration disabling QEMU testing for this suite |
| tests/validation/fs/README.md | Documentation describing test suite, filesystem types, and known filesystem-specific behaviors |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| String expectedPath = String(fileBasePath) + String("/file") + String(count) + String(".txt"); | ||
| TEST_ASSERT_EQUAL_STRING_MESSAGE(expectedPath.c_str(), e.path(), "File path mismatch"); | ||
| String content = e.readString(); | ||
| String expectedContent = "data:" + String(count); | ||
| TEST_ASSERT_EQUAL_STRING_MESSAGE(expectedContent.c_str(), content.c_str(), "File content mismatch"); | ||
| count++; |
There was a problem hiding this comment.
This test assumes files will be listed in creation order (file0.txt, file1.txt, etc.), but directory listing order is not guaranteed by all filesystem implementations. Some filesystems may return files in a different order (e.g., alphabetical, hash-based, or arbitrary). Consider either sorting the results before validation, or checking that all expected files exist without assuming order.
tests/validation/fs/fs.ino
Outdated
| size_t initialUsed = gFS->usedBytes(); | ||
| size_t total = gFS->totalBytes(); | ||
| TEST_ASSERT_GREATER_THAN(0, (int)total); | ||
| TEST_ASSERT_LESS_OR_EQUAL_UINT32(total, (uint32_t)initialUsed); |
There was a problem hiding this comment.
Logic error: this assertion checks that total <= initialUsed, which would mean the used bytes is greater than or equal to total bytes. This should be TEST_ASSERT_LESS_OR_EQUAL_UINT32(initialUsed, (uint32_t)total) to verify that used bytes is less than or equal to total bytes.
| TEST_ASSERT_LESS_OR_EQUAL_UINT32(total, (uint32_t)initialUsed); | |
| TEST_ASSERT_LESS_OR_EQUAL_UINT32(initialUsed, (uint32_t)total); |
Test Results 83 files 83 suites 16m 24s ⏱️ For more details on these errors, see this check. Results for commit 0006d04. ♻️ This comment has been updated with latest results. |
JakubAndrysek
force-pushed
the
test/fs
branch
from
a86edba to
0006d04
Compare
JakubAndrysek
force-pushed
the
test/fs
branch
from
0006d04 to
b67360f
Compare
This pull request introduces a new filesystem validation test suite for ESP32, covering SPIFFS, FFat, and LittleFS implementations. It adds comprehensive documentation, configuration files, and a basic test runner to facilitate automated testing and highlight differences between the filesystems.
Test Suite and Documentation
README.mdexplaining the scope, test categories, and known filesystem-specific behaviours for SPIFFS, FFat, and LittleFS.Test and Configuration Files
partitions.csv) defining regions for each filesystem type to support the test suite.ci.yml) specifying that QEMU is not used for these tests.test_fs.py) that usespytest_embeddedandpytest_embedded_wokwito execute the Unity-based tests on the DUT (Device Under Test).Test scenarios
Tested in Wokwi:
esp32: SUCCESS
esp32c3: SUCCESS
esp32c6: SUCCESS
esp32h2: SUCCESS
esp32p4: SUCCESS
esp32s2: SUCCESS
esp32s3: SUCCESS
Also successfully tested on esp32 and esp32s3.