| Message ID | 20260701074030.1090807-7-twoerner@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | tests: standalone test-suite framework plus the first unit test | expand |
On Wed, 2026-07-01 at 03:40 -0400, Trevor Woerner via lists.yoctoproject.org wrote: > The test suite needs a front door: a short document that says what the > suite is, how it is laid out, and how to run it. This commit adds > tests/docs/README.md to be that overview. > > The README covers: > > - the layout of tests/, file by file; > - that the suite is unit-only and needs no host tools or build > environment, so it runs from a plain checkout; > - how to install the test extras and invoke run-tests.sh, pointing at > run-tests.sh --help as the authoritative, always-current list of > options rather than duplicating them here where they would drift; > - that anything run-tests.sh does not recognise is passed through to > pytest; > - a pointer to linting.md for how ruff is applied; > - a small table of the other documents under tests/docs/. > > It deliberately does not enumerate every run-tests.sh option, since the > runner is expected to grow more of them; the table and the --help > output stay the source of truth. > > AI-Generated: codex/claude-opus 4.7 (xhigh) > Signed-off-by: Trevor Woerner <twoerner@gmail.com> > --- > changes in v3: > - document the standard pytest scratch-directory mechanisms > (TMPDIR and --basetemp) in the suite README, so there is no need > for a custom temporary-directory variable. > changes in v2: > - v1 submitted the entire test suite as a single commit; v2 breaks > the work into a reviewable series, and this patch is one step of it. > --- > tests/docs/README.md | 82 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 82 insertions(+) > create mode 100644 tests/docs/README.md > > diff --git a/tests/docs/README.md b/tests/docs/README.md > new file mode 100644 > index 000000000000..ba7bd5e192a3 > --- /dev/null > +++ b/tests/docs/README.md > @@ -0,0 +1,82 @@ > +# wic test suite > + > +A standalone test suite for the wic source tree. It runs from a plain > +checkout with nothing but `pytest` -- no bitbake, no OpenEmbedded > +build, and no target image -- so wic's logic can be exercised and kept > +stable as the code changes. > + > +## Contents > + > +- [Layout](#layout) > +- [Running](#running) > +- [Linting](#linting) > +- [Documentation](#documentation) A table of contents like this will easily fall out of sync with the rest of the file. Claude likes to generate them, but it doesn't always remember to update them. I don't think it's needed for such a small file. If you think it's worth keeping, use something like md-toc (https://github.com/frnmst/md-toc) and integrate it with pre-commit. > + > +## Layout > + > +``` > +tests/ > + conftest.py session banner describing the run > + run-tests.sh wrapper for running the suite > + unit/ unit tests that import wic modules directly > + docs/ this documentation > +``` > + > +The suite is unit-only: every test under `tests/unit/` imports a wic > +module in-process and asserts on its behaviour, so none of it needs > +host tools or a build environment. > + > +## Running > + > +Install wic with its test extras, then run the suite: > + > +```bash > +pip install -e ".[tests]" > +tests/run-tests.sh > +``` > + > +`run-tests.sh` works from anywhere in the checkout. It can also report > +branch coverage of the wic source; run `tests/run-tests.sh --help` for > +the current list of options. > + > +Anything `run-tests.sh` does not recognise is handed straight to > +pytest, so the whole pytest command line is available: > + > +```bash > +tests/run-tests.sh -k filemap -v # one area, verbose > +tests/run-tests.sh tests/unit # a single tier or file > +``` > + > +## Scratch files > + > +Tests that need scratch space use pytest's `tmp_path` fixture, so there > +is no wic-specific temporary-directory setting. Scratch directories are > +created under pytest's base temporary directory and cleaned up according > +to the retention policy in `pyproject.toml` (a passing test's directory > +is removed, a failing one is kept). > + > +By default pytest roots that base directory at the system temporary > +directory (usually `/tmp`). If that fills up, or you want the scratch > +files somewhere else, use the standard pytest mechanisms rather than a > +custom variable. Both are passed straight through by `run-tests.sh`: > + > +```bash > +TMPDIR=/path/to/scratch tests/run-tests.sh # honoured by pytest > +tests/run-tests.sh --basetemp=/path/to/scratch > +``` > + > +`--basetemp` puts everything under the given directory and clears it at > +the start of each run; `TMPDIR` keeps the last few sessions there. Claude loves to write! You can just say that pytest's tmp_path fixture is used and link to its docs (https://docs.pytest.org/en/stable/how-to/tmp_path.html#the-tmp-path-fixture). > + > +## Linting > + > +The test suite is linted with ruff and is held to a clean bar. See > +[linting.md](linting.md) for the lint modes and how the source tree is > +treated. > + > +## Documentation > + > +| File | Content | > +|------|---------| > +| [authoring.md](authoring.md) | How to add a unit test to the suite | > +| [linting.md](linting.md) | How ruff is used on the suite | Tables like this add very little value and fall out-of-sync quickly. Best regards,
diff --git a/tests/docs/README.md b/tests/docs/README.md new file mode 100644 index 000000000000..ba7bd5e192a3 --- /dev/null +++ b/tests/docs/README.md @@ -0,0 +1,82 @@ +# wic test suite + +A standalone test suite for the wic source tree. It runs from a plain +checkout with nothing but `pytest` -- no bitbake, no OpenEmbedded +build, and no target image -- so wic's logic can be exercised and kept +stable as the code changes. + +## Contents + +- [Layout](#layout) +- [Running](#running) +- [Linting](#linting) +- [Documentation](#documentation) + +## Layout + +``` +tests/ + conftest.py session banner describing the run + run-tests.sh wrapper for running the suite + unit/ unit tests that import wic modules directly + docs/ this documentation +``` + +The suite is unit-only: every test under `tests/unit/` imports a wic +module in-process and asserts on its behaviour, so none of it needs +host tools or a build environment. + +## Running + +Install wic with its test extras, then run the suite: + +```bash +pip install -e ".[tests]" +tests/run-tests.sh +``` + +`run-tests.sh` works from anywhere in the checkout. It can also report +branch coverage of the wic source; run `tests/run-tests.sh --help` for +the current list of options. + +Anything `run-tests.sh` does not recognise is handed straight to +pytest, so the whole pytest command line is available: + +```bash +tests/run-tests.sh -k filemap -v # one area, verbose +tests/run-tests.sh tests/unit # a single tier or file +``` + +## Scratch files + +Tests that need scratch space use pytest's `tmp_path` fixture, so there +is no wic-specific temporary-directory setting. Scratch directories are +created under pytest's base temporary directory and cleaned up according +to the retention policy in `pyproject.toml` (a passing test's directory +is removed, a failing one is kept). + +By default pytest roots that base directory at the system temporary +directory (usually `/tmp`). If that fills up, or you want the scratch +files somewhere else, use the standard pytest mechanisms rather than a +custom variable. Both are passed straight through by `run-tests.sh`: + +```bash +TMPDIR=/path/to/scratch tests/run-tests.sh # honoured by pytest +tests/run-tests.sh --basetemp=/path/to/scratch +``` + +`--basetemp` puts everything under the given directory and clears it at +the start of each run; `TMPDIR` keeps the last few sessions there. + +## Linting + +The test suite is linted with ruff and is held to a clean bar. See +[linting.md](linting.md) for the lint modes and how the source tree is +treated. + +## Documentation + +| File | Content | +|------|---------| +| [authoring.md](authoring.md) | How to add a unit test to the suite | +| [linting.md](linting.md) | How ruff is used on the suite |
The test suite needs a front door: a short document that says what the suite is, how it is laid out, and how to run it. This commit adds tests/docs/README.md to be that overview. The README covers: - the layout of tests/, file by file; - that the suite is unit-only and needs no host tools or build environment, so it runs from a plain checkout; - how to install the test extras and invoke run-tests.sh, pointing at run-tests.sh --help as the authoritative, always-current list of options rather than duplicating them here where they would drift; - that anything run-tests.sh does not recognise is passed through to pytest; - a pointer to linting.md for how ruff is applied; - a small table of the other documents under tests/docs/. It deliberately does not enumerate every run-tests.sh option, since the runner is expected to grow more of them; the table and the --help output stay the source of truth. AI-Generated: codex/claude-opus 4.7 (xhigh) Signed-off-by: Trevor Woerner <twoerner@gmail.com> --- changes in v3: - document the standard pytest scratch-directory mechanisms (TMPDIR and --basetemp) in the suite README, so there is no need for a custom temporary-directory variable. changes in v2: - v1 submitted the entire test suite as a single commit; v2 breaks the work into a reviewable series, and this patch is one step of it. --- tests/docs/README.md | 82 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 tests/docs/README.md