diff mbox series

[wic,v3,06/10] tests/docs: add the suite overview README

Message ID 20260701074030.1090807-7-twoerner@gmail.com
State New
Headers show
Series tests: standalone test-suite framework plus the first unit test | expand

Commit Message

Trevor Woerner July 1, 2026, 7:40 a.m. UTC
The test suite needs a front door: a short document that says what the
suite is, how it is laid out, and how to run it. This commit adds
tests/docs/README.md to be that overview.

The README covers:

  - the layout of tests/, file by file;
  - that the suite is unit-only and needs no host tools or build
    environment, so it runs from a plain checkout;
  - how to install the test extras and invoke run-tests.sh, pointing at
    run-tests.sh --help as the authoritative, always-current list of
    options rather than duplicating them here where they would drift;
  - that anything run-tests.sh does not recognise is passed through to
    pytest;
  - a pointer to linting.md for how ruff is applied;
  - a small table of the other documents under tests/docs/.

It deliberately does not enumerate every run-tests.sh option, since the
runner is expected to grow more of them; the table and the --help
output stay the source of truth.

AI-Generated: codex/claude-opus 4.7 (xhigh)
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
---
changes in v3:
- document the standard pytest scratch-directory mechanisms
  (TMPDIR and --basetemp) in the suite README, so there is no need
  for a custom temporary-directory variable.
changes in v2:
- v1 submitted the entire test suite as a single commit; v2 breaks
  the work into a reviewable series, and this patch is one step of it.
---
 tests/docs/README.md | 82 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)
 create mode 100644 tests/docs/README.md

Comments

Paul Barker July 6, 2026, 8:47 a.m. UTC | #1
On Wed, 2026-07-01 at 03:40 -0400, Trevor Woerner via
lists.yoctoproject.org wrote:
> The test suite needs a front door: a short document that says what the
> suite is, how it is laid out, and how to run it. This commit adds
> tests/docs/README.md to be that overview.
> 
> The README covers:
> 
>   - the layout of tests/, file by file;
>   - that the suite is unit-only and needs no host tools or build
>     environment, so it runs from a plain checkout;
>   - how to install the test extras and invoke run-tests.sh, pointing at
>     run-tests.sh --help as the authoritative, always-current list of
>     options rather than duplicating them here where they would drift;
>   - that anything run-tests.sh does not recognise is passed through to
>     pytest;
>   - a pointer to linting.md for how ruff is applied;
>   - a small table of the other documents under tests/docs/.
> 
> It deliberately does not enumerate every run-tests.sh option, since the
> runner is expected to grow more of them; the table and the --help
> output stay the source of truth.
> 
> AI-Generated: codex/claude-opus 4.7 (xhigh)
> Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> ---
> changes in v3:
> - document the standard pytest scratch-directory mechanisms
>   (TMPDIR and --basetemp) in the suite README, so there is no need
>   for a custom temporary-directory variable.
> changes in v2:
> - v1 submitted the entire test suite as a single commit; v2 breaks
>   the work into a reviewable series, and this patch is one step of it.
> ---
>  tests/docs/README.md | 82 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
>  create mode 100644 tests/docs/README.md
> 
> diff --git a/tests/docs/README.md b/tests/docs/README.md
> new file mode 100644
> index 000000000000..ba7bd5e192a3
> --- /dev/null
> +++ b/tests/docs/README.md
> @@ -0,0 +1,82 @@
> +# wic test suite
> +
> +A standalone test suite for the wic source tree. It runs from a plain
> +checkout with nothing but `pytest` -- no bitbake, no OpenEmbedded
> +build, and no target image -- so wic's logic can be exercised and kept
> +stable as the code changes.
> +
> +## Contents
> +
> +- [Layout](#layout)
> +- [Running](#running)
> +- [Linting](#linting)
> +- [Documentation](#documentation)

A table of contents like this will easily fall out of sync with the rest
of the file. Claude likes to generate them, but it doesn't always
remember to update them. I don't think it's needed for such a small
file. If you think it's worth keeping, use something like md-toc
(https://github.com/frnmst/md-toc) and integrate it with pre-commit.

> +
> +## Layout
> +
> +```
> +tests/
> +  conftest.py        session banner describing the run
> +  run-tests.sh       wrapper for running the suite
> +  unit/              unit tests that import wic modules directly
> +  docs/              this documentation
> +```
> +
> +The suite is unit-only: every test under `tests/unit/` imports a wic
> +module in-process and asserts on its behaviour, so none of it needs
> +host tools or a build environment.
> +
> +## Running
> +
> +Install wic with its test extras, then run the suite:
> +
> +```bash
> +pip install -e ".[tests]"
> +tests/run-tests.sh
> +```
> +
> +`run-tests.sh` works from anywhere in the checkout. It can also report
> +branch coverage of the wic source; run `tests/run-tests.sh --help` for
> +the current list of options.
> +
> +Anything `run-tests.sh` does not recognise is handed straight to
> +pytest, so the whole pytest command line is available:
> +
> +```bash
> +tests/run-tests.sh -k filemap -v          # one area, verbose
> +tests/run-tests.sh tests/unit             # a single tier or file
> +```
> +
> +## Scratch files
> +
> +Tests that need scratch space use pytest's `tmp_path` fixture, so there
> +is no wic-specific temporary-directory setting. Scratch directories are
> +created under pytest's base temporary directory and cleaned up according
> +to the retention policy in `pyproject.toml` (a passing test's directory
> +is removed, a failing one is kept).
> +
> +By default pytest roots that base directory at the system temporary
> +directory (usually `/tmp`). If that fills up, or you want the scratch
> +files somewhere else, use the standard pytest mechanisms rather than a
> +custom variable. Both are passed straight through by `run-tests.sh`:
> +
> +```bash
> +TMPDIR=/path/to/scratch tests/run-tests.sh   # honoured by pytest
> +tests/run-tests.sh --basetemp=/path/to/scratch
> +```
> +
> +`--basetemp` puts everything under the given directory and clears it at
> +the start of each run; `TMPDIR` keeps the last few sessions there.

Claude loves to write! You can just say that pytest's tmp_path fixture
is used and link to its docs
(https://docs.pytest.org/en/stable/how-to/tmp_path.html#the-tmp-path-fixture).

> +
> +## Linting
> +
> +The test suite is linted with ruff and is held to a clean bar. See
> +[linting.md](linting.md) for the lint modes and how the source tree is
> +treated.
> +
> +## Documentation
> +
> +| File | Content |
> +|------|---------|
> +| [authoring.md](authoring.md) | How to add a unit test to the suite |
> +| [linting.md](linting.md)     | How ruff is used on the suite |

Tables like this add very little value and fall out-of-sync quickly.

Best regards,
diff mbox series

Patch

diff --git a/tests/docs/README.md b/tests/docs/README.md
new file mode 100644
index 000000000000..ba7bd5e192a3
--- /dev/null
+++ b/tests/docs/README.md
@@ -0,0 +1,82 @@ 
+# wic test suite
+
+A standalone test suite for the wic source tree. It runs from a plain
+checkout with nothing but `pytest` -- no bitbake, no OpenEmbedded
+build, and no target image -- so wic's logic can be exercised and kept
+stable as the code changes.
+
+## Contents
+
+- [Layout](#layout)
+- [Running](#running)
+- [Linting](#linting)
+- [Documentation](#documentation)
+
+## Layout
+
+```
+tests/
+  conftest.py        session banner describing the run
+  run-tests.sh       wrapper for running the suite
+  unit/              unit tests that import wic modules directly
+  docs/              this documentation
+```
+
+The suite is unit-only: every test under `tests/unit/` imports a wic
+module in-process and asserts on its behaviour, so none of it needs
+host tools or a build environment.
+
+## Running
+
+Install wic with its test extras, then run the suite:
+
+```bash
+pip install -e ".[tests]"
+tests/run-tests.sh
+```
+
+`run-tests.sh` works from anywhere in the checkout. It can also report
+branch coverage of the wic source; run `tests/run-tests.sh --help` for
+the current list of options.
+
+Anything `run-tests.sh` does not recognise is handed straight to
+pytest, so the whole pytest command line is available:
+
+```bash
+tests/run-tests.sh -k filemap -v          # one area, verbose
+tests/run-tests.sh tests/unit             # a single tier or file
+```
+
+## Scratch files
+
+Tests that need scratch space use pytest's `tmp_path` fixture, so there
+is no wic-specific temporary-directory setting. Scratch directories are
+created under pytest's base temporary directory and cleaned up according
+to the retention policy in `pyproject.toml` (a passing test's directory
+is removed, a failing one is kept).
+
+By default pytest roots that base directory at the system temporary
+directory (usually `/tmp`). If that fills up, or you want the scratch
+files somewhere else, use the standard pytest mechanisms rather than a
+custom variable. Both are passed straight through by `run-tests.sh`:
+
+```bash
+TMPDIR=/path/to/scratch tests/run-tests.sh   # honoured by pytest
+tests/run-tests.sh --basetemp=/path/to/scratch
+```
+
+`--basetemp` puts everything under the given directory and clears it at
+the start of each run; `TMPDIR` keeps the last few sessions there.
+
+## Linting
+
+The test suite is linted with ruff and is held to a clean bar. See
+[linting.md](linting.md) for the lint modes and how the source tree is
+treated.
+
+## Documentation
+
+| File | Content |
+|------|---------|
+| [authoring.md](authoring.md) | How to add a unit test to the suite |
+| [linting.md](linting.md)     | How ruff is used on the suite |