diff mbox series

[meta-python,13/14] python3-fsspec: Upgrade 2026.7.0 -> 2026.9.0

Message ID 20261020123226.4033600-13-leon.anavi@konsulko.com
State Under Review
Headers show
Series [meta-python,01/14] python3-bitarray: Upgrade 3.12.2 -> 3.12.3 | expand

Commit Message

Leon Anavi Oct. 20, 2026, 12:32 p.m. UTC
Switch SRC_URI to enable tests and upgrade to release 2026.9.0:

- ia:// filesystem for files in Internet Archive items: an
  HTTPFileSystem reading from archive.org/download with credentials
  from ia configure's ia.ini
- Allow instance-local memory stores with global_store=False while
  preserving normal instance caching and the default shared store
- Add an adaptive readahead cache
- Implement created and modified for SFTPFileSystem
- Allow filesystem implementations to assign protocol per instance
- Delegate sizes and cat_ranges from DirFileSystem to the wrapped
  filesystem
- Delegate invalidate_cache, ukey and checksum from DirFileSystem
  to the wrapped filesystem
- Support negative offsets in TarFileSystem.cat_file
- Mirror readinto from the PyArrow stream onto ArrowFile
- Allow simplecache files to be removed by age using their
  modification times
- HTTP: retry transient failures (408/425/429/5xx, dropped
  connections, timeouts, truncated range bodies, a 416 inside the
  file) of cat_file, HTTPFile block reads and the size lookup
  behind open, controlled by the new retries, retry_wait,
  retry_statuses and retry_exceptions options

Fixes:

- Compute HadoopFileSystem.fsid from its configured host and port
  instead of accessing attributes no longer exposed by PyArrow
- Normalize dot path components in TarFileSystem listings and
  lookups, while preserving original archive names for reading
- Make merge_offset_ranges O(n log n) and keep merged blocks within
  max_block, replacing the quadratic nested-range filter
- Fix transactions with compressed files
- Make MemoryFileSystem.rm raise FileNotFoundError for a path that
  does not exist, as rm_file and the other filesystems do, so
  deleting a missing key from a memory-backed mapper raises
  KeyError
- Create missing ZIP archives in append mode without truncating
  existing archives
- Close cached readers when exiting an open_files context
- Avoid mutating live BlockCache and BackgroundBlockCache instances
  when pickling
- End the transaction even when a commit or discard raises, so the
  filesystem is not left in transaction mode and deferred temporary
  files are cleaned up
- Honor a caller-owned output file in get_file
- Preserve existing files when checking FSMap write access
- Normalize ZIP paths in pipe_file and find
- Reject get destinations that resolve above the target
- Pass maxdepth through in AsyncFileSystem._rm
- Do not let isfile swallow BaseException
- Keep reference filesystem get destinations inside the target
- Return relative names from DirFileSystem.glob and find
- Raise FileNotFoundError for missing ZIP and TAR members
- Stop collapsing three-character drive-relative paths to the drive
  root
- Use / as the only glob separator on every platform
- Distinguish omitted and None defaults in FSMap.pop
- Forward an Arrow file's flush and closed state to its stream
- Keep reference filesystem cat_file offsets inside the referenced
  part
- Raise FileNotFoundError from archive ls for a missing path
- Download uncached files in simplecache.cat_ranges
- Fix tail on simplecache and filecache
- Get whole-file reference sizes asynchronously in
  ReferenceFileSystem._info
- Report the target size for links in TarFileSystem
- Keep the async generator returned by AsyncFileSystemWrapper._walk
- Join the default output filename with / in _expand_paths
- Make names relative in DirFileSystem.walk(detail=True)
- Ignore a leading slash in TarFileSystem paths
- Fix cat_ranges on whole-file caches
- Normalize paths in AsyncFileSystemWrapper like the wrapped
  filesystem
- The SFTP host key policy is configurable with the new
  host_key_policy argument of SFTPFileSystem, which accepts
  "auto_add" (default, unchanged behaviour), "warning" or "reject",
  or any paramiko.MissingHostKeyPolicy

Other:

- Use trusted publishing for releases and trigger the release
  workflow on tags
- Remove the stale async wrapper documentation warning
- Add Zarr to the test_downstream extra so the Xarray/Zarr test
  runs again
- Clarify that rm handles directories
- Fix grammar in features.rst
- Patch HAS_CPYTHON_API where _fast_slice reads it in tests
- Percent-decode the username and password parsed from URLs in
  infer_storage_options so that backends (ftp, sftp, smb, ...)
  receive the real credentials rather than their URL-encoded form;
  a password whose bare % produces bytes that are not valid UTF-8
  (for example pass%ab) is left untouched so that pre-existing URLs
  with unescaped % continue to work

Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>
---
 .../python/python3-fsspec_2026.7.0.bb         | 24 ------------
 .../python/python3-fsspec_2026.9.0.bb         | 37 +++++++++++++++++++
 2 files changed, 37 insertions(+), 24 deletions(-)
 delete mode 100644 meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb
 create mode 100644 meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb
diff mbox series

Patch

diff --git a/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb b/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb
deleted file mode 100644
index d34b67c428..0000000000
--- a/meta-python/recipes-devtools/python/python3-fsspec_2026.7.0.bb
+++ /dev/null
@@ -1,24 +0,0 @@ 
-SUMMARY = "A specification that python filesystems should adhere to."
-HOMEPAGE = "https://github.com/fsspec/filesystem_spec"
-LICENSE = "BSD-3-Clause"
-LIC_FILES_CHKSUM = "file://LICENSE;md5=b38a11bf4dcdfc66307f8515ce1fbaa6"
-
-SRC_URI[sha256sum] = "c803c40f4cf860b49dea58ee3e1c33cb9c790520e233537e1340049f89b82a88"
-
-inherit pypi python_hatchling ptest
-
-
-DEPENDS = "python3-hatch-vcs-native python3-hatchling-native"
-
-RDEPENDS:${PN}-ptest += "\
-	python3-pytest \
-	python3-pytest-mock \
-	python3-pytest-asyncio \
-	python3-pytest-cov \
-	python3-pytest-benchmark \
-	python3-aiohttp \
-	python3-numpy \
-	python3-requests \
-"
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb b/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb
new file mode 100644
index 0000000000..43d405f017
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-fsspec_2026.9.0.bb
@@ -0,0 +1,37 @@ 
+SUMMARY = "A specification that python filesystems should adhere to."
+HOMEPAGE = "https://github.com/fsspec/filesystem_spec"
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=b38a11bf4dcdfc66307f8515ce1fbaa6"
+
+SRC_URI = "git://github.com/fsspec/filesystem_spec;protocol=https;branch=master;tag=${PV}"
+SRCREV = "0f76baaae4a227b085b18e8469e6cc1792a95ade"
+
+inherit python_hatchling ptest-python-pytest
+
+DEPENDS = "python3-hatch-vcs-native python3-hatchling-native"
+
+RDEPENDS:${PN}-ptest += "\
+    python3-pytest \
+    python3-pytest-mock \
+    python3-pytest-asyncio \
+    python3-pytest-cov \
+    python3-pytest-benchmark \
+    python3-aiohttp \
+    python3-numpy \
+    python3-requests \
+    python3-pandas \
+    python3-paramiko \
+"
+
+do_install_ptest:append() {
+    install -d ${D}${PTEST_PATH}/tests ${D}${PTEST_PATH}/implementations
+    cp -rf ${S}/fsspec/tests/* ${D}${PTEST_PATH}/tests/
+    cp -rf ${S}/fsspec/implementations/tests ${D}${PTEST_PATH}/implementations/
+    touch ${D}${PTEST_PATH}/implementations/__init__.py
+    # Tests import helpers via fsspec.tests / fsspec.implementations.tests,
+    # which aren't in the installed package
+    grep -rlE 'fsspec\.(implementations\.)?tests' ${D}${PTEST_PATH} \
+        | xargs -r sed -i -E 's/fsspec\.(implementations\.)?tests\b/\1tests/g'
+}
+
+BBCLASSEXTEND = "native nativesdk"