mbox series

[0/2] fetch: package identity of crate, gomod and npm URLs

Message ID 20261118092831.201831-1-stondo@gmail.com
Headers show
Series fetch: package identity of crate, gomod and npm URLs | expand

Message

Stefano Tondo Nov. 18, 2026, 9:28 a.m. UTC
Hi,

This series adds the fetcher API from the RFC "spdx30: name, version
and purl for dependency downloads" [1]. Richard asked for a fetcher
API that returns structured data with the purl, so that OE-Core does
not read fetcher internals.

Patch 1 adds FetchMethod.package_identity(). It returns the name, the
version and the purl of the downloaded package, or None. It uses only
the URL and the datastore, so it works before the fetch and without
network access.

Patch 2 implements it for the crate, gomod, gomodgit and npm fetchers,
with unit tests.

The RFC also covered the downloads that gitsm and npmsw add by
themselves. That part is not in this series. Richard and Alexander
prefer to list those downloads in SRC_URI. I will send that change
for npm in a separate series.

An OE-Core series that uses the API follows on the OE-Core list.

[1] https://lore.kernel.org/openembedded-core/20261105122145.543282-1-stondo@gmail.com/

Stefano Tondo (2):
  fetch: Add package_identity() to the fetchers
  fetch: Return the package identity of crate, gomod and npm URLs

 lib/bb/fetch/__init__.py | 27 +++++++++++++++++
 lib/bb/fetch/crate.py    |  7 +++++
 lib/bb/fetch/gomod.py    | 16 ++++++++++
 lib/bb/fetch/npm.py      |  7 +++++
 lib/bb/tests/fetch.py    | 63 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 120 insertions(+)

Comments

Paul Barker Nov. 19, 2026, 9:50 a.m. UTC | #1
On Wed Nov 18, 2026 at 9:28 AM GMT, Stefano Tondo wrote:
> Hi,
>
> This series adds the fetcher API from the RFC "spdx30: name, version
> and purl for dependency downloads" [1]. Richard asked for a fetcher
> API that returns structured data with the purl, so that OE-Core does
> not read fetcher internals.
>
> Patch 1 adds FetchMethod.package_identity(). It returns the name, the
> version and the purl of the downloaded package, or None. It uses only
> the URL and the datastore, so it works before the fetch and without
> network access.
>
> Patch 2 implements it for the crate, gomod, gomodgit and npm fetchers,
> with unit tests.
>
> The RFC also covered the downloads that gitsm and npmsw add by
> themselves. That part is not in this series. Richard and Alexander
> prefer to list those downloads in SRC_URI. I will send that change
> for npm in a separate series.
>
> An OE-Core series that uses the API follows on the OE-Core list.
>
> [1] https://lore.kernel.org/openembedded-core/20261105122145.543282-1-stondo@gmail.com/

Hi Stefano,

I can't speak for Richard but to me this looks like it's moving in the
right direction.

As the OE-Core changes rely on package_identity() returning a dict with
name, version and purl, I think we should instead make the data
structure more explicit and enforcible.

Perhaps a dataclass in bb.fetch, then package_identity() can return an
instance of this dataclass?

Something like:

    @dataclass
    class PackageIdentity:
        name: str
        version: str
        purl: str

Best regards,