| Message ID | 20250227090853.1632280-1-l.goehrs@pengutronix.de |
|---|---|
| Headers | show |
| Series | fetch2/github_release_artifact: fetcher for (private) release artifacts | expand |
On Thu, 2025-02-27 at 10:08 +0100, Leonard Göhrs via lists.yoctoproject.org wrote: > This fetcher enables downloading artifacts attached to GitHub releases > in private repositories. > > Authentication is provided using tokens via the `GH_TOKEN` variable > or a `token=` URI parameter. > > Changes from v1: > > - Provide examples on how to provide the `GH_TOKEN` more securely than > just hardcoding it in the recipe in the commit message. > E.g. via the `local.conf` or environment variables. > - Indent the code example in the documentation `.rst` using three > spaces instead of two. > - Document the `token=` URI parameter along with the `GH_TOKEN` > variable. > > Thanks to Alexander Kanavin and Antonin Godard for their review feedback > so far. I'm torn on this. I am leaning towards merging it however I realised there are a few issues. Firstly, it is failing during testing on Ubuntu 20.04 workers: https://autobuilder.yoctoproject.org/valkyrie/#/builders/54/builds/1129 Secondly, it adds support for putting the tokens in SRC_URI urls. I'm not in favour of this since it encourages people to write them in recipes where it is all too easy to unintentionally share them. I do not think we should enable this. Finally, the naming of the variable, GH_TOKEN is not in keeping with the rest of the codebase. It needs to be prefixed so something like BB_FETCH_GHRA_TOKEN which makes it clear it is bitbake, the fetcher and the GHRA fetcher. Cheers, Richard
I still remember the first CVE against YP - https://nvd.nist.gov/vuln/detail/CVE-2017-9731
It was reported by us because credentials from SRC_URI were being published in ipk packages and thus being freely readable in rootfs (with package-management image feature)...
But unfortunately, it's still impossible to handle all types of credentials in a way that bitbake does not see them.
I believe that github release artifact fetcher can be implemented by just modifying FETCHCMD_wget variable.
This is imho much safer (wrt leaking credentials in logs) than adding credentials to SRC_URI.
We're successfully using following with GitLab (not GitHub, very similar auth I think).
REGISTRY_CREDENTIALS = "${@d.getVar('GITLAB_TOKEN') if d.getVar('GITLAB_TOKEN') else (d.getVar('CI_JOB_TOKEN') if d.getVar('CI_JOB_TOKEN') else '')}"
REGISTRY_HEADER = "${@'JOB-TOKEN' if d.getVar('CI_JOB_TOKEN') else 'PRIVATE-TOKEN'}"
python() {
if d.getVar("REGISTRY_CREDENTIALS"):
d.setVar("FETCHCMD_wget", '/usr/bin/env wget -t 2 -T 100 --header "' + d.getVar("REGISTRY_HEADER") + ': ' + d.getVar("REGISTRY_CREDENTIALS") + '"')
}
do_fetch[vardepsexclude] += "CI_JOB_TOKEN GITLAB_TOKEN"
Maybe we could do gitlab-registry-credentials.bbclass out of this and similarly github-releases-credentials.bbclass by slight modification of it.
With little change in bitbake to define FETCHCMD_wget this could be even nicer with FETCHCMD_wget:append instead of anonymous python function.
Let me know if there would be interest, I could try to submit something.
Peter
> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-
> devel@lists.openembedded.org> On Behalf Of Richard Purdie via
> lists.openembedded.org
> Sent: Thursday, March 6, 2025 12:12
> To: lgo@pengutronix.de; bitbake-devel@lists.openembedded.org
> Cc: docs@lists.yoctoproject.org; yocto@pengutronix.de
> Subject: Re: [bitbake-devel] [docs] [PATCH v2 0/5] fetch2/github_release_artifact:
> fetcher for (private) release artifacts
>
> On Thu, 2025-02-27 at 10:08 +0100, Leonard Göhrs via lists.yoctoproject.org wrote:
> > This fetcher enables downloading artifacts attached to GitHub releases
> > in private repositories.
> >
> > Authentication is provided using tokens via the `GH_TOKEN` variable
> > or a `token=` URI parameter.
> >
> > Changes from v1:
> >
> > - Provide examples on how to provide the `GH_TOKEN` more securely than
> > just hardcoding it in the recipe in the commit message.
> > E.g. via the `local.conf` or environment variables.
> > - Indent the code example in the documentation `.rst` using three
> > spaces instead of two.
> > - Document the `token=` URI parameter along with the `GH_TOKEN`
> > variable.
> >
> > Thanks to Alexander Kanavin and Antonin Godard for their review feedback
> > so far.
>
> I'm torn on this. I am leaning towards merging it however I realised
> there are a few issues. Firstly, it is failing during testing on
> Ubuntu 20.04 workers:
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/54/builds/1129
>
> Secondly, it adds support for putting the tokens in SRC_URI urls. I'm
> not in favour of this since it encourages people to write them in
> recipes where it is all too easy to unintentionally share them. I do
> not think we should enable this.
>
> Finally, the naming of the variable, GH_TOKEN is not in keeping with
> the rest of the codebase. It needs to be prefixed so something like
> BB_FETCH_GHRA_TOKEN which makes it clear it is bitbake, the fetcher and
> the GHRA fetcher.
>
> Cheers,
>
> Richard
>
This fetcher enables downloading artifacts attached to GitHub releases in private repositories. Authentication is provided using tokens via the `GH_TOKEN` variable or a `token=` URI parameter. Changes from v1: - Provide examples on how to provide the `GH_TOKEN` more securely than just hardcoding it in the recipe in the commit message. E.g. via the `local.conf` or environment variables. - Indent the code example in the documentation `.rst` using three spaces instead of two. - Document the `token=` URI parameter along with the `GH_TOKEN` variable. Thanks to Alexander Kanavin and Antonin Godard for their review feedback so far.