| Message ID | 20211206164119.4076119-1-alex@linutronix.de |
|---|---|
| State | Accepted, archived |
| Commit | ba3aa8591327d43935f000c6884637997438ecb2 |
| Headers | show |
| Series | fetch2/wget: do not hardcode tarball compressors in version check | expand |
On 06.12.21 17:41, Alexander Kanavin wrote: > Signed-off-by: Alexander Kanavin <alex@linutronix.de> > --- > bitbake/lib/bb/fetch2/wget.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index fd9b304961..3021ef359e 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -583,7 +583,7 @@ class Wget(FetchMethod): > > # src.rpm extension was added only for rpm package. Can be removed if the rpm > # packaged will always be considered as having to be manually upgraded > - psuffix_regex = r"(tar\.gz|tgz|tar\.bz2|zip|xz|tar\.lz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)" > + psuffix_regex = r"(tar\.\w+|tgz|zip|xz|rpm|bz2|orig\.tar\.\w+|src\.tar\.\w+|src\.tgz|svnr\d+\.tar\.\w+|stable\.tar\.\w+|src\.rpm)" I've seen tar.gz.md5, tar.gz.sig, tar.siz or even tar.sum files on a bunch of server - unfortunately those would now be considered as well. Not sure if that causes any negative effect on the actual version parsing... > > # match name, version and archive type of a package > package_regex_comp = re.compile(r"(?P<name>%s?\.?v?)(?P<pver>%s)(?P<arch>%s)?[\.-](?P<type>%s$)" > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#13133): https://lists.openembedded.org/g/bitbake-devel/message/13133 > Mute This Topic: https://lists.openembedded.org/mt/87544042/3647476 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [kweihmann@outlook.com] > -=-=-=-=-=-=-=-=-=-=-=- >
I'd hope that those always come in tandem with the actual tarballs, if that proves to be not the case, we can tweak things further. Alex On Mon, 6 Dec 2021 at 17:50, Konrad Weihmann <kweihmann@outlook.com> wrote: > > > On 06.12.21 17:41, Alexander Kanavin wrote: > > Signed-off-by: Alexander Kanavin <alex@linutronix.de> > > --- > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/bitbake/lib/bb/fetch2/wget.py > b/bitbake/lib/bb/fetch2/wget.py > > index fd9b304961..3021ef359e 100644 > > --- a/bitbake/lib/bb/fetch2/wget.py > > +++ b/bitbake/lib/bb/fetch2/wget.py > > @@ -583,7 +583,7 @@ class Wget(FetchMethod): > > > > # src.rpm extension was added only for rpm package. Can be > removed if the rpm > > # packaged will always be considered as having to be manually > upgraded > > - psuffix_regex = > r"(tar\.gz|tgz|tar\.bz2|zip|xz|tar\.lz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)" > > + psuffix_regex = > r"(tar\.\w+|tgz|zip|xz|rpm|bz2|orig\.tar\.\w+|src\.tar\.\w+|src\.tgz|svnr\d+\.tar\.\w+|stable\.tar\.\w+|src\.rpm)" > > I've seen tar.gz.md5, tar.gz.sig, tar.siz or even tar.sum files on a > bunch of server - unfortunately those would now be considered as well. > Not sure if that causes any negative effect on the actual version > parsing... > > > > > # match name, version and archive type of a package > > package_regex_comp = > re.compile(r"(?P<name>%s?\.?v?)(?P<pver>%s)(?P<arch>%s)?[\.-](?P<type>%s$)" > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#13133): > https://lists.openembedded.org/g/bitbake-devel/message/13133 > > Mute This Topic: https://lists.openembedded.org/mt/87544042/3647476 > > Group Owner: bitbake-devel+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ > kweihmann@outlook.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > >
On Mon, 2021-12-06 at 17:41 +0100, Alexander Kanavin wrote: > Signed-off-by: Alexander Kanavin <alex@linutronix.de> > --- > bitbake/lib/bb/fetch2/wget.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py > index fd9b304961..3021ef359e 100644 > --- a/bitbake/lib/bb/fetch2/wget.py > +++ b/bitbake/lib/bb/fetch2/wget.py > @@ -583,7 +583,7 @@ class Wget(FetchMethod): > > # src.rpm extension was added only for rpm package. Can be removed if the rpm > # packaged will always be considered as having to be manually upgraded > - psuffix_regex = r"(tar\.gz|tgz|tar\.bz2|zip|xz|tar\.lz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)" > + psuffix_regex = r"(tar\.\w+|tgz|zip|xz|rpm|bz2|orig\.tar\.\w+|src\.tar\.\w+|src\.tgz|svnr\d+\.tar\.\w+|stable\.tar\.\w+|src\.rpm)" > > # match name, version and archive type of a package > package_regex_comp = re.compile(r"(?P<name>%s?\.?v?)(?P<pver>%s)(?P<arch>%s)?[\.-](?P<type>%s$)" Looks good, is there something we should be adding to lib/bb/tests/fetch.py so that we don't forget this lesson? Cheers, Richard
On Mon, 6 Dec 2021 at 23:47, Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > > # match name, version and archive type of a package > > package_regex_comp = > re.compile(r"(?P<name>%s?\.?v?)(?P<pver>%s)(?P<arch>%s)?[\.-](?P<type>%s$)" > > Looks good, is there something we should be adding to > lib/bb/tests/fetch.py so > that we don't forget this lesson? > I don't think so, 'test that various tar.something suffixes are ok' is already covered, and it's not possible to make a test for 'any tar.suffix is okay'. Alex
On Tue, 7 Dec 2021 at 11:24, Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > The issue was a .orig.tar.XXX so perhaps we're missing a test for that > though? > > I appreciate we can't cover everything, my worry is that if we change this > code > again, we may lose this corner case... > The issue was that orig.tar.gz was covered by the regex (as it was explicitly listed), but orig.tar.bz2 was not. The test already checks various .tar.XXX suffixes, so I don't think there's much value in also adding .orig.tar.XXX and stable.tar.XXX and src.tar.XXX checks. They've all been changed to use \w (i.e. alphanumeric suffix). Alex
On Tue, 7 Dec 2021 at 12:28, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > I appreciate we can't cover everything, my worry is that if we change this > code > >> again, we may lose this corner case... >> > > The issue was that orig.tar.gz was covered by the regex (as it was > explicitly listed), but orig.tar.bz2 was not. The test already checks > various .tar.XXX suffixes, so I don't think there's much value in also > adding .orig.tar.XXX and stable.tar.XXX and src.tar.XXX checks. They've all > been changed to use \w (i.e. alphanumeric suffix). > Let me look again actually, maybe I misunderstood what the test does. Alex
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index fd9b304961..3021ef359e 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -583,7 +583,7 @@ class Wget(FetchMethod): # src.rpm extension was added only for rpm package. Can be removed if the rpm # packaged will always be considered as having to be manually upgraded - psuffix_regex = r"(tar\.gz|tgz|tar\.bz2|zip|xz|tar\.lz|rpm|bz2|orig\.tar\.gz|tar\.xz|src\.tar\.gz|src\.tgz|svnr\d+\.tar\.bz2|stable\.tar\.gz|src\.rpm)" + psuffix_regex = r"(tar\.\w+|tgz|zip|xz|rpm|bz2|orig\.tar\.\w+|src\.tar\.\w+|src\.tgz|svnr\d+\.tar\.\w+|stable\.tar\.\w+|src\.rpm)" # match name, version and archive type of a package package_regex_comp = re.compile(r"(?P<name>%s?\.?v?)(?P<pver>%s)(?P<arch>%s)?[\.-](?P<type>%s$)"
Signed-off-by: Alexander Kanavin <alex@linutronix.de> --- bitbake/lib/bb/fetch2/wget.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)