| Message ID | 20241014125406.2322310-1-alex.kanavin@gmail.com |
|---|---|
| State | Accepted, archived |
| Commit | 138117dfc05248dcb48707c838c8c1cc09f12d07 |
| Headers | show |
| Series | fetch2/wget.py: use default wget user-agent in upstream version checks | expand |
On 14 Oct 2024, at 13:54, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > From: Alexander Kanavin <alex@linutronix.de> > > This has been set to various mozilla versions from this point onwards without > explanation or specific examples: > https://git.yoctoproject.org/poky/commit/?h=master-next&id=ba06cc8943d021f406c3d292ce415b77319ca5c6 > > However, sourceforge has recently banned the current 'fake mozilla' version > we've been supplying, resulting in broken version checks: Of course for fun we’ve just discovered that the openjdk mercurial repository rejects wget/curl and you need to fake a browser U-A to download files… Ross
This has been discussed before. Taking things from mercurial with wget is explicitly disallowed and there’s an official location for those tarballs. I’ll find that thread when I get to a real computer. Alex On Tue 21. Jan 2025 at 18.03, Ross Burton <Ross.Burton@arm.com> wrote: > On 14 Oct 2024, at 13:54, Alexander Kanavin via lists.openembedded.org > <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > > > From: Alexander Kanavin <alex@linutronix.de> > > > > This has been set to various mozilla versions from this point onwards > without > > explanation or specific examples: > > > https://git.yoctoproject.org/poky/commit/?h=master-next&id=ba06cc8943d021f406c3d292ce415b77319ca5c6 > > > > However, sourceforge has recently banned the current 'fake mozilla' > version > > we've been supplying, resulting in broken version checks: > > Of course for fun we’ve just discovered that the openjdk mercurial > repository rejects wget/curl and you need to fake a browser U-A to download > files… > > Ross
On Tue, 21 Jan 2025 at 18:29, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > This has been discussed before. Taking things from mercurial with wget is explicitly disallowed and there’s an official location for those tarballs. I’ll find that thread when I get to a real computer. Found it: https://lists.openembedded.org/g/bitbake-devel/message/16359 Alex
On Tue, Jan 21, 2025 at 9:58 AM Alexander Kanavin <alex.kanavin@gmail.com> wrote: > On Tue, 21 Jan 2025 at 18:29, Alexander Kanavin via > lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> > wrote: > > > > This has been discussed before. Taking things from mercurial with wget > is explicitly disallowed and there’s an official location for those > tarballs. I’ll find that thread when I get to a real computer. > > Found it: > https://lists.openembedded.org/g/bitbake-devel/message/16359 > > Just to add some more insights for folks unaware, the https://openjdk-sources.osci.io/openjdk8/openjdk8u272-ga.tar.xz type tarballs are complete, and include hotspot, corba, jaxp, jaxws, etc. We do need other download locations for the ports though (aarch64 and aarch32), at least until we update to a hypothetical release where those ports are merged into mainline OpenJDK. In the past, each of the components (jdk, hotspot, corba, jaxp, jaxws, etc.) had to be tracked (changesets) and downloaded separately. There is a lot of complicated machinery in meta-java to handle that old model. If someone is motivated to help simplify and update (for master/walnascar, we can discuss backport policy once things actually build and merge) that would be a helpful contribution. > Alex >
https://lists.openembedded.org/g/bitbake-devel/message/16352 User-agent issue will be happen time to time and it will never stop. This is why i push a patch/solution to be able to configure it easly from recipe codes, but nobody from you was interested it. Now it would have easily solved this problem.
On Tue, 21 Jan 2025 at 22:02, Livius via lists.openembedded.org <egyszeregy=freemail.hu@lists.openembedded.org> wrote: > https://lists.openembedded.org/g/bitbake-devel/message/16352 > > User-agent issue will be happened time to time and it will never stop. This is why i pushed a patch/solution to be able to configure it easly from recipe codes, but nobody from you was interested in it. Now it would have easily solved this problem. The correct solution is to fix meta-java. Please focus your energy there, rather than trying to portray this as some kind of persistent, wide-spread issue (it isn't). Alex
On Tue, 21 Jan 2025 at 22:41, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > The correct solution is to fix meta-java. Please focus your energy > there, rather than trying to portray this as some kind of persistent, > wide-spread issue (it isn't). Specifically, this means adjusting SRC_URIs according to https://lists.openembedded.org/g/bitbake-devel/message/16359 Alex
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index e8b1292433f..493a5b62ee2 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -463,7 +463,7 @@ class Wget(FetchMethod): f = tempfile.NamedTemporaryFile() with tempfile.TemporaryDirectory(prefix="wget-index-") as workdir, tempfile.NamedTemporaryFile(dir=workdir, prefix="wget-listing-") as f: fetchcmd = self.basecmd - fetchcmd += " -O " + f.name + " --user-agent='" + self.user_agent + "' '" + uri + "'" + fetchcmd += " -O " + f.name + " '" + uri + "'" try: self._runwget(ud, d, fetchcmd, True, workdir=workdir) fetchresult = f.read()