Message ID | 20230112213301.11768-1-s.zhmylev@yadro.com |
---|---|
State | New |
Headers | show |
Series | package_rpm: allow arch overriding | expand |
On Fri, 2023-01-13 at 00:33 +0300, Sergey Zhmylev wrote: > From: Sergei Zhmylev <s.zhmylev@yadro.com> > > Currently arch is being calculated from either MACHINE_ARCH or > TUNE_PKGARCH. Some recipes do override PACKAGE_ARCH intentionally. > This commit makes possible to override ARCH for RPM packages > separately in order to simplify common rpm naming conformance. > > Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com> > --- > meta/classes-global/package_rpm.bbclass | 4 ++++ > 1 file changed, 4 insertions(+) > > diff --git a/meta/classes-global/package_rpm.bbclass b/meta/classes-global/package_rpm.bbclass > index 85d0bd7fce..3fb7466778 100644 > --- a/meta/classes-global/package_rpm.bbclass > +++ b/meta/classes-global/package_rpm.bbclass > @@ -678,6 +678,10 @@ python do_package_rpm () { > bb.utils.mkdirhier(pkgwritedir) > os.chmod(pkgwritedir, 0o755) > > + rpm_arch_override = d.getVar("RPM_ARCH_OVERRIDE") > + if package_arch != "noarch" and rpm_arch_override: > + pkgarch = rpm_arch_override > + > cmd = rpmbuild > cmd = cmd + " --noclean --nodeps --short-circuit --target " + pkgarch + " --buildroot " + pkgd > cmd = cmd + " --define '_topdir " + workdir + "' --define '_rpmdir " + pkgwritedir + "'" This makes me very nervous. PACKAGE_ARCH can be overridden and I know of BSPs which inject a whole new level of architecture into the system but they don't need to make rpm backend specific changes to make it work. If we were to add such a thing we'd need a much clearer picture of when it should be used and why it is needed. Wouldn't all package backends need to adjust this? Cheers, Richard
Exactly! Even more: when I build for "genericx86_64", I get mix of packages with architecture one of: "x86_64", "genericx86_64", "core2_64", and "noarch". Our audit team highlited this as this is not compliant with common RPM naming convention. In that case override architecture string only for packaging would be, from my perspective, the best workaround. At the same time I doubt we have to change other package backends as they're less strict while architecture overriding effectively looses (probably) important information. -- With best wishes, Sergei Zhmylev Engineering consultant OS development department On Thu, 2023-01-12 at 22:32 +0000, Richard Purdie wrote: > «Внимание! Данное письмо от внешнего адресата!» > > On Fri, 2023-01-13 at 00:33 +0300, Sergey Zhmylev wrote: > > From: Sergei Zhmylev <s.zhmylev@yadro.com> > > > > Currently arch is being calculated from either MACHINE_ARCH or > > TUNE_PKGARCH. Some recipes do override PACKAGE_ARCH intentionally. > > This commit makes possible to override ARCH for RPM packages > > separately in order to simplify common rpm naming conformance. > > > > Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com> > > --- > > meta/classes-global/package_rpm.bbclass | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/meta/classes-global/package_rpm.bbclass > > b/meta/classes-global/package_rpm.bbclass > > index 85d0bd7fce..3fb7466778 100644 > > --- a/meta/classes-global/package_rpm.bbclass > > +++ b/meta/classes-global/package_rpm.bbclass > > @@ -678,6 +678,10 @@ python do_package_rpm () { > > bb.utils.mkdirhier(pkgwritedir) > > os.chmod(pkgwritedir, 0o755) > > > > + rpm_arch_override = d.getVar("RPM_ARCH_OVERRIDE") > > + if package_arch != "noarch" and rpm_arch_override: > > + pkgarch = rpm_arch_override > > + > > cmd = rpmbuild > > cmd = cmd + " --noclean --nodeps --short-circuit --target " + > > pkgarch + " --buildroot " + pkgd > > cmd = cmd + " --define '_topdir " + workdir + "' --define > > '_rpmdir " + pkgwritedir + "'" > > This makes me very nervous. PACKAGE_ARCH can be overridden and I know > of BSPs which inject a whole new level of architecture into the > system > but they don't need to make rpm backend specific changes to make it > work. > > If we were to add such a thing we'd need a much clearer picture of > when > it should be used and why it is needed. Wouldn't all package backends > need to adjust this? > > Cheers, > > Richard >
On Fri, 13 Jan 2023 at 09:25, Sergey Zhmylev <s.zhmylev@yadro.com> wrote: > > Exactly! Even more: when I build for "genericx86_64", I get mix of > packages with architecture one of: "x86_64", "genericx86_64", > "core2_64", and "noarch". Our audit team highlited this as this is not > compliant with common RPM naming convention. Can you please link to (or describe with references) the common naming convention? Alex
Sure, RPM naming convention from rpm.org: http://ftp.rpm.org/max-rpm/ch-rpm-file-format.html As you can see, there are nothing about "core2_64" architecture. -- With best wishes, Sergei Zhmylev Engineering consultant OS development department On Fri, 2023-01-13 at 09:34 +0100, Alexander Kanavin wrote: > «Внимание! Данное письмо от внешнего адресата!» > > On Fri, 13 Jan 2023 at 09:25, Sergey Zhmylev <s.zhmylev@yadro.com> > wrote: > > > > Exactly! Even more: when I build for "genericx86_64", I get mix of > > packages with architecture one of: "x86_64", "genericx86_64", > > "core2_64", and "noarch". Our audit team highlited this as this is > > not > > compliant with common RPM naming convention. > > Can you please link to (or describe with references) the common > naming > convention? > > Alex >
This document hasn't been maintained in decades, and is not in any way official. Alex On Fri, 13 Jan 2023 at 10:14, Sergey Zhmylev <s.zhmylev@yadro.com> wrote: > > Sure, RPM naming convention from rpm.org: > http://ftp.rpm.org/max-rpm/ch-rpm-file-format.html > As you can see, there are nothing about "core2_64" architecture. > > -- > With best wishes, > Sergei Zhmylev > Engineering consultant > OS development department > > > > > > > > > On Fri, 2023-01-13 at 09:34 +0100, Alexander Kanavin wrote: > > «Внимание! Данное письмо от внешнего адресата!» > > > > On Fri, 13 Jan 2023 at 09:25, Sergey Zhmylev <s.zhmylev@yadro.com> > > wrote: > > > > > > Exactly! Even more: when I build for "genericx86_64", I get mix of > > > packages with architecture one of: "x86_64", "genericx86_64", > > > "core2_64", and "noarch". Our audit team highlited this as this is > > > not > > > compliant with common RPM naming convention. > > > > Can you please link to (or describe with references) the common > > naming > > convention? > > > > Alex > >
On Fri, 2023-01-13 at 09:14 +0000, Sergey Zhmylev wrote: > Sure, RPM naming convention from rpm.org: > http://ftp.rpm.org/max-rpm/ch-rpm-file-format.html > As you can see, there are nothing about "core2_64" architecture. We build target binaries to different levels of tuning, there are packages which run everywhere (noarch), there are binaries which are optimised to a specific tune and there is machine specific output. People can inject other levels of packages as well. Giving a hint of the kind of thing a given rpm contains seems like a perfectly reasonable thing for us to be doing. If you come from the rpm world, yes it is slightly unusual but as the document above says, it works fine. We're not "in-compliant", we're just adapting rpm usage to the environment it is being used in. So no, I don't think we should be taking a patch which would just confuse things for what seems to be a cosmetic reason. Can you perhaps convince your audit team this is a sensible usage given how the project builds and uses the rpms? Cheers, Richard
On Fri, 13 Jan 2023 at 10:50, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > Can you perhaps convince your audit team this is a sensible usage given > how the project builds and uses the rpms? One more thing I want to mention is that if you want to align the architectures so that you can install RPMs from Fedora, RHEL or any other RPM-based binary distro, don't. It's not going to work at all, or will cause you lots of pain and suffering. We use rpm for its packaging and dependency resolution functions, but no one ever promised any kind of 'compatibility' with Red Hat products. Alex
> One more thing I want to mention is that if you want to align the > architectures so that you can install RPMs from Fedora, RHEL or any > other RPM-based binary distro, don't. It's not going to work at all, > or will cause you lots of pain and suffering. We use rpm for its > packaging and dependency resolution functions, but no one ever > promised any kind of 'compatibility' with Red Hat products. > Indeed, I'm not going to align those names for this. Moreover, for this particular case rpm(1) supports several parameters to disable arch/os/etc checks. > So no, I don't think we should be taking a patch which would just > confuse things for what seems to be a cosmetic reason. > The patch proposed is not about a cosmetic change, it's about precise control over what packaging system produces. Given the recipes can override variables with a priority over any confs/classes, it looks like there is no proper way to overload the values (in particular PACKAGE_ARCH) used during packaging process. Some variables could be used to adjust TUNE flags and their values should stay intact prior to packaging. Anyways, your word will be the last but I am still sure that "more control" is better than "no conrol". Kind wishes, Sergei
> On 13 Jan 2023, at 12:29, Sergey Zhmylev via lists.openembedded.org <s.zhmylev=yadro.com@lists.openembedded.org> wrote: >> So no, I don't think we should be taking a patch which would just >> confuse things for what seems to be a cosmetic reason. >> > > The patch proposed is not about a cosmetic change, it's about precise > control over what packaging system produces. Given the recipes can > override variables with a priority over any confs/classes, it looks > like there is no proper way to overload the values (in particular > PACKAGE_ARCH) used during packaging process. Some variables could be > used to adjust TUNE flags and their values should stay intact prior to > packaging. What is the use-case for this level of control then, and why would it be specific to RPM? Ross
> > What is the use-case for this level of control then, and why would it > be specific to RPM? > Despite Alexander's words about this document, most of (I suppose, actually all) wide-used rpm-based distros use the same well-known rpm packages naming convention: red hat, opensuse, centos, oracle enterprise linux, fedora, ... There are a lot of useful tools in this huge world of enterprise-level distros, especially all those staff related to static security scans/audit and around SBOM collection and processing. Those tools rely on some basic rules of rpm naming for their analysis and do not handle architectures like "genericx86_64". I.e. this level of control opens for us a tremendous window of possibilities when our build pretends to be a mature distro. BR, Sergei
On Fri, Jan 13, 2023 at 11:01 AM Sergey Zhmylev <s.zhmylev@yadro.com> wrote: > > > > > What is the use-case for this level of control then, and why would it > > be specific to RPM? > > > > Despite Alexander's words about this document, most of (I suppose, > actually all) wide-used rpm-based distros use the same well-known rpm > packages naming convention: red hat, opensuse, centos, oracle > enterprise linux, fedora, ... > > There are a lot of useful tools in this huge world of enterprise-level > distros, especially all those staff related to static security > scans/audit and around SBOM collection and processing. Those tools > rely on some basic rules of rpm naming for their analysis and do not > handle architectures like "genericx86_64". You mean things that the Yocto Project does itself ? :) My humorous answer is on purpose. If there are gaps in the tooling, the project is interested in filing those gaps, and doing it in a way that anyone using the project could leverage. Not just the users of a specific packaging format. It isn't that we wouldn't want to enable that sort of tools / packaging capability, but if it opens us up to complexity or other issues down the road, it needs to be carefully considered. Which is what the questions in the thread are trying to determine. > > I.e. this level of control opens for us a tremendous window of > possibilities when our build pretends to be a mature distro. > FWIW. I would not characterize a distro created with the Yocto project as "immature". Cheers, Bruce > BR, Sergei > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#175836): https://lists.openembedded.org/g/openembedded-core/message/175836 > Mute This Topic: https://lists.openembedded.org/mt/96233100/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Fri, 13 Jan 2023 at 17:01, Sergey Zhmylev <s.zhmylev@yadro.com> wrote: > There are a lot of useful tools in this huge world of enterprise-level > distros, especially all those staff related to static security > scans/audit and around SBOM collection and processing. Those tools > rely on some basic rules of rpm naming for their analysis and do not > handle architectures like "genericx86_64". Can you list the tools please? Are they under open source licenses? What do they do that Yocto doesn't yet offer? Yocto does CVE scans, and SBOM generation as well, so what is it that is missing? It also goes both ways: you can figure out why the tools do not handle architectures in a generic way and submit a patch to respective upstreams so that they do. 'Enterprise-level distros' means vendor lock-in and expensive subscription fees. Precisely the thing that most or all of us here do not want. Yes, we are not mature. Alex -
diff --git a/meta/classes-global/package_rpm.bbclass b/meta/classes-global/package_rpm.bbclass index 85d0bd7fce..3fb7466778 100644 --- a/meta/classes-global/package_rpm.bbclass +++ b/meta/classes-global/package_rpm.bbclass @@ -678,6 +678,10 @@ python do_package_rpm () { bb.utils.mkdirhier(pkgwritedir) os.chmod(pkgwritedir, 0o755) + rpm_arch_override = d.getVar("RPM_ARCH_OVERRIDE") + if package_arch != "noarch" and rpm_arch_override: + pkgarch = rpm_arch_override + cmd = rpmbuild cmd = cmd + " --noclean --nodeps --short-circuit --target " + pkgarch + " --buildroot " + pkgd cmd = cmd + " --define '_topdir " + workdir + "' --define '_rpmdir " + pkgwritedir + "'"