| Message ID | 20260819042417.22411-1-giancarlo.cicellyn@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | bitbake-setup: preserve build config when init script is missing | expand |
On Wed, 19 Aug 2026 at 06:24, Giancarlo Cicellyn Comneno via
lists.openembedded.org
<giancarlo.cicellyn=gmail.com@lists.openembedded.org> wrote:
> + os.remove(os.path.join(self.testrepopath, 'oe-init-build-env'))
This seems like a contrived scenario, so I'd like to understand it
better. What prompted you to ask AI to write a patch for it? How can
this happen in actual usage?
Alex
Hi Alex, Thanks for questioning this. This came from auditing the failure paths in `bitbake-setup`, rather than from a reported OE-Core incident. I went back and checked whether the scenario used by the test is actually permitted by the setup/update model. `checkout_layers()` updates configured git sources and selects the repository providing the setup helpers based on the presence of `scripts/oe-setup-build`. It then creates `oe-init-build-env-dir` pointing at that repository, but there is no invariant or configuration validation requiring that repository to also contain `oe-init-build-env`. For a configuration using `bb-layers` rather than `oe-template`, `setup_bitbake_build()` does explicitly handle a missing `oe-init-build-env`, so that state is already considered possible by the code. The problematic ordering is that on update an existing `build/conf` is renamed and a new one is generated before that check happens. If the updated source revision no longer provides `oe-init-build-env`, the function logs the error and returns, leaving the regenerated configuration in place; the caller then continues as though setup succeeded. The test removes `oe-init-build-env` from the next revision specifically to reproduce that transition. I don't have evidence that OE-Core itself has made such a transition, and I should have made that clear in the commit message. So the motivation is defensive handling of a supported source-update failure path, not a failure I've observed in OE-Core. If you consider the disappearance of `oe-init-build-env` outside the supported contract despite the current handling for it, then I agree the patch shouldn't be kept. Giancarlo On Wed, 19 Aug 2026 10:36:32 +0200, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > On Wed, 19 Aug 2026 at 06:24, Giancarlo Cicellyn Comneno via > lists.openembedded.org > <giancarlo.cicellyn=gmail.com@lists.openembedded.org> wrote: > > + os.remove(os.path.join(self.testrepopath, 'oe-init-build-env')) > > This seems like a contrived scenario, so I'd like to understand it > better. What prompted you to ask AI to write a patch for it? How can > this happen in actual usage? > > Alex
On Wed, 19 Aug 2026 at 12:09, Giancarlo Cicellyn Comneno <giancarlo.cicellyn@gmail.com> wrote: > So the motivation is defensive handling of a supported source-update > failure path, not a failure I've observed in OE-Core. > > If you consider the disappearance of `oe-init-build-env` outside the > supported contract despite the current handling for it, then I agree > the patch shouldn't be kept. > > Giancarlo Giancarlo, please do not use AI to write responses in a conversation. I want to talk to *you*, not to your AI agent. If you do this, then anything coming from you will be simply discarded on the grounds that you don't understand your own patches. Specifically here, I'd like to hear from *you* what instructions you gave to AI. Was it something along the lines of 'find problematic code paths in bitbake-setup and fix them?'. You need to be clear and transparent about it, if you want to maintain trust in your submissions. Thanks, Alex
Hi Alex, I want to clarify a few things directly. First, regarding AI use: I have not found anything in the published Yocto/OpenEmbedded contribution policy stating that review replies must be written personally without AI assistance. On the contrary, the reason the patch contains the `AI-Generated:` trailer is precisely because I was trying to follow the project's published policy correctly. I did not add that casually; I added it because the contributor documentation explicitly requires AI-generated code to be disclosed before the human `Signed-off-by`. I understand that you personally expect review conversations to be written without AI assistance. That expectation was not clear to me before, but now it is. I accept that this is how you prefer to interact with contributors, even though for me using AI assistance is part of how I work and learn, and it saves time that I can spend on the technical work itself and on the other projects I handle in parallel. About the prompt: I do not remember the exact wording I gave the AI. I am a human being, not ChatGPT-9 :) What I can tell you accurately is that the goal was to learn on the job by auditing `bitbake-setup`, looking for problematic or fragile failure paths, and trying to validate anything found with tests before proposing a fix. This patch came out of that process. It did not come from a real OE-Core incident or from a user bug report, and I already made that clear in my previous reply. There is one point, however, that I would still like to understand technically, because I do not think it has been answered yet. My understanding of the current code is: - `checkout_layers()` selects the repository providing the setup helpers based on the presence of `scripts/oe-setup-build`; - it creates `oe-init-build-env-dir` pointing at that repository; - I could not find an invariant or validation rule requiring that repository to also contain `oe-init-build-env`; - for the `bb-layers` path, `setup_bitbake_build()` explicitly handles the case where `oe-init-build-env` is missing; - but that check currently happens only after an existing `build/conf` has already been moved aside and a new one has been generated. So I would still like to understand which part of that reasoning is wrong. Is your technical objection that the source transition exercised by the test is outside the supported contract of `bitbake-setup`? Or is the concern instead that the transition is technically possible but too theoretical / unrealistic to justify an upstream patch? I am asking because those are two different conclusions, and I would genuinely like to learn which one applies here. If there is an invariant or project assumption I missed, I am happy to understand it and withdraw the patch on that basis. If instead the path is valid but simply not considered worth handling because there is no known real-world occurrence, that is also useful feedback for me. In any case, thank you for the time you have spent reviewing this. I am here primarily to learn by doing useful work, and I have no interest in defending a patch whose premise does not hold. Regards, Giancarlo On Wed, 19 Aug 2026 12:19:10 +0200, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > On Wed, 19 Aug 2026 at 12:09, Giancarlo Cicellyn Comneno > <giancarlo.cicellyn@gmail.com> wrote: > > > So the motivation is defensive handling of a supported source-update > > failure path, not a failure I've observed in OE-Core. > > > > If you consider the disappearance of `oe-init-build-env` outside the > > supported contract despite the current handling for it, then I agree > > the patch shouldn't be kept. > > > > Giancarlo > > Giancarlo, > > please do not use AI to write responses in a conversation. I want to > talk to *you*, not to your AI agent. If you do this, then anything > coming from you will be simply discarded on the grounds that you don't > understand your own patches. > > Specifically here, I'd like to hear from *you* what instructions you > gave to AI. Was it something along the lines of 'find problematic code > paths in bitbake-setup and fix them?'. You need to be clear and > transparent about it, if you want to maintain trust in your > submissions. > > Thanks, > Alex
On Wed, 19 Aug 2026 at 12:51, Giancarlo Cicellyn Comneno <giancarlo.cicellyn@gmail.com> wrote: > I understand that you personally expect review conversations to be > written without AI assistance. That expectation was not clear to me > before, but now it is. I accept that this is how you prefer to > interact with contributors, even though for me using AI assistance is > part of how I work and learn, and it saves time that I can spend on > the technical work itself and on the other projects I handle in > parallel. There's a growing community consensus about this: reviewers want to talk with humans, and giving them AI generated replies is likely to antagonize and frustrate them. It gets worse when people keep doing it despite being asked to stop, and then get defensive about it. E.g. Rust has already spelled it out clearly: https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/#mechanically-copy-pasting-llm-output-is-a-waste-of-time They say: === We will often get people who respond to review comments by copy-pasting them into their LLM, then copy-pasting its response back onto GitHub. Bluntly: this is a waste of everyone's time. If we wanted an LLM's opinion, we could have asked it ourselves. We want to hear your thoughts, not a machine's. Moreover, this is a breach of trust between the reviewer and the author. Our assumption when we review is that we're talking to a real person who wants to do their best work. Pasting LLM text creates suspicion: does the author actually care? Is there a person here at all? === I have also written about this here, which touches on the reasons I want interaction with humans: https://www.linkedin.com/feed/update/urn:li:activity:7490778888923541504/ Do take your own reputation much more seriously than any possible time savings (which often end up negated through actual human review and having to manually establish and then rework AI mistakes or incorrect assumptions). Yocto is going to tweak the policy to more clearly say what is okay and what isn't, the exact wording is under consideration: https://lists.openembedded.org/g/openembedded-core/message/243686 > What I can tell you accurately is that the goal was to learn on the > job by auditing `bitbake-setup`, looking for problematic or fragile > failure paths, and trying to validate anything found with tests before > proposing a fix. This patch came out of that process. It did not come > from a real OE-Core incident or from a user bug report, and I already > made that clear in my previous reply. The patches would've been better received if this was done first: - research whether the scenarios they fix are realistic *before* submission. You are free to use AI any way you want in that activity. - if that research was inconclusive, or you lack experience to make that judgement, then mark the patches as RFC, and add your own commentary to the commit messages: specifically that you have asked AI to find problems, but you are not sure if the discovered problems are real, and seek community feedback. At the very least, this last bit should've been written in a response to my question, by *you*. Keep this in mind: you can't assume reviewers are compensated for doing reviews, and all too often in open source they're not, or their time is stretched extremely thin. For example, I'm not getting paid to have this conversation. So any needed research needs to happen up front, so that you don't end up asking me, or anyone, to explain things for you, or prove to you that the patches aren't necessary. > Is your technical objection that the source transition exercised by > the test is outside the supported contract of `bitbake-setup`? > > Or is the concern instead that the transition is technically possible > but too theoretical / unrealistic to justify an upstream patch? > > I am asking because those are two different conclusions, and I would > genuinely like to learn which one applies here. The questions to ask, as part of that upfront research I mentioned above, is neither of these. It's "What are these files used for? How likely are they to be removed?" scripts/oe-setup-build is used to set up a bitbake build directory from a configuration template in a layer. Oe-init-build-env is used to set bitbake-specific variables in a unix environment so that bitbake can find metadata and the bitbake build directory set up by oe-setup-build, and then run actual yocto builds. Both files are a core part of yocto workflows, they're standard, documented, and especially oe-init-build-env is used by almost everyone. So they will stay in oe-core repository until the end of time; it's completely unrealistic to remove them from that repo. > In any case, thank you for the time you have spent reviewing this. I > am here primarily to learn by doing useful work, and I have no > interest in defending a patch whose premise does not hold. If you'd like to help, it's best to first establish what would be helpful. Let me assure you: AI assisted bug hunting is not where help would be most appreciated. Again, it's best to ask the community first what you could do, given your level of experience and particular interests and goals. Or read the weekly status emails such as one I linked above, where the most pressing issues are highlighted. Alex
It has become fairly clear that we have incompatible views on what constitutes acceptable use of AI. For that reason, thank you for the discussion, but I’d prefer to move on. On Wed, 19 Aug 2026 14:49:38 +0200, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > On Wed, 19 Aug 2026 at 12:51, Giancarlo Cicellyn Comneno > <giancarlo.cicellyn@gmail.com> wrote: > > I understand that you personally expect review conversations to be > > written without AI assistance. That expectation was not clear to me > > before, but now it is. I accept that this is how you prefer to > > interact with contributors, even though for me using AI assistance is > > part of how I work and learn, and it saves time that I can spend on > > the technical work itself and on the other projects I handle in > > parallel. > > There's a growing community consensus about this: reviewers want to > talk with humans, and giving them AI generated replies is likely to > antagonize and frustrate them. It gets worse when people keep doing it > despite being asked to stop, and then get defensive about it. > > E.g. Rust has already spelled it out clearly: > https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/#mechanically-copy-pasting-llm-output-is-a-waste-of-time > > They say: > === > We will often get people who respond to review comments by > copy-pasting them into their LLM, then copy-pasting its response back > onto GitHub. Bluntly: this is a waste of everyone's time. If we wanted > an LLM's opinion, we could have asked it ourselves. We want to hear > your thoughts, not a machine's. > > Moreover, this is a breach of trust between the reviewer and the > author. Our assumption when we review is that we're talking to a real > person who wants to do their best work. Pasting LLM text creates > suspicion: does the author actually care? Is there a person here at > all? > === > > I have also written about this here, which touches on the reasons I > want interaction with humans: > https://www.linkedin.com/feed/update/urn:li:activity:7490778888923541504/ > > Do take your own reputation much more seriously than any possible time > savings (which often end up negated through actual human review and > having to manually establish and then rework AI mistakes or incorrect > assumptions). > > Yocto is going to tweak the policy to more clearly say what is okay > and what isn't, the exact wording is under consideration: > https://lists.openembedded.org/g/openembedded-core/message/243686 > > > What I can tell you accurately is that the goal was to learn on the > > job by auditing `bitbake-setup`, looking for problematic or fragile > > failure paths, and trying to validate anything found with tests before > > proposing a fix. This patch came out of that process. It did not come > > from a real OE-Core incident or from a user bug report, and I already > > made that clear in my previous reply. > > The patches would've been better received if this was done first: > > - research whether the scenarios they fix are realistic *before* > submission. You are free to use AI any way you want in that activity. > > - if that research was inconclusive, or you lack experience to make > that judgement, then mark the patches as RFC, and add your own > commentary to the commit messages: specifically that you have asked AI > to find problems, but you are not sure if the discovered problems are > real, and seek community feedback. > > At the very least, this last bit should've been written in a response > to my question, by *you*. > > Keep this in mind: you can't assume reviewers are compensated for > doing reviews, and all too often in open source they're not, or their > time is stretched extremely thin. For example, I'm not getting paid to > have this conversation. So any needed research needs to happen up > front, so that you don't end up asking me, or anyone, to explain > things for you, or prove to you that the patches aren't necessary. > > > Is your technical objection that the source transition exercised by > > the test is outside the supported contract of `bitbake-setup`? > > > > Or is the concern instead that the transition is technically possible > > but too theoretical / unrealistic to justify an upstream patch? > > > > I am asking because those are two different conclusions, and I would > > genuinely like to learn which one applies here. > > The questions to ask, as part of that upfront research I mentioned > above, is neither of these. It's "What are these files used for? How > likely are they to be removed?" > > scripts/oe-setup-build is used to set up a bitbake build directory > from a configuration template in a layer. Oe-init-build-env is used to > set bitbake-specific variables in a unix environment so that bitbake > can find metadata and the bitbake build directory set up by > oe-setup-build, and then run actual yocto builds. > > Both files are a core part of yocto workflows, they're standard, > documented, and especially oe-init-build-env is used by almost > everyone. So they will stay in oe-core repository until the end of > time; it's completely unrealistic to remove them from that repo. > > > In any case, thank you for the time you have spent reviewing this. I > > am here primarily to learn by doing useful work, and I have no > > interest in defending a patch whose premise does not hold. > > If you'd like to help, it's best to first establish what would be > helpful. Let me assure you: AI assisted bug hunting is not where help > would be most appreciated. Again, it's best to ask the community first > what you could do, given your level of experience and particular > interests and goals. Or read the weekly status emails such as one I > linked above, where the most pressing issues are highlighted. > > Alex
It has become fairly clear that we have incompatible views on what constitutes acceptable use of AI. For that reason, thank you for the discussion, but I’d prefer to move on. On Wed, 19 Aug 2026 14:49:38 +0200, Alexander Kanavin <alex.kanavin@gmail.com> wrote: > On Wed, 19 Aug 2026 at 12:51, Giancarlo Cicellyn Comneno > <giancarlo.cicellyn@gmail.com> wrote: > > I understand that you personally expect review conversations to be > > written without AI assistance. That expectation was not clear to me > > before, but now it is. I accept that this is how you prefer to > > interact with contributors, even though for me using AI assistance is > > part of how I work and learn, and it saves time that I can spend on > > the technical work itself and on the other projects I handle in > > parallel. > > There's a growing community consensus about this: reviewers want to > talk with humans, and giving them AI generated replies is likely to > antagonize and frustrate them. It gets worse when people keep doing it > despite being asked to stop, and then get defensive about it. > > E.g. Rust has already spelled it out clearly: > https://blog.rust-lang.org/inside-rust/2026/08/05/rust-langrust-is-adopting-an-llm-policy/#mechanically-copy-pasting-llm-output-is-a-waste-of-time > > They say: > === > We will often get people who respond to review comments by > copy-pasting them into their LLM, then copy-pasting its response back > onto GitHub. Bluntly: this is a waste of everyone's time. If we wanted > an LLM's opinion, we could have asked it ourselves. We want to hear > your thoughts, not a machine's. > > Moreover, this is a breach of trust between the reviewer and the > author. Our assumption when we review is that we're talking to a real > person who wants to do their best work. Pasting LLM text creates > suspicion: does the author actually care? Is there a person here at > all? > === > > I have also written about this here, which touches on the reasons I > want interaction with humans: > https://www.linkedin.com/feed/update/urn:li:activity:7490778888923541504/ > > Do take your own reputation much more seriously than any possible time > savings (which often end up negated through actual human review and > having to manually establish and then rework AI mistakes or incorrect > assumptions). > > Yocto is going to tweak the policy to more clearly say what is okay > and what isn't, the exact wording is under consideration: > https://lists.openembedded.org/g/openembedded-core/message/243686 > > > What I can tell you accurately is that the goal was to learn on the > > job by auditing `bitbake-setup`, looking for problematic or fragile > > failure paths, and trying to validate anything found with tests before > > proposing a fix. This patch came out of that process. It did not come > > from a real OE-Core incident or from a user bug report, and I already > > made that clear in my previous reply. > > The patches would've been better received if this was done first: > > - research whether the scenarios they fix are realistic *before* > submission. You are free to use AI any way you want in that activity. > > - if that research was inconclusive, or you lack experience to make > that judgement, then mark the patches as RFC, and add your own > commentary to the commit messages: specifically that you have asked AI > to find problems, but you are not sure if the discovered problems are > real, and seek community feedback. > > At the very least, this last bit should've been written in a response > to my question, by *you*. > > Keep this in mind: you can't assume reviewers are compensated for > doing reviews, and all too often in open source they're not, or their > time is stretched extremely thin. For example, I'm not getting paid to > have this conversation. So any needed research needs to happen up > front, so that you don't end up asking me, or anyone, to explain > things for you, or prove to you that the patches aren't necessary. > > > Is your technical objection that the source transition exercised by > > the test is outside the supported contract of `bitbake-setup`? > > > > Or is the concern instead that the transition is technically possible > > but too theoretical / unrealistic to justify an upstream patch? > > > > I am asking because those are two different conclusions, and I would > > genuinely like to learn which one applies here. > > The questions to ask, as part of that upfront research I mentioned > above, is neither of these. It's "What are these files used for? How > likely are they to be removed?" > > scripts/oe-setup-build is used to set up a bitbake build directory > from a configuration template in a layer. Oe-init-build-env is used to > set bitbake-specific variables in a unix environment so that bitbake > can find metadata and the bitbake build directory set up by > oe-setup-build, and then run actual yocto builds. > > Both files are a core part of yocto workflows, they're standard, > documented, and especially oe-init-build-env is used by almost > everyone. So they will stay in oe-core repository until the end of > time; it's completely unrealistic to remove them from that repo. > > > In any case, thank you for the time you have spent reviewing this. I > > am here primarily to learn by doing useful work, and I have no > > interest in defending a patch whose premise does not hold. > > If you'd like to help, it's best to first establish what would be > helpful. Let me assure you: AI assisted bug hunting is not where help > would be most appreciated. Again, it's best to ask the community first > what you could do, given your level of experience and particular > interests and goals. Or read the weekly status emails such as one I > linked above, where the most pressing issues are highlighted. > > Alex
diff --git a/bin/bitbake-setup b/bin/bitbake-setup index 34260139c..d214ad568 100755 --- a/bin/bitbake-setup +++ b/bin/bitbake-setup @@ -354,6 +354,10 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c if template and not os.path.exists(oesetupbuild): raise Exception("Cannot complete setting up a bitbake build directory from OpenEmbedded template '{}' as oe-setup-build was not found in any layers; please use oe-init-build-env manually.".format(template)) + oeinitbuildenvdir = os.path.join(layerdir, 'oe-init-build-env-dir') + if not template and not os.path.exists(os.path.join(oeinitbuildenvdir, "oe-init-build-env")): + raise Exception("Could not find oe-init-build-env in any of the layers; please use another mechanism to initialize the bitbake environment") + bitbake_confdir = os.path.join(bitbake_builddir, 'conf') backup_bitbake_confdir = add_unique_timestamp_to_path(os.path.join(bitbake_builddir, 'conf-backup')) upstream_bitbake_confdir = add_unique_timestamp_to_path(os.path.join(bitbake_builddir, 'conf-upstream')) @@ -368,10 +372,6 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c if template: bb.process.run([oesetupbuild, "setup", "-c", template, "-b", bitbake_builddir, "--no-shell"]) else: - oeinitbuildenvdir = os.path.join(layerdir, 'oe-init-build-env-dir') - if not os.path.exists(os.path.join(oeinitbuildenvdir, "oe-init-build-env")): - logger.error("Could not find oe-init-build-env in any of the layers; please use another mechanism to initialize the bitbake environment") - return _make_init_build_env(bitbake_builddir, os.path.realpath(oeinitbuildenvdir)) _prepend_passthrough_to_init_build_env(bitbake_builddir) diff --git a/lib/bb/tests/setup.py b/lib/bb/tests/setup.py index 53a07ee3a..ab18abd0a 100644 --- a/lib/bb/tests/setup.py +++ b/lib/bb/tests/setup.py @@ -578,6 +578,63 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"])) out = self.runbbsetup(["init", "--non-interactive", "-L", "test-repo", self.testrepopath, "--setup-dir-name", custom_setup_dir, "test-config-1", "gadget"]) _check_local_sources(custom_setup_dir) + def test_update_preserves_build_conf_when_init_build_env_missing(self): + if 'BBPATH' in os.environ: + del os.environ['BBPATH'] + os.chdir(self.tempdir) + + self.runbbsetup([ + "settings", "set", "default", "registry", + "'git://{};protocol=file;branch=master;rev=master'".format( + self.registrypath + ), + ]) + self.add_file_to_testrepo('test-file', 'initial\n') + self.add_json_config_to_registry( + 'test-config-1.conf.json', 'master', 'master' + ) + + self.runbbsetup([ + "init", + "--non-interactive", + "test-config-1", + "gadget-notemplate", + ]) + + setuppath = self.get_setup_path( + 'test-config-1', 'gadget-notemplate' + ) + local_conf = os.path.join( + setuppath, 'build', 'conf', 'local.conf' + ) + + user_content = 'USER_SETTING = "preserve-me"\n' + with open(local_conf, 'w') as f: + f.write(user_content) + + os.remove(os.path.join(self.testrepopath, 'oe-init-build-env')) + self.git(['add', '-u'], cwd=self.testrepopath) + self.git( + ['commit', '-m', 'Remove oe-init-build-env'], + cwd=self.testrepopath, + ) + + os.environ['BBPATH'] = os.path.join(setuppath, 'build') + try: + with self.assertRaisesRegex( + bb.process.ExecutionError, + "Could not find oe-init-build-env", + ): + self.runbbsetup([ + "update", + "--update-bb-conf=yes", + ]) + finally: + del os.environ['BBPATH'] + + with open(local_conf) as f: + self.assertEqual(f.read(), user_content) + def test_vscode(self): if 'BBPATH' in os.environ: del os.environ['BBPATH']
setup_bitbake_build() moves an existing build/conf aside before checking whether oe-init-build-env is available. If the script disappears between setup and update, the update currently returns without restoring the previous configuration, leaving a freshly generated build/conf in place and reporting success. Check for oe-init-build-env before modifying build/conf and fail explicitly when it is unavailable. Add a regression test verifying that the update fails and preserves the user's local.conf. Tests: LC_ALL=C LANG=C ./bin/bitbake-selftest bb.tests.setup AI-Generated: Uses OpenAI ChatGPT Signed-off-by: Giancarlo Cicellyn Comneno <giancarlo.cicellyn@gmail.com> --- bin/bitbake-setup | 8 +++--- lib/bb/tests/setup.py | 57 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+), 4 deletions(-)