diff mbox series

[2/2] oe-setup-vscode: improve VSCode bitbake-setup compatibility

Message ID 20251224150805.3312684-3-adrian.freihofer@siemens.com
State Changes Requested
Headers show
Series oe-setup-vscode: improve bitbake-setup compatibility | expand

Commit Message

AdrianF Dec. 24, 2025, 3:07 p.m. UTC 
From: Adrian Freihofer <adrian.freihofer@siemens.com>

VSCode is started in the workspace folder where the .vscode directory
is located. With the new bitbake-setup based build layout, this starts
VSCode in the openembedded-core directory, which means only files in
that layer are visible. This is not usable for most projects which
span multiple layers.

Enhance the VSCode setup to work with the new bitbake-setup layout by
moving the .vscode directory location from
     poky-master/layers/oe-init-build-env-dir/.vscode
to
     poky-master/.vscode
bringing all layers back into the scope of VSCode.

The oe-init-build-env script now detects the build layout by checking
for the presence of a layers directory, and adjusts the .vscode
location accordingly.

Additionally, the file exclusion patterns have been refined to improve
VSCode performance:
- Separated files.exclude from search.exclude patterns for better control
- Added exclusions for oe-init-build-env-dir and oe-scripts directories
- Prevented indexing of build artifacts and temporary files

The configuration now also detects whether to use oe-init-build-env or
init-build-env based on the detected build mode.

Usage example:
1. bitbake-setup creates a directory structure like:
     poky-master/
       layers/
         openembedded-core/
         meta-yourlayer/
       build/
         init-build-env

2. Create the .vscode directory in poky-master by running:
   . build/init-build-env

   The settings.json contains:
    "bitbake.pathToBitbakeFolder": ".../poky-master/layers/bitbake",
    "bitbake.pathToBuildFolder": "${workspaceFolder}/build",
    "bitbake.pathToEnvScript": "${workspaceFolder}/build/init-build-env",
    "bitbake.workingDirectory": "${workspaceFolder}",

3. Start VSCode in poky-master

Note: Starting VSCode before the .vscode directory exists may cause
VSCode to index the entire workspace, leading to high CPU and memory
usage. Future improvements could include creating a minimal .vscode
directory during bitbake-setup initialization.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 oe-init-build-env       | 16 ++++++++--
 scripts/oe-setup-vscode | 68 +++++++++++++++++++++++------------------
 2 files changed, 52 insertions(+), 32 deletions(-)

Comments

Alexander Kanavin Dec. 26, 2025, 8:46 p.m. UTC | #1 
On Wed, 24 Dec 2025 at 16:08, Adrian Freihofer via
lists.openembedded.org
<adrian.freihofer=siemens.com@lists.openembedded.org> wrote:
> VSCode is started in the workspace folder where the .vscode directory
> is located. With the new bitbake-setup based build layout, this starts
> VSCode in the openembedded-core directory, which means only files in
> that layer are visible. This is not usable for most projects which
> span multiple layers.
...
>  # Generate an initial configuration for VSCode and the yocto-bitbake plugin.
> -if command -v code > /dev/null && [ ! -d "$OEROOT/.vscode" ]; then
> -    oe-setup-vscode "$OEROOT" "$BUILDDIR"

I'm not sure I understand how 'seeing multiple layers' worked before,
or did it? If .vscode is placed into $OEROOT, then only $OEROOT is
visible, e.g. only the former integrated poky repo, but not other
layers.

Otherwise I guess this should be held until we figure out the changes
to bitbake-setup?

Alex
Adrian Freihofer Dec. 27, 2025, 12:06 p.m. UTC | #2 
Hi Alex

On Fri, 2025-12-26 at 21:46 +0100, Alexander Kanavin via
lists.openembedded.org wrote:
> On Wed, 24 Dec 2025 at 16:08, Adrian Freihofer via
> lists.openembedded.org
> <adrian.freihofer=siemens.com@lists.openembedded.org> wrote:
> > VSCode is started in the workspace folder where the .vscode
> > directory
> > is located. With the new bitbake-setup based build layout, this
> > starts
> > VSCode in the openembedded-core directory, which means only files
> > in
> > that layer are visible. This is not usable for most projects which
> > span multiple layers.
> ...
> >  # Generate an initial configuration for VSCode and the yocto-
> > bitbake plugin.
> > -if command -v code > /dev/null && [ ! -d "$OEROOT/.vscode" ]; then
> > -    oe-setup-vscode "$OEROOT" "$BUILDDIR"
> 
> I'm not sure I understand how 'seeing multiple layers' worked before,
> or did it? If .vscode is placed into $OEROOT, then only $OEROOT is
> visible, e.g. only the former integrated poky repo, but not other
> layers.
> 
> Otherwise I guess this should be held until we figure out the changes
> to bitbake-setup?
> 

My goal is to support making bitbake-setup a first-class user
experience while still preserving the simple, good old way without it.
My understanding of the old way is:

git clone poky && cd poky
. oe-init-build-env

We (and many others) use a slightly more advanced but still static
directory structure based on this basic concept. The following example
is very close to what running bitbake-setup init twice (once for poky-
master and once for oe-nodistro-master) would create:

build
poky-master
conf
site.conf
bblayers.conf
templateconf.cfg
oe-nodistro-master
conf
site.conf
bblayers.conf
templateconf.cfg
bitbake
meta-foo
scripts
oe-setup-vscode
meta-yocto
openembedded-core
scripts
oe-buildenv-internal
oe-setup-vscode
oe-init-build-env
oe-init-build-env

This can be used like this:

git clone --recurse-submodules foo-project ... && cd foo-project
. oe-init-build-env build/poky-master

Switching to oe-nodistro-master is as simple as starting a new shell
and running:

. oe-init-build-env build/oe-nodistro-master

The oe-init-build-env in the top level is a slightly customized copy of
poky/oe-init-build-env. It can, for example, modify the PATH variable
to prioritize scripts found in meta-foo/scripts over scripts found in
openembedded-core/scripts. This allows overriding scripts like we have
it for all the bitbake-related files in layers.

The logic: Set up the build environment with PATH (so bitbake can be
found), OEROOT, and BUILDDIR, then run further scripts which use these
variables.

I hope this explains:
 * How 'seeing multiple layers' worked before (my understanding)
 * Why I ended up with calling init-build-env (respectively the old oe-
   init-build-env) rather then calling separate scripts from bitbake-
   setup:
    - Keep it independent from bitbake-setup (at least for now)
    - Benefit from bitbake-setup: It has to run before everything else
      runs which is the right time for dropping a VSCode configuration.
    - Support customizing the .vscode/settings.json for project specific
      layers

I'm not against creating the VSCode configuration directly from
bitbake-setup. But I think it is more complicated than it looks like. I
also don't want to be the guy who polluted all the git repositories
whit editor specific code :-). OE-core is enough for now.

I hope this at least explains, why I ended up whit these patches.

Thank you.
Regards,
Adrian


> Alex
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#228551):
> https://lists.openembedded.org/g/openembedded-core/message/228551
> Mute This Topic: https://lists.openembedded.org/mt/116929993/4454582
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub [
> adrian.freihofer@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
Alexander Kanavin Dec. 27, 2025, 6:02 p.m. UTC | #3 
On Sat, 27 Dec 2025 at 13:06, <adrian.freihofer@gmail.com> wrote:
> My goal is to support making bitbake-setup a first-class user
> experience while still preserving the simple, good old way without it.
> My understanding of the old way is:
>
> git clone poky && cd poky
> . oe-init-build-env

Poky is no longer being updated, so this old way is gone and we don't
need to worry about it. What still exists is checking out the separate
repositories it was made of, usually into the same directory, but not
always, together with any additional layers or other repositories
needed in a given build. So you need to know where that top directory
is, and I would rather avoid guessing that from oe-init-build-env, or
oe-setup-vscode, and instead pass it in explicitly as a command line
parameter, which means running oe-setup-vscode from bitbake-setup
directly.

> We (and many others) use a slightly more advanced but still static
> directory structure based on this basic concept. The following example
> is very close to what running bitbake-setup init twice (once for poky-
> master and once for oe-nodistro-master) would create:
>
> build
> poky-master
> conf
> site.conf
> bblayers.conf
> templateconf.cfg
> oe-nodistro-master
> conf
> site.conf
> bblayers.conf
> templateconf.cfg
> bitbake
> meta-foo
> scripts
> oe-setup-vscode
> meta-yocto
> openembedded-core
> scripts
> oe-buildenv-internal
> oe-setup-vscode
> oe-init-build-env
> oe-init-build-env
>
> This can be used like this:
>
> git clone --recurse-submodules foo-project ... && cd foo-project
> . oe-init-build-env build/poky-master
>
> Switching to oe-nodistro-master is as simple as starting a new shell
> and running:
>
> . oe-init-build-env build/oe-nodistro-master
>
> The oe-init-build-env in the top level is a slightly customized copy of
> poky/oe-init-build-env. It can, for example, modify the PATH variable
> to prioritize scripts found in meta-foo/scripts over scripts found in
> openembedded-core/scripts. This allows overriding scripts like we have
> it for all the bitbake-related files in layers.

Sadly the formatting/indentation in the directory list above was lost,
so I can't quite make sense of the above.

> The logic: Set up the build environment with PATH (so bitbake can be
> found), OEROOT, and BUILDDIR, then run further scripts which use these
> variables.
>
> I hope this explains:
>  * How 'seeing multiple layers' worked before (my understanding)
>  * Why I ended up with calling init-build-env (respectively the old oe-
>    init-build-env) rather then calling separate scripts from bitbake-
>    setup:
>     - Keep it independent from bitbake-setup (at least for now)
>     - Benefit from bitbake-setup: It has to run before everything else
>       runs which is the right time for dropping a VSCode configuration.
>     - Support customizing the .vscode/settings.json for project specific
>       layers
>
> I'm not against creating the VSCode configuration directly from
> bitbake-setup. But I think it is more complicated than it looks like. I
> also don't want to be the guy who polluted all the git repositories
> whit editor specific code :-). OE-core is enough for now.

What I am proposing is this:
1. oe-setup-vscode does not try to guess things, or deduce things from
$PATH or other environment variables. It takes everything needed to
create vscode-specific files on its command line, as parameters. That
includes where bitbake is, if vscode has to know that, and where build
environment script is as well (e.g. the full path with a name).
2. How about rewriting oe-setup-vscode in python? Does it have to be shell?
3. bitbake-setup runs oe-setup-vscode directly, passing in all the
needed parameters (it knows all of them)
4. standard oe-init-build-env from oe-core either does not run
oe-setup-vscode at all anymore, or perhaps there could be some
minimally useful setup it could still create, but I can't see how you
would deduce the top dir with all the layers and the build dir easily
and correctly in the general case. What it does now (passing $OEROOT,
e.g. just oe-core location, to oe-setup-vscode and have vscode config
created there) is not really useful (as you have explained), or is it?

Alex
diff mbox series

Patch

diff --git a/oe-init-build-env b/oe-init-build-env
index 82382f2707..6547cdd297 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -49,8 +49,20 @@  export OEROOT
 }
 
 # Generate an initial configuration for VSCode and the yocto-bitbake plugin.
-if command -v code > /dev/null && [ ! -d "$OEROOT/.vscode" ]; then
-    oe-setup-vscode "$OEROOT" "$BUILDDIR"
+if command -v code > /dev/null ; then
+    # bitbake-setup mode
+    if [ -d "$BUILDDIR/../layers" ]; then
+        echo "bitbake-setup mode detected for VSCode setup."
+        if [ ! -d "$BUILDDIR/../.vscode" ]; then
+            oe-setup-vscode "$BUILDDIR/../" "$BUILDDIR"
+        fi
+    # poky mode
+    else
+        echo "poky mode detected for VSCode setup."
+        if [ ! -d "$OEROOT/.vscode" ]; then
+            oe-setup-vscode "$OEROOT" "$BUILDDIR"
+        fi
+    fi
 fi
 
 unset OEROOT
diff --git a/scripts/oe-setup-vscode b/scripts/oe-setup-vscode
index 6818641d42..0fea1c4808 100755
--- a/scripts/oe-setup-vscode
+++ b/scripts/oe-setup-vscode
@@ -20,47 +20,52 @@  if [ ! -d "$OEINIT" ] || [ ! -d "$BUILDDIR" ]; then
     exit 1
 fi
 
-VSCODE_SETTINGS=$VSCODEDIR/settings.json
+mkdir -p "$VSCODEDIR"
+VSCODE_SETTINGS_TMP=$(mktemp -p "$VSCODEDIR")
 ws_builddir="$(echo "$BUILDDIR" | sed -e "s|$OEINIT|\${workspaceFolder}|g")"
+
+# poky mode or bitbake-setup mode
+ws_build_env="\${workspaceFolder}/oe-init-build-env"
+if [ -f "$BUILDDIR/init-build-env" ]; then
+    ws_build_env="${ws_builddir}/init-build-env"
+fi
+
 bb_dir="$(dirname "$(which bitbake)" | sed 's|/bin$||')"
 
+files_exclude="\"**/_build/**\": true,
+        \"**/.git/**\": true,
+        \"**/cache/**\": true,
+        \"**/downloads/**\": true,
+        \"**/node_modules/**\": true,
+        \"**/oe-init-build-env-dir/**\": true,
+        \"**/oe-scripts/**\": true,
+        \"**/sstate-cache/**\": true,
+        \"**/workspace/attic/**\": true"
+
+search_exclude="\"**/buildhistory/**\": true,
+        \"**/oe-logs/**\": true,
+        \"**/oe-workdir/**\": true,
+        \"**/tmp*/**\": true,
+        \"**/workspace/sources/**\": true"
+
 # If BUILDDIR is in scope of VSCode ensure VSCode does not try to index the build folder.
 # This would lead to a busy CPU and finally to an OOM exception.
-mkdir -p "$VSCODEDIR"
-cat <<EOMsettings > "$VSCODE_SETTINGS"
+cat <<EOMsettings > "$VSCODE_SETTINGS_TMP"
 {
     "bitbake.pathToBitbakeFolder": "$bb_dir",
-    "bitbake.pathToEnvScript": "\${workspaceFolder}/oe-init-build-env",
     "bitbake.pathToBuildFolder": "$ws_builddir",
-    "bitbake.commandWrapper": "",
+    "bitbake.pathToEnvScript": "$ws_build_env",
     "bitbake.workingDirectory": "\${workspaceFolder}",
     "files.exclude": {
-        "**/.git/**": true,
-        "**/_build/**": true,
-        "**/buildhistory/**": true,
-        "**/cache/**": true,
-        "**/downloads/**": true,
-        "**/node_modules/**": true,
-        "**/oe-logs/**": true,
-        "**/oe-workdir/**": true,
-        "**/sstate-cache/**": true,
-        "**/tmp*/**": true,
-        "**/workspace/attic/**": true,
-        "**/workspace/sources/**": true
+        $files_exclude
+    },
+    "search.exclude": {
+        $files_exclude,
+        $search_exclude
     },
     "files.watcherExclude": {
-        "**/.git/**": true,
-        "**/_build/**": true,
-        "**/buildhistory/**": true,
-        "**/cache/**": true,
-        "**/downloads/**": true,
-        "**/node_modules/**": true,
-        "**/oe-logs/**": true,
-        "**/oe-workdir/**": true,
-        "**/sstate-cache/**": true,
-        "**/tmp*/**": true,
-        "**/workspace/attic/**": true,
-        "**/workspace/sources/**": true
+        $files_exclude,
+        $search_exclude
     },
     "python.analysis.exclude": [
         "**/_build/**",
@@ -74,11 +79,14 @@  cat <<EOMsettings > "$VSCODE_SETTINGS"
         "**/sstate-cache/**",
         "**/tmp*/**",
         "**/workspace/attic/**",
-        "**/workspace/sources/**"
+        "**/workspace/sources/**",
+        "**/oe-init-build-env-dir/**",
+        "**/oe-scripts/**"
     ]
 }
 EOMsettings
 
+mv -f "$VSCODE_SETTINGS_TMP" "$VSCODEDIR/settings.json"
 
 # Ask the user if the yocto-bitbake extension should be installed
 VSCODE_EXTENSIONS=$VSCODEDIR/extensions.json