dev-manual/debugging: improve gdbserver debugfs docs

From: Chris Laplante <chris.laplante@agilent.com>

1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
2. Fix the 'cp -a packages-split/bash-dbg/* path/debugfs' step to not
   literally try to copy a file called '*'

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
---
 documentation/dev-manual/debugging.rst | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

Chris Laplante via lists.yoctoproject.org <chris.laplante=
agilent.com@lists.yoctoproject.org> schrieb am Mo., 21. Okt. 2024, 17:23:

> From: Chris Laplante <chris.laplante@agilent.com>
>
> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
> 2. Fix the 'cp -a packages-split/bash-dbg/* path/debugfs' step to not
>    literally try to copy a file called '*'
>
> Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
> ---
>  documentation/dev-manual/debugging.rst | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/documentation/dev-manual/debugging.rst
> b/documentation/dev-manual/debugging.rst
> index 92458a0c37..6b5d2dd820 100644
> --- a/documentation/dev-manual/debugging.rst
> +++ b/documentation/dev-manual/debugging.rst
> @@ -1006,8 +1006,9 @@ debugger.
>
>        IMAGE_GEN_DEBUGFS = "1"
>        IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
> +      IMAGE_FSTYPES:append = " tar.bz2"
>
> -   These options cause the
> +   The first two lines cause the
>     OpenEmbedded build system to generate a special companion filesystem
>     fragment, which contains the matching source and debug symbols to
>     your deployable filesystem. The build system does this by looking at
> @@ -1016,8 +1017,9 @@ debugger.
>
>     The companion debug filesystem is not a complete filesystem, but only
>     contains the debug fragments. This filesystem must be combined with
> -   the full filesystem for debugging. Subsequent steps in this procedure
> -   show how to combine the partial filesystem with the full filesystem.
> +   the full filesystem for debugging. The third line ensures that the
> +   build system generates a tar.bz2-formatted copy of the full filesystem,
> +   which will be used in subsequent steps in this procedure.
>

Would a link to
https://docs.yoctoproject.org/sdk-manual/extensible.html#devtool-ide-sdk-configures-ides-for-the-extensible-sdk
be helpful here?
This tool creates this setup automatically, even without creating a big tar
and extracting it again.

Regards
Adrian

>
>  #. *Configure the system to include gdbserver in the target filesystem:*
>
> @@ -1142,7 +1144,7 @@ debugger.
>        $ bitbake -c devshell bash
>        $ cd ..
>        $ scp packages-split/bash/bin/bash target:/bin/bash
> -      $ cp -a packages-split/bash-dbg/\* path/debugfs
> +      $ cp -a packages-split/bash-dbg/* path/debugfs
>
>  Debugging with the GNU Project Debugger (GDB) on the Target
>  ===========================================================
> --
> 2.43.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#5541):
> https://lists.yoctoproject.org/g/docs/message/5541
> Mute This Topic: https://lists.yoctoproject.org/mt/109134583/4454582
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> adrian.freihofer@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

Hi Chris,

On Mon Oct 21, 2024 at 5:22 PM CEST, Chris Laplante via lists.yoctoproject.org wrote:
> From: Chris Laplante <chris.laplante@agilent.com>
>
> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
> 2. Fix the 'cp -a packages-split/bash-dbg/* path/debugfs' step to not
>    literally try to copy a file called '*'
>
> Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
> ---
>  documentation/dev-manual/debugging.rst | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst
> index 92458a0c37..6b5d2dd820 100644
> --- a/documentation/dev-manual/debugging.rst
> +++ b/documentation/dev-manual/debugging.rst
> @@ -1006,8 +1006,9 @@ debugger.
>
>        IMAGE_GEN_DEBUGFS = "1"
>        IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
> +      IMAGE_FSTYPES:append = " tar.bz2"
>
> -   These options cause the
> +   The first two lines cause the
>     OpenEmbedded build system to generate a special companion filesystem
>     fragment, which contains the matching source and debug symbols to
>     your deployable filesystem. The build system does this by looking at
> @@ -1016,8 +1017,9 @@ debugger.
>
>     The companion debug filesystem is not a complete filesystem, but only
>     contains the debug fragments. This filesystem must be combined with
> -   the full filesystem for debugging. Subsequent steps in this procedure
> -   show how to combine the partial filesystem with the full filesystem.
> +   the full filesystem for debugging. The third line ensures that the
> +   build system generates a tar.bz2-formatted copy of the full filesystem,
> +   which will be used in subsequent steps in this procedure.
>
>  #. *Configure the system to include gdbserver in the target filesystem:*
>
> @@ -1142,7 +1144,7 @@ debugger.
>        $ bitbake -c devshell bash
>        $ cd ..
>        $ scp packages-split/bash/bin/bash target:/bin/bash
> -      $ cp -a packages-split/bash-dbg/\* path/debugfs
> +      $ cp -a packages-split/bash-dbg/* path/debugfs
>
>  Debugging with the GNU Project Debugger (GDB) on the Target
>  ===========================================================

Thanks, I verified and indeed, we do need this addition for subsequent steps.

Reviewed-by: Antonin Godard <antonin.godard@bootlin.com>

Cheers,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Adrian,

On Mon Oct 21, 2024 at 9:54 PM CEST, Adrian Freihofer wrote:
> Chris Laplante via lists.yoctoproject.org <chris.laplante=
> agilent.com@lists.yoctoproject.org> schrieb am Mo., 21. Okt. 2024, 17:23:
>
>> From: Chris Laplante <chris.laplante@agilent.com>
>>
>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
>> 2. Fix the 'cp -a packages-split/bash-dbg/* path/debugfs' step to not
>>    literally try to copy a file called '*'
>>
>> Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
>> ---
>>  documentation/dev-manual/debugging.rst | 10 ++++++----
>>  1 file changed, 6 insertions(+), 4 deletions(-)
>>
>> diff --git a/documentation/dev-manual/debugging.rst
>> b/documentation/dev-manual/debugging.rst
>> index 92458a0c37..6b5d2dd820 100644
>> --- a/documentation/dev-manual/debugging.rst
>> +++ b/documentation/dev-manual/debugging.rst
>> @@ -1006,8 +1006,9 @@ debugger.
>>
>>        IMAGE_GEN_DEBUGFS = "1"
>>        IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
>> +      IMAGE_FSTYPES:append = " tar.bz2"
>>
>> -   These options cause the
>> +   The first two lines cause the
>>     OpenEmbedded build system to generate a special companion filesystem
>>     fragment, which contains the matching source and debug symbols to
>>     your deployable filesystem. The build system does this by looking at
>> @@ -1016,8 +1017,9 @@ debugger.
>>
>>     The companion debug filesystem is not a complete filesystem, but only
>>     contains the debug fragments. This filesystem must be combined with
>> -   the full filesystem for debugging. Subsequent steps in this procedure
>> -   show how to combine the partial filesystem with the full filesystem.
>> +   the full filesystem for debugging. The third line ensures that the
>> +   build system generates a tar.bz2-formatted copy of the full filesystem,
>> +   which will be used in subsequent steps in this procedure.
>>
>
> Would a link to
> https://docs.yoctoproject.org/sdk-manual/extensible.html#devtool-ide-sdk-configures-ides-for-the-extensible-sdk
> be helpful here?
> This tool creates this setup automatically, even without creating a big tar
> and extracting it again.

Indeed, this would make sense I agree. I would perhaps add it after the sentence
a bit further up after "The following steps show you how to debug using the GNU
project debugger." as a `.. note::`. What do you think? Would you be able to add
that note?

Cheers,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Chris,

On 10/21/24 5:22 PM, Chris Laplante via lists.yoctoproject.org wrote:
> [You don't often get email from chris.laplante=agilent.com@lists.yoctoproject.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> From: Chris Laplante <chris.laplante@agilent.com>
> 
> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2

Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and 
IMAGE_GEN_DEBUGFS are set?

We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py that 
doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's fine?

meta/classes-recipe/image.bbclass and 
meta/classes-recipe/image_types.bbclass are the only places where 
IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're adding 
the content of IMAGE_FSTYPES_DEBUGFS to those of IMAGE_FSTYPES if I am 
not misreading it?

Cheers,
Quentin

> 2. Fix the 'cp -a packages-split/bash-dbg/* path/debugfs' step to not
>     literally try to copy a file called '*'
> 
> Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
> ---
>   documentation/dev-manual/debugging.rst | 10 ++++++----
>   1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst
> index 92458a0c37..6b5d2dd820 100644
> --- a/documentation/dev-manual/debugging.rst
> +++ b/documentation/dev-manual/debugging.rst
> @@ -1006,8 +1006,9 @@ debugger.
> 
>         IMAGE_GEN_DEBUGFS = "1"
>         IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
> +      IMAGE_FSTYPES:append = " tar.bz2"
> 
> -   These options cause the
> +   The first two lines cause the
>      OpenEmbedded build system to generate a special companion filesystem
>      fragment, which contains the matching source and debug symbols to
>      your deployable filesystem. The build system does this by looking at
> @@ -1016,8 +1017,9 @@ debugger.
> 
>      The companion debug filesystem is not a complete filesystem, but only
>      contains the debug fragments. This filesystem must be combined with
> -   the full filesystem for debugging. Subsequent steps in this procedure
> -   show how to combine the partial filesystem with the full filesystem.
> +   the full filesystem for debugging. The third line ensures that the
> +   build system generates a tar.bz2-formatted copy of the full filesystem,
> +   which will be used in subsequent steps in this procedure.
> 
>   #. *Configure the system to include gdbserver in the target filesystem:*
> 
> @@ -1142,7 +1144,7 @@ debugger.
>         $ bitbake -c devshell bash
>         $ cd ..
>         $ scp packages-split/bash/bin/bash target:/bin/bash
> -      $ cp -a packages-split/bash-dbg/\* path/debugfs
> +      $ cp -a packages-split/bash-dbg/* path/debugfs
> 
>   Debugging with the GNU Project Debugger (GDB) on the Target
>   ===========================================================
> --
> 2.43.0
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#5541): https://lists.yoctoproject.org/g/docs/message/5541
> Mute This Topic: https://lists.yoctoproject.org/mt/109134583/6293953
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [quentin.schulz@cherry.de]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Hi Quentin,

On Tue Oct 22, 2024 at 1:57 PM CEST, Quentin Schulz via lists.yoctoproject.org wrote:
>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
>
> Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and
> IMAGE_GEN_DEBUGFS are set?
>
> We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py that
> doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's fine?

I think it might be because qemu.inc adds it by default (not a oeqa expert
though :) ). Here's what I have by default with qemu:

```
$ bitbake-getvar IMAGE_FSTYPES -r core-image-minimal
#
# $IMAGE_FSTYPES [3 operations]
#   append /build/../work/openembedded-core/meta/conf/machine/include/qemu.inc:18
#     "tar.bz2 ext4"
#   set /build/../work/openembedded-core/meta/conf/documentation.conf:215
#     [doc] "Formats of root filesystem images that you want to have created."
#   set? /build/../work/openembedded-core/meta/conf/bitbake.conf:844
#     "tar.gz"
# pre-expansion value:
#   " tar.bz2 ext4"
IMAGE_FSTYPES=" tar.bz2 ext4"
```

> meta/classes-recipe/image.bbclass and
> meta/classes-recipe/image_types.bbclass are the only places where
> IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're adding
> the content of IMAGE_FSTYPES_DEBUGFS to those of IMAGE_FSTYPES if I am
> not misreading it?

In image_types, the following:

  fstypes |= set((d.getVar('IMAGE_FSTYPES_DEBUGFS') or "").split())

Is an union, so if tar.bz2 is not in IMAGE_FSTYPES it won't work actually.

About the other IMAGE_FSTYPES_DEBUGFS references, it does look like it should
add IMAGE_FSTYPES_DEBUGFS to IMAGE_FSTYPES, but it didn't when I tested it, so
I'm not entirely sure :/ Maybe this is a problem in oecore?

Cheers,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi Antonin,

On 10/22/24 2:10 PM, Antonin Godard wrote:
> Hi Quentin,
> 
> On Tue Oct 22, 2024 at 1:57 PM CEST, Quentin Schulz via lists.yoctoproject.org wrote:
>>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
>>
>> Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and
>> IMAGE_GEN_DEBUGFS are set?
>>
>> We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py that
>> doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's fine?
> 
> I think it might be because qemu.inc adds it by default (not a oeqa expert
> though :) ). Here's what I have by default with qemu:
> 
> ```
> $ bitbake-getvar IMAGE_FSTYPES -r core-image-minimal
> #
> # $IMAGE_FSTYPES [3 operations]
> #   append /build/../work/openembedded-core/meta/conf/machine/include/qemu.inc:18
> #     "tar.bz2 ext4"
> #   set /build/../work/openembedded-core/meta/conf/documentation.conf:215
> #     [doc] "Formats of root filesystem images that you want to have created."
> #   set? /build/../work/openembedded-core/meta/conf/bitbake.conf:844
> #     "tar.gz"
> # pre-expansion value:
> #   " tar.bz2 ext4"
> IMAGE_FSTYPES=" tar.bz2 ext4"
> ```
> 
>> meta/classes-recipe/image.bbclass and
>> meta/classes-recipe/image_types.bbclass are the only places where
>> IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're adding
>> the content of IMAGE_FSTYPES_DEBUGFS to those of IMAGE_FSTYPES if I am
>> not misreading it?
> 
> In image_types, the following:
> 
>    fstypes |= set((d.getVar('IMAGE_FSTYPES_DEBUGFS') or "").split())
> 
> Is an union, so if tar.bz2 is not in IMAGE_FSTYPES it won't work actually.
> 

It's a union, yes, but that's why it'll work :)

 >>> a = set([1, 2, 3])
 >>> b = set([3, 4, 5])
 >>> a | b
{1, 2, 3, 4, 5}
 >>> c = set([5, 6, 7])
 >>> c |= a
 >>> c
{1, 2, 3, 5, 6, 7}

c.f. https://docs.python.org/3/library/stdtypes.html#frozenset.union

> About the other IMAGE_FSTYPES_DEBUGFS references, it does look like it should
> add IMAGE_FSTYPES_DEBUGFS to IMAGE_FSTYPES, but it didn't when I tested it, so
> I'm not entirely sure :/ Maybe this is a problem in oecore?
> 

That's basically what i'm trying to figure out. Should this really be 
documented or is it rather a bug that we should fix (and could document 
for older releases)?

Cheers,
Quentin

On Tue Oct 22, 2024 at 2:17 PM CEST, Quentin Schulz wrote:
> Hi Antonin,
>
> On 10/22/24 2:10 PM, Antonin Godard wrote:
>> Hi Quentin,
>>
>> On Tue Oct 22, 2024 at 1:57 PM CEST, Quentin Schulz via lists.yoctoproject.org wrote:
>>>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
>>>
>>> Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and
>>> IMAGE_GEN_DEBUGFS are set?
>>>
>>> We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py that
>>> doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's fine?
>>
>> I think it might be because qemu.inc adds it by default (not a oeqa expert
>> though :) ). Here's what I have by default with qemu:
>>
>> ```
>> $ bitbake-getvar IMAGE_FSTYPES -r core-image-minimal
>> #
>> # $IMAGE_FSTYPES [3 operations]
>> #   append /build/../work/openembedded-core/meta/conf/machine/include/qemu.inc:18
>> #     "tar.bz2 ext4"
>> #   set /build/../work/openembedded-core/meta/conf/documentation.conf:215
>> #     [doc] "Formats of root filesystem images that you want to have created."
>> #   set? /build/../work/openembedded-core/meta/conf/bitbake.conf:844
>> #     "tar.gz"
>> # pre-expansion value:
>> #   " tar.bz2 ext4"
>> IMAGE_FSTYPES=" tar.bz2 ext4"
>> ```
>>
>>> meta/classes-recipe/image.bbclass and
>>> meta/classes-recipe/image_types.bbclass are the only places where
>>> IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're adding
>>> the content of IMAGE_FSTYPES_DEBUGFS to those of IMAGE_FSTYPES if I am
>>> not misreading it?
>>
>> In image_types, the following:
>>
>>    fstypes |= set((d.getVar('IMAGE_FSTYPES_DEBUGFS') or "").split())
>>
>> Is an union, so if tar.bz2 is not in IMAGE_FSTYPES it won't work actually.
>>
>
> It's a union, yes, but that's why it'll work :)
>
>  >>> a = set([1, 2, 3])
>  >>> b = set([3, 4, 5])
>  >>> a | b
> {1, 2, 3, 4, 5}
>  >>> c = set([5, 6, 7])
>  >>> c |= a
>  >>> c
> {1, 2, 3, 5, 6, 7}
>
> c.f. https://docs.python.org/3/library/stdtypes.html#frozenset.union

Indeed, my bad, read that too quickly. Thanks :)

Cheers,
Antonin

--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Hi all,

> Hi Antonin,
> 
> On 10/22/24 2:10 PM, Antonin Godard wrote:
> > Hi Quentin,
> >
> > On Tue Oct 22, 2024 at 1:57 PM CEST, Quentin Schulz via
> lists.yoctoproject.org wrote:
> >>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
> >>
> >> Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and
> >> IMAGE_GEN_DEBUGFS are set?
> >>
> >> We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py
> >> that doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's
> fine?
> >
> > I think it might be because qemu.inc adds it by default (not a oeqa
> > expert though :) ). Here's what I have by default with qemu:
> >
> > ```
> > $ bitbake-getvar IMAGE_FSTYPES -r core-image-minimal # #
> > $IMAGE_FSTYPES [3 operations]
> > #   append /build/../work/openembedded-
> core/meta/conf/machine/include/qemu.inc:18
> > #     "tar.bz2 ext4"
> > #   set /build/../work/openembedded-
> core/meta/conf/documentation.conf:215
> > #     [doc] "Formats of root filesystem images that you want to have
> created."
> > #   set? /build/../work/openembedded-core/meta/conf/bitbake.conf:844
> > #     "tar.gz"
> > # pre-expansion value:
> > #   " tar.bz2 ext4"
> > IMAGE_FSTYPES=" tar.bz2 ext4"
> > ```
> >
> >> meta/classes-recipe/image.bbclass and
> >> meta/classes-recipe/image_types.bbclass are the only places where
> >> IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're
> >> adding the content of IMAGE_FSTYPES_DEBUGFS to those of
> IMAGE_FSTYPES
> >> if I am not misreading it?
> >
> > In image_types, the following:
> >
> >    fstypes |= set((d.getVar('IMAGE_FSTYPES_DEBUGFS') or "").split())
> >
> > Is an union, so if tar.bz2 is not in IMAGE_FSTYPES it won't work actually.
> >
> 
> It's a union, yes, but that's why it'll work :)
> 
>  >>> a = set([1, 2, 3])
>  >>> b = set([3, 4, 5])
>  >>> a | b
> {1, 2, 3, 4, 5}
>  >>> c = set([5, 6, 7])
>  >>> c |= a
>  >>> c
> {1, 2, 3, 5, 6, 7}
> 
> c.f.
> https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs
> .python.org%2F3%2Flibrary%2Fstdtypes.html%23frozenset.union&data=05
> %7C02%7Cchris.laplante%40agilent.com%7Cbf8562bbfb1548b643be08dcf2
> 9393ec%7Ca9c0bc098b46420693512ba12fb4a5c0%7C0%7C0%7C6386519
> 62912305989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata
> =41m%2FAwTD9msyScku1ytng1IKYXPKbakuOSgpMFdLaUo%3D&reserved=0
> 
> > About the other IMAGE_FSTYPES_DEBUGFS references, it does look like it
> > should add IMAGE_FSTYPES_DEBUGFS to IMAGE_FSTYPES, but it didn't
> when
> > I tested it, so I'm not entirely sure :/ Maybe this is a problem in oecore?
> >
> 
> That's basically what i'm trying to figure out. Should this really be documented
> or is it rather a bug that we should fix (and could document for older
> releases)?

The imagetypes_getdepends function you are referring to (in meta/classes-recipe/image_types.bbclass) doesn't create any do_image_* tasks; it only determines image construction dependencies and generates a bunch of "<dep>:do_populate_sysroot"s.

The code for adding the tasks is in meta/classes-recipe/image.bbclass. I checked and the reason it works in selftest is indeed because of qemu.inc:

# $IMAGE_FSTYPES [3 operations]
#   append /home/laplante/repos/poky/meta/conf/machine/include/qemu.inc:18
#     "tar.bz2 ext4"
#   set /home/laplante/repos/poky/meta/conf/documentation.conf:215
#     [doc] "Formats of root filesystem images that you want to have created."
#   set? /home/laplante/repos/poky/meta/conf/bitbake.conf:846
#     "tar.gz"
# pre-expansion value:
#   " tar.bz2 ext4"
IMAGE_FSTYPES=" tar.bz2 ext4"


One simple solution (untested) could be:


diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
index 00f1d58f23..c7ad272339 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -400,6 +400,7 @@ python () {
         debugfs_fstypes = d.getVar('IMAGE_FSTYPES_DEBUGFS').split()
         for t in debugfs_fstypes:
             alltypes.append("debugfs_" + t)
+            alltypes.append(t)

     def _add_type(t):
         baset = _image_base_type(t)


I can test and send in that patch if desired. 


Thanks,
Chris

Hi Chris,

On 10/22/24 4:39 PM, chris.laplante@agilent.com wrote:
> Hi all,
> 
>> Hi Antonin,
>>
>> On 10/22/24 2:10 PM, Antonin Godard wrote:
>>> Hi Quentin,
>>>
>>> On Tue Oct 22, 2024 at 1:57 PM CEST, Quentin Schulz via
>> lists.yoctoproject.org wrote:
>>>>> 1. Make it clear that IMAGE_FSTYPES has to contain tar.bz2
>>>>
>>>> Yes, but isn't it done automatically if IMAGE_FSTYPES_DEBUGFS and
>>>> IMAGE_GEN_DEBUGFS are set?
>>>>
>>>> We have a selftest here: meta/lib/oeqa/selftest/cases/gdbserver.py
>>>> that doesn't seem to be adding tar.bz2 to IMAGE_FSTYPES, so I assume it's
>> fine?
>>>
>>> I think it might be because qemu.inc adds it by default (not a oeqa
>>> expert though :) ). Here's what I have by default with qemu:
>>>
>>> ```
>>> $ bitbake-getvar IMAGE_FSTYPES -r core-image-minimal # #
>>> $IMAGE_FSTYPES [3 operations]
>>> #   append /build/../work/openembedded-
>> core/meta/conf/machine/include/qemu.inc:18
>>> #     "tar.bz2 ext4"
>>> #   set /build/../work/openembedded-
>> core/meta/conf/documentation.conf:215
>>> #     [doc] "Formats of root filesystem images that you want to have
>> created."
>>> #   set? /build/../work/openembedded-core/meta/conf/bitbake.conf:844
>>> #     "tar.gz"
>>> # pre-expansion value:
>>> #   " tar.bz2 ext4"
>>> IMAGE_FSTYPES=" tar.bz2 ext4"
>>> ```
>>>
>>>> meta/classes-recipe/image.bbclass and
>>>> meta/classes-recipe/image_types.bbclass are the only places where
>>>> IMAGE_FSTYPES_DEBUGFS is actually used, and it seems that we're
>>>> adding the content of IMAGE_FSTYPES_DEBUGFS to those of
>> IMAGE_FSTYPES
>>>> if I am not misreading it?
>>>
>>> In image_types, the following:
>>>
>>>     fstypes |= set((d.getVar('IMAGE_FSTYPES_DEBUGFS') or "").split())
>>>
>>> Is an union, so if tar.bz2 is not in IMAGE_FSTYPES it won't work actually.
>>>
>>
>> It's a union, yes, but that's why it'll work :)
>>
>>   >>> a = set([1, 2, 3])
>>   >>> b = set([3, 4, 5])
>>   >>> a | b
>> {1, 2, 3, 4, 5}
>>   >>> c = set([5, 6, 7])
>>   >>> c |= a
>>   >>> c
>> {1, 2, 3, 5, 6, 7}
>>
>> c.f.
>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs%2F&data=05%7C02%7Cquentin.schulz%40cherry.de%7C24532c178926442d8e9908dcf2a74a1b%7C5e0e1b5221b54e7b83bb514ec460677e%7C0%7C0%7C638652047574497237%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=TmSlV%2BfHq995xV%2Fmbd4RA92UZ9o8KEkeIfh4%2FtJlgzA%3D&reserved=0
>> .python.org%2F3%2Flibrary%2Fstdtypes.html%23frozenset.union&data=05
>> %7C02%7Cchris.laplante%40agilent.com%7Cbf8562bbfb1548b643be08dcf2
>> 9393ec%7Ca9c0bc098b46420693512ba12fb4a5c0%7C0%7C0%7C6386519
>> 62912305989%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
>> JQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata
>> =41m%2FAwTD9msyScku1ytng1IKYXPKbakuOSgpMFdLaUo%3D&reserved=0
>>
>>> About the other IMAGE_FSTYPES_DEBUGFS references, it does look like it
>>> should add IMAGE_FSTYPES_DEBUGFS to IMAGE_FSTYPES, but it didn't
>> when
>>> I tested it, so I'm not entirely sure :/ Maybe this is a problem in oecore?
>>>
>>
>> That's basically what i'm trying to figure out. Should this really be documented
>> or is it rather a bug that we should fix (and could document for older
>> releases)?
> 
> The imagetypes_getdepends function you are referring to (in meta/classes-recipe/image_types.bbclass) doesn't create any do_image_* tasks; it only determines image construction dependencies and generates a bunch of "<dep>:do_populate_sysroot"s.
> 
> The code for adding the tasks is in meta/classes-recipe/image.bbclass. I checked and the reason it works in selftest is indeed because of qemu.inc:
> 
> # $IMAGE_FSTYPES [3 operations]
> #   append /home/laplante/repos/poky/meta/conf/machine/include/qemu.inc:18
> #     "tar.bz2 ext4"
> #   set /home/laplante/repos/poky/meta/conf/documentation.conf:215
> #     [doc] "Formats of root filesystem images that you want to have created."
> #   set? /home/laplante/repos/poky/meta/conf/bitbake.conf:846
> #     "tar.gz"
> # pre-expansion value:
> #   " tar.bz2 ext4"
> IMAGE_FSTYPES=" tar.bz2 ext4"
> 
> 
> One simple solution (untested) could be:
> 
> 
> diff --git a/meta/classes-recipe/image.bbclass b/meta/classes-recipe/image.bbclass
> index 00f1d58f23..c7ad272339 100644
> --- a/meta/classes-recipe/image.bbclass
> +++ b/meta/classes-recipe/image.bbclass
> @@ -400,6 +400,7 @@ python () {
>           debugfs_fstypes = d.getVar('IMAGE_FSTYPES_DEBUGFS').split()
>           for t in debugfs_fstypes:
>               alltypes.append("debugfs_" + t)
> +            alltypes.append(t)
> 
>       def _add_type(t):
>           baset = _image_base_type(t)
> 
> 
> I can test and send in that patch if desired.
> 

We need to send at least one patch.

Either this is the intended behavior, then I would recommend we make the 
selftest use a different fstypes than the one in default IMAGE_FSTYPES 
so that it's explicit what needs to be done.

Or we fix the missing fstype(s) from IMAGE_FSTYPES_DEBUGFS by adding 
them to IMAGE_FSTYPES automagically. I don't know if this is desired, 
Richard would probably know. If we go this route, I still believe we 
should change the fstype used in the selftest to use one that isn't in 
the default IMAGE_FSTYPES to make sure it is actually added by the logic.

So... don't know how to go forward with that :/

Cheers,
Quentin

Hi Quentin,

> > One simple solution (untested) could be:
> >
> >
> > diff --git a/meta/classes-recipe/image.bbclass
> > b/meta/classes-recipe/image.bbclass
> > index 00f1d58f23..c7ad272339 100644
> > --- a/meta/classes-recipe/image.bbclass
> > +++ b/meta/classes-recipe/image.bbclass
> > @@ -400,6 +400,7 @@ python () {
> >           debugfs_fstypes = d.getVar('IMAGE_FSTYPES_DEBUGFS').split()
> >           for t in debugfs_fstypes:
> >               alltypes.append("debugfs_" + t)
> > +            alltypes.append(t)
> >
> >       def _add_type(t):
> >           baset = _image_base_type(t)
> >
> >
> > I can test and send in that patch if desired.
> >
> 
> We need to send at least one patch.
> 
> Either this is the intended behavior, then I would recommend we make the
> selftest use a different fstypes than the one in default IMAGE_FSTYPES so that
> it's explicit what needs to be done.
> 
> Or we fix the missing fstype(s) from IMAGE_FSTYPES_DEBUGFS by adding
> them to IMAGE_FSTYPES automagically. I don't know if this is desired, Richard
> would probably know. If we go this route, I still believe we should change the
> fstype used in the selftest to use one that isn't in the default IMAGE_FSTYPES
> to make sure it is actually added by the logic.
> 
> So... don't know how to go forward with that :/

Personally, I think it should just automagically add the corresponding types to IMAGE_FSTYPES. Definitely agree with adding a test too. I'll work on a patch series.

Thanks,
Chris