[07/47] gawk: upgrade 5.3.0 -> 5.3.1

From: Wang Mingyu <wangmy@fujitsu.com>

0001-m4-readline-add-missing-includes.patch
removed since it's included in 5.3.1

Changelog:
===========
- More subtle issues related to uninitialized array elements have
  been fixed.
- A number of bugs in the debugger related to handling of arrays
  have been fixed.
- Some subtle bugs in the API have been fixed.
- Use of MPFR is now possible again on 32-bit Power PC Mac systems.
- Race conditions around broken pipes for system() and read and write
  pipes should now be closed off.
- Support for OSF/1 has been removed.
- The never-documented --nostalgia option has been removed. It was
  causing bug reports.
- The implementation of printf/sprintf has been thoroughly reworked
  in order to make the code more maintainable and to fix a goodly
  number of corner cases.
- As usual, there have been several minor code cleanups and bug fixes.

Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>
---
 ...001-m4-readline-add-missing-includes.patch | 38 -------------------
 .../gawk/{gawk_5.3.0.bb => gawk_5.3.1.bb}     |  3 +-
 2 files changed, 1 insertion(+), 40 deletions(-)
 delete mode 100644 meta/recipes-extended/gawk/gawk/0001-m4-readline-add-missing-includes.patch
 rename meta/recipes-extended/gawk/{gawk_5.3.0.bb => gawk_5.3.1.bb} (95%)

On Tue, Nov 12, 2024 at 05:24:03PM +0800, wangmy via lists.openembedded.org wrote:
> From: Wang Mingyu <wangmy@fujitsu.com>
> 
> 0001-m4-readline-add-missing-includes.patch
> removed since it's included in 5.3.1
> 
> Changelog:
> ===========
> - More subtle issues related to uninitialized array elements have
>   been fixed.
> - A number of bugs in the debugger related to handling of arrays
>   have been fixed.
> - Some subtle bugs in the API have been fixed.
> - Use of MPFR is now possible again on 32-bit Power PC Mac systems.
> - Race conditions around broken pipes for system() and read and write
>   pipes should now be closed off.
> - Support for OSF/1 has been removed.
> - The never-documented --nostalgia option has been removed. It was
>   causing bug reports.
> - The implementation of printf/sprintf has been thoroughly reworked
>   in order to make the code more maintainable and to fix a goodly
>   number of corner cases.
> - As usual, there have been several minor code cleanups and bug fixes.
> 

It looks like this patch is breaking some ptests on the autobuilder:

Failed ptests:
{'gawk': ['printf-corners']}

https://valkyrie.yoctoproject.org/#/builders/61/builds/388/steps/12/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/73/builds/390/steps/12/logs/stdio

Can you have a look at this please?

On Mon, Nov 18, 2024 at 01:36:47PM +0000, Mingyu Wang (Fujitsu) wrote:
> I didn't find ptest error of gawk from the log.
> I did the ptest of gawk by running ptest-runner and there's no error occurred.
> Can you tell me how to reproduce the error.
> 

Ok, reproducing the ptest build locally is indeed not straightforward. I
used the following procedure to reproduce something close to the setup we have
on the autobuilder, and I did reproduce the ptest failure.

First, if you want to use the exact same branch, you can take
mathieu/master-next-e78c1f31b592 from
git://git.yoctoproject.org/poky-contrib.

Then, I created a new empty configuration and added entries in
local.conf, to correspond roughly to what is described in
https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py. There is
probably a lot of things not relevant here, but it was good enough to
build and reproduce locally.

rm -rf build
source poky/oe-init-build-env
QEMU_USE_KVM = 'True'
INHERIT += 'report-error'
PREMIRRORS = ''
BB_GENERATE_MIRROR_TARBALLS = '1'
BB_NUMBER_THREADS = '16'
BB_NUMBER_PARSE_THREADS = '16'
PARALLEL_MAKE = '-j 16 -l 75'
BB_PRESSURE_MAX_CPU = '20000'
BB_PRESSURE_MAX_IO = '20000'
XZ_MEMLIMIT = '5%'
XZ_THREADS = '8'
ZSTD_THREADS = '8'
BB_TASK_NICE_LEVEL = '5'
BB_TASK_NICE_LEVEL:task-testimage = '0'
BB_TASK_IONICE_LEVEL = '2.7'
BB_TASK_IONICE_LEVEL:task-testimage = '2.1'
IMAGE_CLASSES += 'testimage'
TEST_QEMUBOOT_TIMEOUT = '1500'
SANITY_TESTED_DISTROS = ''
SDK_EXT_TYPE = 'minimal'
SDK_INCLUDE_TOOLCHAIN = '1'
ESDK_LOCALCONF_REMOVE:append = 'BB_HASHSERVE'
BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G
STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,30K
HALT,${TMPDIR},100M,1K HALT,${DL_DIR},100M HALT,${SSTATE_DIR},100M
HALT,/tmp,10M,1K'
BB_HEARTBEAT_EVENT = '60'
BB_LOG_HOST_STAT_ON_INTERVAL = '1'
BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 3'
BB_LOG_HOST_STAT_ON_FAILURE = '1'
BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l'
SDK_TOOLCHAIN_LANGS += 'rust'
BB_SERVER_TIMEOUT = '60'
BB_LOADFACTOR_MAX = '1.5'

TEST_SUITES = 'ping ssh ptest'
INHERIT += 'image-buildinfo'
IMAGE_BUILDINFO_VARS:append = ' IMAGE_BASENAME IMAGE_NAME'
EOF

Then built my image and ran ptests. The runqemu-gen-tapdevs might or
might not be needed, more details in
documentation/dev-manual/runtime-testing.rst.

bitbake core-image-ptest-gawk
sudo ${BUILDDIR}/../scripts/runqemu-gen-tapdevs 1000 4
bitbake core-image-ptest-gawk:do_testimage

And I got this output:

WARNING: core-image-ptest-gawk-1.0-r0 do_testimage: There were failing ptests.
Traceback (most recent call last):
  File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
    self.do_ptestrunner()
  File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/runtime/cases/ptest.py", line 120, in do_ptestrunner
    self.fail(failmsg)
AssertionError: 
Failed ptests:
{'gawk': ['printf-corners']}

I'd also note that the failure log shows where you can find the
detailed testimage log:
http://valkyrie.yocto.io/pub/non-release/20241115-76/testresults/qemux86-64-ptest/core-image-ptest-gawk/

There's a bit more detail there about the failure (and some strange
binary dump from tar that shouldn't be there).

Alex

On Mon, 18 Nov 2024 at 17:21, Mathieu Dubois-Briand via
lists.openembedded.org
<mathieu.dubois-briand=bootlin.com@lists.openembedded.org> wrote:
>
> On Mon, Nov 18, 2024 at 01:36:47PM +0000, Mingyu Wang (Fujitsu) wrote:
> > I didn't find ptest error of gawk from the log.
> > I did the ptest of gawk by running ptest-runner and there's no error occurred.
> > Can you tell me how to reproduce the error.
> >
>
> Ok, reproducing the ptest build locally is indeed not straightforward. I
> used the following procedure to reproduce something close to the setup we have
> on the autobuilder, and I did reproduce the ptest failure.
>
> First, if you want to use the exact same branch, you can take
> mathieu/master-next-e78c1f31b592 from
> git://git.yoctoproject.org/poky-contrib.
>
> Then, I created a new empty configuration and added entries in
> local.conf, to correspond roughly to what is described in
> https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py. There is
> probably a lot of things not relevant here, but it was good enough to
> build and reproduce locally.
>
> rm -rf build
> source poky/oe-init-build-env
> QEMU_USE_KVM = 'True'
> INHERIT += 'report-error'
> PREMIRRORS = ''
> BB_GENERATE_MIRROR_TARBALLS = '1'
> BB_NUMBER_THREADS = '16'
> BB_NUMBER_PARSE_THREADS = '16'
> PARALLEL_MAKE = '-j 16 -l 75'
> BB_PRESSURE_MAX_CPU = '20000'
> BB_PRESSURE_MAX_IO = '20000'
> XZ_MEMLIMIT = '5%'
> XZ_THREADS = '8'
> ZSTD_THREADS = '8'
> BB_TASK_NICE_LEVEL = '5'
> BB_TASK_NICE_LEVEL:task-testimage = '0'
> BB_TASK_IONICE_LEVEL = '2.7'
> BB_TASK_IONICE_LEVEL:task-testimage = '2.1'
> IMAGE_CLASSES += 'testimage'
> TEST_QEMUBOOT_TIMEOUT = '1500'
> SANITY_TESTED_DISTROS = ''
> SDK_EXT_TYPE = 'minimal'
> SDK_INCLUDE_TOOLCHAIN = '1'
> ESDK_LOCALCONF_REMOVE:append = 'BB_HASHSERVE'
> BB_DISKMON_DIRS = 'STOPTASKS,${TMPDIR},1G,100K STOPTASKS,${DL_DIR},1G
> STOPTASKS,${SSTATE_DIR},1G STOPTASKS,/tmp,100M,30K
> HALT,${TMPDIR},100M,1K HALT,${DL_DIR},100M HALT,${SSTATE_DIR},100M
> HALT,/tmp,10M,1K'
> BB_HEARTBEAT_EVENT = '60'
> BB_LOG_HOST_STAT_ON_INTERVAL = '1'
> BB_LOG_HOST_STAT_CMDS_INTERVAL = 'oe-time-dd-test.sh -c 100 -t 3'
> BB_LOG_HOST_STAT_ON_FAILURE = '1'
> BB_LOG_HOST_STAT_CMDS_FAILURE = 'oe-time-dd-test.sh -l'
> SDK_TOOLCHAIN_LANGS += 'rust'
> BB_SERVER_TIMEOUT = '60'
> BB_LOADFACTOR_MAX = '1.5'
>
> TEST_SUITES = 'ping ssh ptest'
> INHERIT += 'image-buildinfo'
> IMAGE_BUILDINFO_VARS:append = ' IMAGE_BASENAME IMAGE_NAME'
> EOF
>
> Then built my image and ran ptests. The runqemu-gen-tapdevs might or
> might not be needed, more details in
> documentation/dev-manual/runtime-testing.rst.
>
> bitbake core-image-ptest-gawk
> sudo ${BUILDDIR}/../scripts/runqemu-gen-tapdevs 1000 4
> bitbake core-image-ptest-gawk:do_testimage
>
> And I got this output:
>
> WARNING: core-image-ptest-gawk-1.0-r0 do_testimage: There were failing ptests.
> Traceback (most recent call last):
>   File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
>     return func(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^
>   File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
>     return func(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^
>   File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
>     return func(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^
>   File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/runtime/cases/ptest.py", line 27, in test_ptestrunner_expectfail
>     self.do_ptestrunner()
>   File "/home/mathieu/projects/swat/builds/poky-contrib/meta/lib/oeqa/runtime/cases/ptest.py", line 120, in do_ptestrunner
>     self.fail(failmsg)
> AssertionError:
> Failed ptests:
> {'gawk': ['printf-corners']}
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#207221): https://lists.openembedded.org/g/openembedded-core/message/207221
> Mute This Topic: https://lists.openembedded.org/mt/109531966/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>