diff mbox series

[v3,01/13] useradd_base.bbclass: do not use awk

Message ID 20260126073809.468495-2-adrian.freihofer@siemens.com
State New
Headers show
Series IDE SDK Improvements | expand

Commit Message

AdrianF Jan. 26, 2026, 7:37 a.m. UTC 
From: Adrian Freihofer <adrian.freihofer@siemens.com>

The exception bellow occurred during
  bitbake build-sysroots:do_build_target_sysroot
Re-trying the same command again can "solve" the problem, which indicates
that the problem is transient.
Transient probably means that libncurses.so.5 was not available at the
time when the useradd command was executed inside the sysroot population.
However, adding gawk-native to useradd.bbclass
DEPENDS:append:class-target does not help.

As a workaround this avoids using awk in useradd_base.bbclass.

ERROR: build-sysroots-1.0-r0 do_build_target_sysroot:
  Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_build_target_sysroot(d)
     0003:
File: '...poky-master/layers/openembedded-core/meta/recipes-core/meta/build-sysroots.bb',
      lineno: 46, function: do_build_target_sysroot
     0042:    targetsysroot = d.getVar("STANDALONE_SYSROOT")
     0043:    nativesysroot = d.getVar("STANDALONE_SYSROOT_NATIVE")
     0044:    import os
     0045:    os.environ['PATH'] = "%s/bin:%s/usr/bin:%s" %
	          (nativesysroot, nativesysroot, os.environ['PATH'])
 *** 0046:    staging_populate_sysroot_dir(targetsysroot, nativesysroot, False, d)
     0047:}
     0048:do_build_target_sysroot[cleandirs] = "${STANDALONE_SYSROOT}"
     0049:do_build_target_sysroot[nostamp] = "1"
     0050:addtask do_build_target_sysroot
File: '...poky-master/layers/openembedded-core/meta/classes-global/staging.bbclass',
      lineno: 249, function: staging_populate_sysroot_dir
     0245:                        continue
     0246:
     0247:    staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
     0248:    for p in sorted(postinsts):
 *** 0249:        bb.note("Running postinst {}, output:\n{}".format(
                      p, subprocess.check_output(p, shell=True,
                      stderr=subprocess.STDOUT)))
     0250:
     0251:#
     0252:# Manifests here are complicated. The main sysroot area has the unpacked sstate
     0253:# which us unrelocated and tracked by the main sstate manifests. Each recipe
File: '/usr/lib64/python3.13/subprocess.py', lineno: 472, function: check_output
     0468:        else:
     0469:            empty = b''
     0470:        kwargs['input'] = empty
     0471:
 *** 0472:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     0473:               **kwargs).stdout
     0474:
     0475:
     0476:class CompletedProcess(object):
File: '/usr/lib64/python3.13/subprocess.py', lineno: 577, function: run
     0573:            # We don't call process.wait() as .__exit__ does that for us.
     0574:            raise
     0575:        retcode = process.poll()
     0576:        if check and retcode:
 *** 0577:            raise CalledProcessError(retcode, process.args,
     0578:                                     output=stdout, stderr=stderr)
     0579:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0580:
     0581:
Exception: subprocess.CalledProcessError:
    Command '...poky-master/build/tmp/sysroots/qemux86-64/usr/bin/postinst-useradd-01group-cmake-example'
             returned non-zero exit status 1.

Subprocess output:
...poky-master/build/tmp/sysroots/x86_64/usr/sbin/useradd
Running groupadd commands...
NOTE: cmake-example: Performing groupadd with
  [--root ...poky-master/build/tmp/sysroots/qemux86-64 --system cmake-example]
awk: error while loading shared libraries: libncurses.so.5:
     cannot open shared object file: No such file or directory
groupadd: group 'cmake-example' already exists
ERROR: cmake-example: groupadd command did not succeed.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes/useradd_base.bbclass | 49 +++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 13 deletions(-)

Comments

ChenQi Jan. 26, 2026, 8:18 a.m. UTC | #1 
I have concern on this patch.

You're modifying a basic bbclass to fix one case without finding out the 
root cause.
By doing this, you're putting unnecessary restriction on the use of 
commands in this bbclass. Other commands with similar dependencies could 
potentially encounter the same issue in the future.
I think if you find out the root cause, you'll come up with a better 
fix. This would not only resolve the current case but also prevent 
similar issues.

Regards,
Qi


On 1/26/26 15:37, Adrian Freihofer via lists.openembedded.org wrote:
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
>
> The exception bellow occurred during
>    bitbake build-sysroots:do_build_target_sysroot
> Re-trying the same command again can "solve" the problem, which indicates
> that the problem is transient.
> Transient probably means that libncurses.so.5 was not available at the
> time when the useradd command was executed inside the sysroot population.
> However, adding gawk-native to useradd.bbclass
> DEPENDS:append:class-target does not help.
>
> As a workaround this avoids using awk in useradd_base.bbclass.
>
> ERROR: build-sysroots-1.0-r0 do_build_target_sysroot:
>    Error executing a python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>       0001:
>   *** 0002:do_build_target_sysroot(d)
>       0003:
> File: '...poky-master/layers/openembedded-core/meta/recipes-core/meta/build-sysroots.bb',
>        lineno: 46, function: do_build_target_sysroot
>       0042:    targetsysroot = d.getVar("STANDALONE_SYSROOT")
>       0043:    nativesysroot = d.getVar("STANDALONE_SYSROOT_NATIVE")
>       0044:    import os
>       0045:    os.environ['PATH'] = "%s/bin:%s/usr/bin:%s" %
> 	          (nativesysroot, nativesysroot, os.environ['PATH'])
>   *** 0046:    staging_populate_sysroot_dir(targetsysroot, nativesysroot, False, d)
>       0047:}
>       0048:do_build_target_sysroot[cleandirs] = "${STANDALONE_SYSROOT}"
>       0049:do_build_target_sysroot[nostamp] = "1"
>       0050:addtask do_build_target_sysroot
> File: '...poky-master/layers/openembedded-core/meta/classes-global/staging.bbclass',
>        lineno: 249, function: staging_populate_sysroot_dir
>       0245:                        continue
>       0246:
>       0247:    staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
>       0248:    for p in sorted(postinsts):
>   *** 0249:        bb.note("Running postinst {}, output:\n{}".format(
>                        p, subprocess.check_output(p, shell=True,
>                        stderr=subprocess.STDOUT)))
>       0250:
>       0251:#
>       0252:# Manifests here are complicated. The main sysroot area has the unpacked sstate
>       0253:# which us unrelocated and tracked by the main sstate manifests. Each recipe
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 472, function: check_output
>       0468:        else:
>       0469:            empty = b''
>       0470:        kwargs['input'] = empty
>       0471:
>   *** 0472:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>       0473:               **kwargs).stdout
>       0474:
>       0475:
>       0476:class CompletedProcess(object):
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 577, function: run
>       0573:            # We don't call process.wait() as .__exit__ does that for us.
>       0574:            raise
>       0575:        retcode = process.poll()
>       0576:        if check and retcode:
>   *** 0577:            raise CalledProcessError(retcode, process.args,
>       0578:                                     output=stdout, stderr=stderr)
>       0579:    return CompletedProcess(process.args, retcode, stdout, stderr)
>       0580:
>       0581:
> Exception: subprocess.CalledProcessError:
>      Command '...poky-master/build/tmp/sysroots/qemux86-64/usr/bin/postinst-useradd-01group-cmake-example'
>               returned non-zero exit status 1.
>
> Subprocess output:
> ...poky-master/build/tmp/sysroots/x86_64/usr/sbin/useradd
> Running groupadd commands...
> NOTE: cmake-example: Performing groupadd with
>    [--root ...poky-master/build/tmp/sysroots/qemux86-64 --system cmake-example]
> awk: error while loading shared libraries: libncurses.so.5:
>       cannot open shared object file: No such file or directory
> groupadd: group 'cmake-example' already exists
> ERROR: cmake-example: groupadd command did not succeed.
>
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
>   meta/classes/useradd_base.bbclass | 49 +++++++++++++++++++++++--------
>   1 file changed, 36 insertions(+), 13 deletions(-)
>
> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
> index 5e1c699118..2d42864cbd 100644
> --- a/meta/classes/useradd_base.bbclass
> +++ b/meta/classes/useradd_base.bbclass
> @@ -20,7 +20,8 @@ perform_groupadd () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupadd with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   	if test "x$group_exists" = "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupadd \$opts\" || true
> @@ -37,7 +38,8 @@ perform_useradd () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing useradd with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" = "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c  \"$PSEUDO useradd \$opts\" || true
> @@ -54,8 +56,26 @@ perform_groupmems () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupmems with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
> -	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
> +	local groupname=
> +	found_groupname=0
> +	for opt in $opts; do
> +		if [ "$found_groupname" = "1" ]; then
> +			groupname=$opt
> +			break
> +		elif [ "$opt" = "-g" ] || [ "$opt" = "--group" ]; then
> +			found_groupname=1
> +		fi
> +	done
> +	local username=
> +	found_username=0
> +	for opt in $opts; do
> +		if [ "$found_username" = "1" ]; then
> +			username=$opt
> +			break
> +		elif [ "$opt" = "-a" ] || [ "$opt" = "--add" ]; then
> +			found_username=1
> +		fi
> +	done
>   	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
>   	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*$" $rootdir/etc/group || true`"
>   	if test "x$mem_exists" = "x"; then
> @@ -73,14 +93,13 @@ perform_groupdel () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupdel with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   
>   	if test "x$group_exists" != "x"; then
> -		local awk_input='BEGIN {FS=":"}; $1=="'$groupname'" { print $3 }'
> -		local groupid=`echo "$awk_input" | awk -f- $rootdir/etc/group`
> -		local awk_check_users='BEGIN {FS=":"}; $4=="'$groupid'" {print $1}'
> -		local other_users=`echo "$awk_check_users" | awk -f- $rootdir/etc/passwd`
> +		local groupid=$(grep "^$groupname:" "$rootdir/etc/group" | cut -d: -f3)
> +		local other_users=$(grep ":$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)
>   
>   		if test "x$other_users" = "x"; then
>   			eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupdel \$opts\" || true
> @@ -100,7 +119,8 @@ perform_userdel () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing userdel with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO userdel \$opts\" || true
> @@ -120,7 +140,8 @@ perform_groupmod () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupmod with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   	if test "x$group_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupmod \$opts\"
> @@ -139,7 +160,8 @@ perform_usermod () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing usermod with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO usermod \$opts\"
> @@ -157,7 +179,8 @@ perform_passwd_expire () {
>   	local opts="$2"
>   	bbnote "${PN}: Performing equivalent of passwd --expire with [$opts]"
>   	# Directly set sp_lstchg to 0 without using the passwd command: Only root can do that
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done	
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO sed --follow-symlinks -i \''s/^\('$username':[^:]*\):[^:]*:/\1:0:/'\' $rootdir/etc/shadow \" || true
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#229962): https://lists.openembedded.org/g/openembedded-core/message/229962
> Mute This Topic: https://lists.openembedded.org/mt/117464897/7304865
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Alexander Kanavin Jan. 26, 2026, 12:28 p.m. UTC | #2 
On Mon, 26 Jan 2026 at 09:18, Chen Qi via lists.openembedded.org
<Qi.Chen=windriver.com@lists.openembedded.org> wrote:
>
> I have concern on this patch.
>
> You're modifying a basic bbclass to fix one case without finding out the
> root cause.
> By doing this, you're putting unnecessary restriction on the use of
> commands in this bbclass. Other commands with similar dependencies could
> potentially encounter the same issue in the future.
> I think if you find out the root cause, you'll come up with a better
> fix. This would not only resolve the current case but also prevent
> similar issues.

I agree, even before the search for the root cause, this does need
steps to reproduce, so others can also look properly into what the
issue is.

Alex
Adrian Freihofer Jan. 26, 2026, 2:01 p.m. UTC | #3 
On Mon, 2026-01-26 at 13:28 +0100, Alexander Kanavin via
lists.openembedded.org wrote:
> On Mon, 26 Jan 2026 at 09:18, Chen Qi via lists.openembedded.org
> <Qi.Chen=windriver.com@lists.openembedded.org> wrote:
> > 
> > I have concern on this patch.
> > 
> > You're modifying a basic bbclass to fix one case without finding
> > out the
> > root cause.
> > By doing this, you're putting unnecessary restriction on the use of
> > commands in this bbclass. Other commands with similar dependencies
> > could
> > potentially encounter the same issue in the future.
> > I think if you find out the root cause, you'll come up with a
> > better
> > fix. This would not only resolve the current case but also prevent
> > similar issues.
> 
> I agree, even before the search for the root cause, this does need
> steps to reproduce, so others can also look properly into what the
> issue is.
> 
> Alex


I think the root cause is that awk is called without having a
dependency on it. A proper fix would be adding this dependency.
Question is: where? I guess it is from the build-sysroots-1.0-r0
do_build_target_sysroot task on itself.

How to reproduce this is more or less described in the commit message.

My patches add:
  +USERADD_PACKAGES = "${PN}"
  +GROUPADD_PARAM:${PN} = "--system ${BPN}"
  +USERADD_PARAM:${PN} = "--system --home /var/lib/${BPN}
   --no-create-home --shell /bin/false --gid ${BPN} ${BPN}"
to a package.

Then the test case
devtool.DevtoolIdeSdkTests.test_devtool_ide_sdk_shared_sysroots
creates an dynamic SDK including this package.

That means, in the context of
  build-sysroots-1.0-r0 do_build_target_sysroot

postinst-base-passwd explodes like this:

  Exception: subprocess.CalledProcessError: Command
  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/
   build-st-254283/tmp/sysroots/qemux86-64/usr/bin/
   postinst-base-passwd' returned non-zero exit status 1.

  awk: error while loading shared libraries: libtinfo.so.5:
       cannot open shared object file: No such file or directory

https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/4983396/raw_inline
I'm not sure if this is solvable in a better way because my
understanding is that the dependency is circular.

If the task gets restarted, it works because the missing libraries are
then available.


Another fix is removing this dependency by not calling awk, what I
propose here.


Adrian

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#229983):
> https://lists.openembedded.org/g/openembedded-core/message/229983
> Mute This Topic: https://lists.openembedded.org/mt/117464897/4454582
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe:
> https://lists.openembedded.org/g/openembedded-core/unsub [
> adrian.freihofer@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
Peter Kjellerstedt Jan. 26, 2026, 5:41 p.m. UTC | #4 
Barring the result of the discussion on whether this is the right thing
to do or not, below are a couple of fixes in case it is determined that
this is what we want.

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Adrian Freihofer via lists.openembedded.org
> Sent: den 26 januari 2026 08:37
> To: openembedded-core@lists.openembedded.org
> Cc: Adrian Freihofer <adrian.freihofer@siemens.com>
> Subject: [OE-core] [PATCH v3 01/13] useradd_base.bbclass: do not use awk
> 
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
> 
> The exception bellow occurred during

bellow -> below

>   bitbake build-sysroots:do_build_target_sysroot
> Re-trying the same command again can "solve" the problem, which indicates
> that the problem is transient.
> Transient probably means that libncurses.so.5 was not available at the
> time when the useradd command was executed inside the sysroot population.
> However, adding gawk-native to useradd.bbclass
> DEPENDS:append:class-target does not help.
> 
> As a workaround this avoids using awk in useradd_base.bbclass.
> 
> ERROR: build-sysroots-1.0-r0 do_build_target_sysroot:
>   Error executing a python function in exec_func_python() autogenerated:
> 
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:do_build_target_sysroot(d)
>      0003:
> File: '...poky-master/layers/openembedded-core/meta/recipes-core/meta/build-sysroots.bb',
>       lineno: 46, function: do_build_target_sysroot
>      0042:    targetsysroot = d.getVar("STANDALONE_SYSROOT")
>      0043:    nativesysroot = d.getVar("STANDALONE_SYSROOT_NATIVE")
>      0044:    import os
>      0045:    os.environ['PATH'] = "%s/bin:%s/usr/bin:%s" %
> 	          (nativesysroot, nativesysroot, os.environ['PATH'])
>  *** 0046:    staging_populate_sysroot_dir(targetsysroot, nativesysroot, False, d)
>      0047:}
>      0048:do_build_target_sysroot[cleandirs] = "${STANDALONE_SYSROOT}"
>      0049:do_build_target_sysroot[nostamp] = "1"
>      0050:addtask do_build_target_sysroot
> File: '...poky-master/layers/openembedded-core/meta/classes-global/staging.bbclass',
>       lineno: 249, function: staging_populate_sysroot_dir
>      0245:                        continue
>      0246:
>      0247:    staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
>      0248:    for p in sorted(postinsts):
>  *** 0249:        bb.note("Running postinst {}, output:\n{}".format(
>                       p, subprocess.check_output(p, shell=True,
>                       stderr=subprocess.STDOUT)))
>      0250:
>      0251:#
>      0252:# Manifests here are complicated. The main sysroot area has the unpacked sstate
>      0253:# which us unrelocated and tracked by the main sstate manifests. Each recipe
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 472, function: check_output
>      0468:        else:
>      0469:            empty = b''
>      0470:        kwargs['input'] = empty
>      0471:
>  *** 0472:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>      0473:               **kwargs).stdout
>      0474:
>      0475:
>      0476:class CompletedProcess(object):
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 577, function: run
>      0573:            # We don't call process.wait() as .__exit__ does that for us.
>      0574:            raise
>      0575:        retcode = process.poll()
>      0576:        if check and retcode:
>  *** 0577:            raise CalledProcessError(retcode, process.args,
>      0578:                                     output=stdout, stderr=stderr)
>      0579:    return CompletedProcess(process.args, retcode, stdout, stderr)
>      0580:
>      0581:
> Exception: subprocess.CalledProcessError:
>     Command '...poky-master/build/tmp/sysroots/qemux86-64/usr/bin/postinst-useradd-01group-cmake-example'
>              returned non-zero exit status 1.
> 
> Subprocess output:
> ...poky-master/build/tmp/sysroots/x86_64/usr/sbin/useradd
> Running groupadd commands...
> NOTE: cmake-example: Performing groupadd with
>   [--root ...poky-master/build/tmp/sysroots/qemux86-64 --system cmake-example]
> awk: error while loading shared libraries: libncurses.so.5:
>      cannot open shared object file: No such file or directory
> groupadd: group 'cmake-example' already exists
> ERROR: cmake-example: groupadd command did not succeed.
> 
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
>  meta/classes/useradd_base.bbclass | 49 +++++++++++++++++++++++--------
>  1 file changed, 36 insertions(+), 13 deletions(-)
> 
> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
> index 5e1c699118..2d42864cbd 100644
> --- a/meta/classes/useradd_base.bbclass
> +++ b/meta/classes/useradd_base.bbclass
> @@ -20,7 +20,8 @@ perform_groupadd () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing groupadd with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>  	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>  	if test "x$group_exists" = "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupadd \$opts\" || true
> @@ -37,7 +38,8 @@ perform_useradd () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing useradd with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>  	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>  	if test "x$user_exists" = "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c  \"$PSEUDO useradd \$opts\" || true
> @@ -54,8 +56,26 @@ perform_groupmems () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing groupmems with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
> -	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
> +	local groupname=
> +	found_groupname=0
	
This should be a local variable too:

	local found_groupname=0

> +	for opt in $opts; do
> +		if [ "$found_groupname" = "1" ]; then
> +			groupname=$opt
> +			break
> +		elif [ "$opt" = "-g" ] || [ "$opt" = "--group" ]; then
> +			found_groupname=1
> +		fi
> +	done
> +	local username=
> +	found_username=0

This should be a local variable too:

	local found_username=0

> +	for opt in $opts; do
> +		if [ "$found_username" = "1" ]; then
> +			username=$opt
> +			break
> +		elif [ "$opt" = "-a" ] || [ "$opt" = "--add" ]; then
> +			found_username=1
> +		fi
> +	done
>  	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
>  	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*$" $rootdir/etc/group || true`"
>  	if test "x$mem_exists" = "x"; then
> @@ -73,14 +93,13 @@ perform_groupdel () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing groupdel with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>  	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
> 
>  	if test "x$group_exists" != "x"; then
> -		local awk_input='BEGIN {FS=":"}; $1=="'$groupname'" { print $3 }'
> -		local groupid=`echo "$awk_input" | awk -f- $rootdir/etc/group`
> -		local awk_check_users='BEGIN {FS=":"}; $4=="'$groupid'" {print $1}'
> -		local other_users=`echo "$awk_check_users" | awk -f- $rootdir/etc/passwd`
> +		local groupid=$(grep "^$groupname:" "$rootdir/etc/group" | cut -d: -f3)
> +		local other_users=$(grep ":$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)

That is not correct as it may match UIDs as well. This should work:

		local other_users=$(grep -E "^([^:]*:){3}$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)

> 
>  		if test "x$other_users" = "x"; then
>  			eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupdel \$opts\" || true
> @@ -100,7 +119,8 @@ perform_userdel () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing userdel with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>  	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>  	if test "x$user_exists" != "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO userdel \$opts\" || true
> @@ -120,7 +140,8 @@ perform_groupmod () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing groupmod with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>  	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>  	if test "x$group_exists" != "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupmod \$opts\"
> @@ -139,7 +160,8 @@ perform_usermod () {
>  	local rootdir="$1"
>  	local opts="$2"
>  	bbnote "${PN}: Performing usermod with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>  	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>  	if test "x$user_exists" != "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO usermod \$opts\"
> @@ -157,7 +179,8 @@ perform_passwd_expire () {
>  	local opts="$2"
>  	bbnote "${PN}: Performing equivalent of passwd --expire with [$opts]"
>  	# Directly set sp_lstchg to 0 without using the passwd command: Only root can do that
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>  	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>  	if test "x$user_exists" != "x"; then
>  		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO sed --follow-symlinks -i \''s/^\('$username':[^:]*\):[^:]*:/\1:0:/'\' $rootdir/etc/shadow \" || true
> --
> 2.52.0

//Peter
Alexander Kanavin Jan. 26, 2026, 8:18 p.m. UTC | #5 
On Mon, 26 Jan 2026 at 15:01, <adrian.freihofer@gmail.com> wrote:

>   Exception: subprocess.CalledProcessError: Command
>   '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/
>    build-st-254283/tmp/sysroots/qemux86-64/usr/bin/
>    postinst-base-passwd' returned non-zero exit status 1.
>
>   awk: error while loading shared libraries: libtinfo.so.5:
>        cannot open shared object file: No such file or directory
>
> https://autobuilder.yoctoproject.org/valkyrie/api/v2/logs/4983396/raw_inline
> I'm not sure if this is solvable in a better way because my
> understanding is that the dependency is circular.
>
> If the task gets restarted, it works because the missing libraries are
> then available.
>
>
> Another fix is removing this dependency by not calling awk, what I
> propose here.

We've seen similar issues elsewhere (down to the same missing libtinfo
error), and they all could be eventually resolved. User management is
notoriously tricky with these tool dependencies, as it needs to work
in various different scenarios and contexts.

Anyway, can you please do these steps on the command line, and confirm
that the same error occurs without having to run the selftest with
your patches applied, and describe what exact changes need to be made
and where and what commands in what order need to run? Then I could
try to look into it.

Alex
AdrianF Jan. 30, 2026, 2:01 p.m. UTC | #6 
Thank you all for the questions and suggestions. I hope there is now a
proper solution:
- https://bugzilla.yoctoproject.org/show_bug.cgi?id=16135
- https://lists.openembedded.org/g/openembedded-core/message/230185
- https://patchwork.yoctoproject.org/project/oe-core/list/?series=42835

Adrian
diff mbox series

Patch

diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index 5e1c699118..2d42864cbd 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -20,7 +20,8 @@  perform_groupadd () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupadd with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 	if test "x$group_exists" = "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupadd \$opts\" || true
@@ -37,7 +38,8 @@  perform_useradd () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing useradd with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" = "x"; then
 		eval flock -x $rootdir${sysconfdir} -c  \"$PSEUDO useradd \$opts\" || true
@@ -54,8 +56,26 @@  perform_groupmems () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupmems with [$opts]"
-	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
-	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
+	local groupname=
+	found_groupname=0
+	for opt in $opts; do
+		if [ "$found_groupname" = "1" ]; then
+			groupname=$opt
+			break
+		elif [ "$opt" = "-g" ] || [ "$opt" = "--group" ]; then
+			found_groupname=1
+		fi
+	done
+	local username=
+	found_username=0
+	for opt in $opts; do
+		if [ "$found_username" = "1" ]; then
+			username=$opt
+			break
+		elif [ "$opt" = "-a" ] || [ "$opt" = "--add" ]; then
+			found_username=1
+		fi
+	done
 	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
 	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*$" $rootdir/etc/group || true`"
 	if test "x$mem_exists" = "x"; then
@@ -73,14 +93,13 @@  perform_groupdel () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupdel with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 
 	if test "x$group_exists" != "x"; then
-		local awk_input='BEGIN {FS=":"}; $1=="'$groupname'" { print $3 }'
-		local groupid=`echo "$awk_input" | awk -f- $rootdir/etc/group`
-		local awk_check_users='BEGIN {FS=":"}; $4=="'$groupid'" {print $1}'
-		local other_users=`echo "$awk_check_users" | awk -f- $rootdir/etc/passwd`
+		local groupid=$(grep "^$groupname:" "$rootdir/etc/group" | cut -d: -f3)
+		local other_users=$(grep ":$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)
 
 		if test "x$other_users" = "x"; then
 			eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupdel \$opts\" || true
@@ -100,7 +119,8 @@  perform_userdel () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing userdel with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO userdel \$opts\" || true
@@ -120,7 +140,8 @@  perform_groupmod () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupmod with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 	if test "x$group_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupmod \$opts\"
@@ -139,7 +160,8 @@  perform_usermod () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing usermod with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO usermod \$opts\"
@@ -157,7 +179,8 @@  perform_passwd_expire () {
 	local opts="$2"
 	bbnote "${PN}: Performing equivalent of passwd --expire with [$opts]"
 	# Directly set sp_lstchg to 0 without using the passwd command: Only root can do that
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done	
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO sed --follow-symlinks -i \''s/^\('$username':[^:]*\):[^:]*:/\1:0:/'\' $rootdir/etc/shadow \" || true