diff mbox series

[v3,01/13] useradd_base.bbclass: do not use awk

Message ID 20260126073809.468495-2-adrian.freihofer@siemens.com
State New
Headers show
Series IDE SDK Improvements | expand

Commit Message

AdrianF Jan. 26, 2026, 7:37 a.m. UTC 
From: Adrian Freihofer <adrian.freihofer@siemens.com>

The exception bellow occurred during
  bitbake build-sysroots:do_build_target_sysroot
Re-trying the same command again can "solve" the problem, which indicates
that the problem is transient.
Transient probably means that libncurses.so.5 was not available at the
time when the useradd command was executed inside the sysroot population.
However, adding gawk-native to useradd.bbclass
DEPENDS:append:class-target does not help.

As a workaround this avoids using awk in useradd_base.bbclass.

ERROR: build-sysroots-1.0-r0 do_build_target_sysroot:
  Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_build_target_sysroot(d)
     0003:
File: '...poky-master/layers/openembedded-core/meta/recipes-core/meta/build-sysroots.bb',
      lineno: 46, function: do_build_target_sysroot
     0042:    targetsysroot = d.getVar("STANDALONE_SYSROOT")
     0043:    nativesysroot = d.getVar("STANDALONE_SYSROOT_NATIVE")
     0044:    import os
     0045:    os.environ['PATH'] = "%s/bin:%s/usr/bin:%s" %
	          (nativesysroot, nativesysroot, os.environ['PATH'])
 *** 0046:    staging_populate_sysroot_dir(targetsysroot, nativesysroot, False, d)
     0047:}
     0048:do_build_target_sysroot[cleandirs] = "${STANDALONE_SYSROOT}"
     0049:do_build_target_sysroot[nostamp] = "1"
     0050:addtask do_build_target_sysroot
File: '...poky-master/layers/openembedded-core/meta/classes-global/staging.bbclass',
      lineno: 249, function: staging_populate_sysroot_dir
     0245:                        continue
     0246:
     0247:    staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
     0248:    for p in sorted(postinsts):
 *** 0249:        bb.note("Running postinst {}, output:\n{}".format(
                      p, subprocess.check_output(p, shell=True,
                      stderr=subprocess.STDOUT)))
     0250:
     0251:#
     0252:# Manifests here are complicated. The main sysroot area has the unpacked sstate
     0253:# which us unrelocated and tracked by the main sstate manifests. Each recipe
File: '/usr/lib64/python3.13/subprocess.py', lineno: 472, function: check_output
     0468:        else:
     0469:            empty = b''
     0470:        kwargs['input'] = empty
     0471:
 *** 0472:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
     0473:               **kwargs).stdout
     0474:
     0475:
     0476:class CompletedProcess(object):
File: '/usr/lib64/python3.13/subprocess.py', lineno: 577, function: run
     0573:            # We don't call process.wait() as .__exit__ does that for us.
     0574:            raise
     0575:        retcode = process.poll()
     0576:        if check and retcode:
 *** 0577:            raise CalledProcessError(retcode, process.args,
     0578:                                     output=stdout, stderr=stderr)
     0579:    return CompletedProcess(process.args, retcode, stdout, stderr)
     0580:
     0581:
Exception: subprocess.CalledProcessError:
    Command '...poky-master/build/tmp/sysroots/qemux86-64/usr/bin/postinst-useradd-01group-cmake-example'
             returned non-zero exit status 1.

Subprocess output:
...poky-master/build/tmp/sysroots/x86_64/usr/sbin/useradd
Running groupadd commands...
NOTE: cmake-example: Performing groupadd with
  [--root ...poky-master/build/tmp/sysroots/qemux86-64 --system cmake-example]
awk: error while loading shared libraries: libncurses.so.5:
     cannot open shared object file: No such file or directory
groupadd: group 'cmake-example' already exists
ERROR: cmake-example: groupadd command did not succeed.

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
---
 meta/classes/useradd_base.bbclass | 49 +++++++++++++++++++++++--------
 1 file changed, 36 insertions(+), 13 deletions(-)

Comments

ChenQi Jan. 26, 2026, 8:18 a.m. UTC | #1 
I have concern on this patch.

You're modifying a basic bbclass to fix one case without finding out the 
root cause.
By doing this, you're putting unnecessary restriction on the use of 
commands in this bbclass. Other commands with similar dependencies could 
potentially encounter the same issue in the future.
I think if you find out the root cause, you'll come up with a better 
fix. This would not only resolve the current case but also prevent 
similar issues.

Regards,
Qi


On 1/26/26 15:37, Adrian Freihofer via lists.openembedded.org wrote:
> From: Adrian Freihofer <adrian.freihofer@siemens.com>
>
> The exception bellow occurred during
>    bitbake build-sysroots:do_build_target_sysroot
> Re-trying the same command again can "solve" the problem, which indicates
> that the problem is transient.
> Transient probably means that libncurses.so.5 was not available at the
> time when the useradd command was executed inside the sysroot population.
> However, adding gawk-native to useradd.bbclass
> DEPENDS:append:class-target does not help.
>
> As a workaround this avoids using awk in useradd_base.bbclass.
>
> ERROR: build-sysroots-1.0-r0 do_build_target_sysroot:
>    Error executing a python function in exec_func_python() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
>       0001:
>   *** 0002:do_build_target_sysroot(d)
>       0003:
> File: '...poky-master/layers/openembedded-core/meta/recipes-core/meta/build-sysroots.bb',
>        lineno: 46, function: do_build_target_sysroot
>       0042:    targetsysroot = d.getVar("STANDALONE_SYSROOT")
>       0043:    nativesysroot = d.getVar("STANDALONE_SYSROOT_NATIVE")
>       0044:    import os
>       0045:    os.environ['PATH'] = "%s/bin:%s/usr/bin:%s" %
> 	          (nativesysroot, nativesysroot, os.environ['PATH'])
>   *** 0046:    staging_populate_sysroot_dir(targetsysroot, nativesysroot, False, d)
>       0047:}
>       0048:do_build_target_sysroot[cleandirs] = "${STANDALONE_SYSROOT}"
>       0049:do_build_target_sysroot[nostamp] = "1"
>       0050:addtask do_build_target_sysroot
> File: '...poky-master/layers/openembedded-core/meta/classes-global/staging.bbclass',
>        lineno: 249, function: staging_populate_sysroot_dir
>       0245:                        continue
>       0246:
>       0247:    staging_processfixme(fixme, targetdir, targetsysroot, nativesysroot, d)
>       0248:    for p in sorted(postinsts):
>   *** 0249:        bb.note("Running postinst {}, output:\n{}".format(
>                        p, subprocess.check_output(p, shell=True,
>                        stderr=subprocess.STDOUT)))
>       0250:
>       0251:#
>       0252:# Manifests here are complicated. The main sysroot area has the unpacked sstate
>       0253:# which us unrelocated and tracked by the main sstate manifests. Each recipe
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 472, function: check_output
>       0468:        else:
>       0469:            empty = b''
>       0470:        kwargs['input'] = empty
>       0471:
>   *** 0472:    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>       0473:               **kwargs).stdout
>       0474:
>       0475:
>       0476:class CompletedProcess(object):
> File: '/usr/lib64/python3.13/subprocess.py', lineno: 577, function: run
>       0573:            # We don't call process.wait() as .__exit__ does that for us.
>       0574:            raise
>       0575:        retcode = process.poll()
>       0576:        if check and retcode:
>   *** 0577:            raise CalledProcessError(retcode, process.args,
>       0578:                                     output=stdout, stderr=stderr)
>       0579:    return CompletedProcess(process.args, retcode, stdout, stderr)
>       0580:
>       0581:
> Exception: subprocess.CalledProcessError:
>      Command '...poky-master/build/tmp/sysroots/qemux86-64/usr/bin/postinst-useradd-01group-cmake-example'
>               returned non-zero exit status 1.
>
> Subprocess output:
> ...poky-master/build/tmp/sysroots/x86_64/usr/sbin/useradd
> Running groupadd commands...
> NOTE: cmake-example: Performing groupadd with
>    [--root ...poky-master/build/tmp/sysroots/qemux86-64 --system cmake-example]
> awk: error while loading shared libraries: libncurses.so.5:
>       cannot open shared object file: No such file or directory
> groupadd: group 'cmake-example' already exists
> ERROR: cmake-example: groupadd command did not succeed.
>
> Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
> ---
>   meta/classes/useradd_base.bbclass | 49 +++++++++++++++++++++++--------
>   1 file changed, 36 insertions(+), 13 deletions(-)
>
> diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
> index 5e1c699118..2d42864cbd 100644
> --- a/meta/classes/useradd_base.bbclass
> +++ b/meta/classes/useradd_base.bbclass
> @@ -20,7 +20,8 @@ perform_groupadd () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupadd with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   	if test "x$group_exists" = "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupadd \$opts\" || true
> @@ -37,7 +38,8 @@ perform_useradd () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing useradd with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" = "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c  \"$PSEUDO useradd \$opts\" || true
> @@ -54,8 +56,26 @@ perform_groupmems () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupmems with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
> -	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
> +	local groupname=
> +	found_groupname=0
> +	for opt in $opts; do
> +		if [ "$found_groupname" = "1" ]; then
> +			groupname=$opt
> +			break
> +		elif [ "$opt" = "-g" ] || [ "$opt" = "--group" ]; then
> +			found_groupname=1
> +		fi
> +	done
> +	local username=
> +	found_username=0
> +	for opt in $opts; do
> +		if [ "$found_username" = "1" ]; then
> +			username=$opt
> +			break
> +		elif [ "$opt" = "-a" ] || [ "$opt" = "--add" ]; then
> +			found_username=1
> +		fi
> +	done
>   	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
>   	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*$" $rootdir/etc/group || true`"
>   	if test "x$mem_exists" = "x"; then
> @@ -73,14 +93,13 @@ perform_groupdel () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupdel with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   
>   	if test "x$group_exists" != "x"; then
> -		local awk_input='BEGIN {FS=":"}; $1=="'$groupname'" { print $3 }'
> -		local groupid=`echo "$awk_input" | awk -f- $rootdir/etc/group`
> -		local awk_check_users='BEGIN {FS=":"}; $4=="'$groupid'" {print $1}'
> -		local other_users=`echo "$awk_check_users" | awk -f- $rootdir/etc/passwd`
> +		local groupid=$(grep "^$groupname:" "$rootdir/etc/group" | cut -d: -f3)
> +		local other_users=$(grep ":$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)
>   
>   		if test "x$other_users" = "x"; then
>   			eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupdel \$opts\" || true
> @@ -100,7 +119,8 @@ perform_userdel () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing userdel with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO userdel \$opts\" || true
> @@ -120,7 +140,8 @@ perform_groupmod () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing groupmod with [$opts]"
> -	local groupname=`echo "$opts" | awk '{ print $NF }'`
> +	local groupname=
> +	for word in $opts; do groupname=$word; done
>   	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
>   	if test "x$group_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupmod \$opts\"
> @@ -139,7 +160,8 @@ perform_usermod () {
>   	local rootdir="$1"
>   	local opts="$2"
>   	bbnote "${PN}: Performing usermod with [$opts]"
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO usermod \$opts\"
> @@ -157,7 +179,8 @@ perform_passwd_expire () {
>   	local opts="$2"
>   	bbnote "${PN}: Performing equivalent of passwd --expire with [$opts]"
>   	# Directly set sp_lstchg to 0 without using the passwd command: Only root can do that
> -	local username=`echo "$opts" | awk '{ print $NF }'`
> +	local username=
> +	for word in $opts; do username=$word; done	
>   	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
>   	if test "x$user_exists" != "x"; then
>   		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO sed --follow-symlinks -i \''s/^\('$username':[^:]*\):[^:]*:/\1:0:/'\' $rootdir/etc/shadow \" || true
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#229962): https://lists.openembedded.org/g/openembedded-core/message/229962
> Mute This Topic: https://lists.openembedded.org/mt/117464897/7304865
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
diff mbox series

Patch

diff --git a/meta/classes/useradd_base.bbclass b/meta/classes/useradd_base.bbclass
index 5e1c699118..2d42864cbd 100644
--- a/meta/classes/useradd_base.bbclass
+++ b/meta/classes/useradd_base.bbclass
@@ -20,7 +20,8 @@  perform_groupadd () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupadd with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 	if test "x$group_exists" = "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupadd \$opts\" || true
@@ -37,7 +38,8 @@  perform_useradd () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing useradd with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" = "x"; then
 		eval flock -x $rootdir${sysconfdir} -c  \"$PSEUDO useradd \$opts\" || true
@@ -54,8 +56,26 @@  perform_groupmems () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupmems with [$opts]"
-	local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
-	local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
+	local groupname=
+	found_groupname=0
+	for opt in $opts; do
+		if [ "$found_groupname" = "1" ]; then
+			groupname=$opt
+			break
+		elif [ "$opt" = "-g" ] || [ "$opt" = "--group" ]; then
+			found_groupname=1
+		fi
+	done
+	local username=
+	found_username=0
+	for opt in $opts; do
+		if [ "$found_username" = "1" ]; then
+			username=$opt
+			break
+		elif [ "$opt" = "-a" ] || [ "$opt" = "--add" ]; then
+			found_username=1
+		fi
+	done
 	bbnote "${PN}: Running groupmems command with group $groupname and user $username"
 	local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*$" $rootdir/etc/group || true`"
 	if test "x$mem_exists" = "x"; then
@@ -73,14 +93,13 @@  perform_groupdel () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupdel with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 
 	if test "x$group_exists" != "x"; then
-		local awk_input='BEGIN {FS=":"}; $1=="'$groupname'" { print $3 }'
-		local groupid=`echo "$awk_input" | awk -f- $rootdir/etc/group`
-		local awk_check_users='BEGIN {FS=":"}; $4=="'$groupid'" {print $1}'
-		local other_users=`echo "$awk_check_users" | awk -f- $rootdir/etc/passwd`
+		local groupid=$(grep "^$groupname:" "$rootdir/etc/group" | cut -d: -f3)
+		local other_users=$(grep ":$groupid:" "$rootdir/etc/passwd" | cut -d: -f1)
 
 		if test "x$other_users" = "x"; then
 			eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupdel \$opts\" || true
@@ -100,7 +119,8 @@  perform_userdel () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing userdel with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO userdel \$opts\" || true
@@ -120,7 +140,8 @@  perform_groupmod () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing groupmod with [$opts]"
-	local groupname=`echo "$opts" | awk '{ print $NF }'`
+	local groupname=
+	for word in $opts; do groupname=$word; done
 	local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
 	if test "x$group_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO groupmod \$opts\"
@@ -139,7 +160,8 @@  perform_usermod () {
 	local rootdir="$1"
 	local opts="$2"
 	bbnote "${PN}: Performing usermod with [$opts]"
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO usermod \$opts\"
@@ -157,7 +179,8 @@  perform_passwd_expire () {
 	local opts="$2"
 	bbnote "${PN}: Performing equivalent of passwd --expire with [$opts]"
 	# Directly set sp_lstchg to 0 without using the passwd command: Only root can do that
-	local username=`echo "$opts" | awk '{ print $NF }'`
+	local username=
+	for word in $opts; do username=$word; done	
 	local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
 	if test "x$user_exists" != "x"; then
 		eval flock -x $rootdir${sysconfdir} -c \"$PSEUDO sed --follow-symlinks -i \''s/^\('$username':[^:]*\):[^:]*:/\1:0:/'\' $rootdir/etc/shadow \" || true