| Message ID | BN0PR20MB3941083EB28A3B5AB04EFAC5BB7E9@BN0PR20MB3941.namprd20.prod.outlook.com |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [dunfell,RFC] busybox.inc: Create temporary busybox links during install | expand |
All, Ping on this RFC. It works for me, but I have a feeling there is a better way to do this. It still seems a little messy and could probably be simplified for the same effect. Thanks, Bryan > -----Original Message----- > From: Bryan Evenson > Sent: Thursday, December 23, 2021 9:50 AM > To: openembedded-core@lists.openembedded.org > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox links > during install > > Busybox upgrades sometimes fail, especially if there is a major distribution > upgrade and all packages need to be updated. Success is highly dependent > on the package upgrade order. > > Commit [1] attempts to ensure a shell is still present by adding an alternative > to /bin/sh if busybox is the only shell. However, if busybox is not the only > shell present and the other shells are upgrading, it may then be possible that > all shells will be removed during the upgrade process. > > Commit [2] creates temporary symbolic links for all the busybox links during > busybox's postinst step. However, this is too late in the process as some > packages attempt to use 'rm' and 'sed' after update-alternatives removes > the old links and prior to when busybox's postinst step runs. > > This fix is similar to [2] but runs during the preinst step. For opkg, this is the > first step that is guaranteed to run from the new package (prerm is run from > the old package) and will therefore be a backwards-compatible fix for > upgrading older systems. > > Copies the existing busybox binary and the busybox.links files to a temporary > directory and then creates alternative links for all installed busybox > commands. The temporary links and directory are cleaned up during the > postinst step. > > RFC: This works for me, but there may be room for improvement. I don't > know if the current pkg_prerm steps are necessary anymore. However, in > my testing I did need the links for update-alternatives to work in the preinst > step. I am also not certain if the > populate_packages_updatealternatives_append > step is necessary anymore. I have also only tested this fix on dunfell, as I > don't have a working image based on master yet. It may be more > appropriate for this to go to master and then be backported to dunfell, but I > would need assistance in testing. > > [1] https://git.openembedded.org/openembedded- > core/commit/meta/recipes- > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23a > [2] https://git.openembedded.org/openembedded- > core/commit/meta/recipes- > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > d2 > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > --- > meta/recipes-core/busybox/busybox.inc | 57 > ++++++++++++++++++++++++++- > 1 file changed, 55 insertions(+), 2 deletions(-) > > diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes- > core/busybox/busybox.inc > index e0522be729..c85402411b 100644 > --- a/meta/recipes-core/busybox/busybox.inc > +++ b/meta/recipes-core/busybox/busybox.inc > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > pkg_postinst_${PN}_append () { > - # If busybox exists in the remove directory it is because it was the only > shell left. > if [ "x$D" = "x" ] ; then > + # If busybox exists in the remove directory it is because it was the only > shell left. > if [ "x$BUSYBOX" != "x" ] ; then > update-alternatives --remove sh $BUSYBOX > - rm -f $BUSYBOX > fi > + # Remove the temporary alternatives > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > + if [ "$busybox_preinstdir" != '/tmp/busyboxpreinst-*' ] ; then > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > + if [ -e $BUSYBOX ] ; then > + for suffix in "" ".nosuid" ".suid"; do > + if [ -e $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > + while read link; do > + update-alternatives --remove $($BUSYBOX basename > $link) $BUSYBOX > + done < $BUSYBOX_PREINST_DIR/busybox.links$suffix > + fi > + done > + fi > + rm -rf $BUSYBOX_PREINST_DIR > + fi > + done > fi > } > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > fi > } > > +pkg_preinst_${PN} () { > + # Create a temporary copy the busybox binary and the links files. Then, > + # install an alternative link for all the links. Other packages use these > + # commands during their upgrade process. This ensures the links are > available > + # to all the other packages. We do this in the preinst step because it is > + # the first step guaranteed to be used from the new package. The > prerm is > + # used from the old package. Placing this here ensures it runs on > upgrade even > + # on older systems. > + > + if [ "x$D" = "x" ] ; then > + # update-alternatives may need the links from commands added in > the prerm step > + # to operate. Make sure we can get to that path. > + for busybox_rmdir in /tmp/busyboxrm-*; do > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; then > + export PATH=$busybox_rmdir:$PATH > + fi > + done > + > + # Create a temporary directory for the busybox binary and the link lists > + BUSYBOX=${base_bindir}/busybox > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d /tmp/busyboxpreinst- > XXXXXX` > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > + > + # Go through all the links and install an alternative that points to the > temporary > + # busybox binary. > + for suffix in "" ".nosuid" ".suid"; do > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; then > + $BUSYBOX cp ${sysconfdir}/busybox.links$suffix > $BUSYBOX_TMP_DIR > + while read link; do > + update-alternatives --install $link $($BUSYBOX basename $link) > $BUSYBOX_TMP_LOC 1 > + done < $BUSYBOX_TMP_DIR/busybox.links$suffix > + fi > + done > + fi > +} > + > pkg_postrm_${PN} () { > # Add path to remove dir in case we removed our only grep > if [ "x$D" = "x" ] ; then > -- > 2.17.1
Hi Bryan, Sorry, maybe I didn't fully understand the use-case. Are you trying to upgrade the busybox on demand? If yes, that is not a good idea. I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" and creating a custom locks is not a good at all. Cheers, Andrej On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > All, > > Ping on this RFC. It works for me, but I have a feeling there is a > better way to do this. It still seems a little messy and could > probably be simplified for the same effect. > > Thanks, > Bryan > > > -----Original Message----- > > From: Bryan Evenson > > Sent: Thursday, December 23, 2021 9:50 AM > > To: openembedded-core@lists.openembedded.org > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > > links > > during install > > > > Busybox upgrades sometimes fail, especially if there is a major > > distribution > > upgrade and all packages need to be updated. Success is highly > > dependent > > on the package upgrade order. > > > > Commit [1] attempts to ensure a shell is still present by adding an > > alternative > > to /bin/sh if busybox is the only shell. However, if busybox is > > not the only > > shell present and the other shells are upgrading, it may then be > > possible that > > all shells will be removed during the upgrade process. > > > > Commit [2] creates temporary symbolic links for all the busybox > > links during > > busybox's postinst step. However, this is too late in the process > > as some > > packages attempt to use 'rm' and 'sed' after update-alternatives > > removes > > the old links and prior to when busybox's postinst step runs. > > > > This fix is similar to [2] but runs during the preinst step. For > > opkg, this is the > > first step that is guaranteed to run from the new package (prerm is > > run from > > the old package) and will therefore be a backwards-compatible fix > > for > > upgrading older systems. > > > > Copies the existing busybox binary and the busybox.links files to a > > temporary > > directory and then creates alternative links for all installed > > busybox > > commands. The temporary links and directory are cleaned up during > > the > > postinst step. > > > > RFC: This works for me, but there may be room for improvement. I > > don't > > know if the current pkg_prerm steps are necessary anymore. > > However, in > > my testing I did need the links for update-alternatives to work in > > the preinst > > step. I am also not certain if the > > populate_packages_updatealternatives_append > > step is necessary anymore. I have also only tested this fix on > > dunfell, as I > > don't have a working image based on master yet. It may be more > > appropriate for this to go to master and then be backported to > > dunfell, but I > > would need assistance in testing. > > > > [1] https://git.openembedded.org/openembedded- > > core/commit/meta/recipes- > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > a > > [2] https://git.openembedded.org/openembedded- > > core/commit/meta/recipes- > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > d2 > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > --- > > meta/recipes-core/busybox/busybox.inc | 57 > > ++++++++++++++++++++++++++- > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes- > > core/busybox/busybox.inc > > index e0522be729..c85402411b 100644 > > --- a/meta/recipes-core/busybox/busybox.inc > > +++ b/meta/recipes-core/busybox/busybox.inc > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > pkg_postinst_${PN}_append () { > > - # If busybox exists in the remove directory it is because > > it was the only > > shell left. > > if [ "x$D" = "x" ] ; then > > + # If busybox exists in the remove directory it is > > because it was the only > > shell left. > > if [ "x$BUSYBOX" != "x" ] ; then > > update-alternatives --remove sh $BUSYBOX > > - rm -f $BUSYBOX > > fi > > + # Remove the temporary alternatives > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > + if [ "$busybox_preinstdir" != '/tmp/busyboxpreinst- > > *' ] ; then > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > + if [ -e $BUSYBOX ] ; then > > + for suffix in "" ".nosuid" ".suid"; do > > + if [ -e > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > + while read link; do > > + update-alternatives --remove > > $($BUSYBOX basename > > $link) $BUSYBOX > > + done < > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > + fi > > + done > > + fi > > + rm -rf $BUSYBOX_PREINST_DIR > > + fi > > + done > > fi > > } > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > fi > > } > > > > +pkg_preinst_${PN} () { > > + # Create a temporary copy the busybox binary and the links > > files. Then, > > + # install an alternative link for all the links. Other > > packages use these > > + # commands during their upgrade process. This ensures the > > links are > > available > > + # to all the other packages. We do this in the preinst > > step because it is > > + # the first step guaranteed to be used from the new > > package. The > > prerm is > > + # used from the old package. Placing this here ensures it > > runs on > > upgrade even > > + # on older systems. > > + > > + if [ "x$D" = "x" ] ; then > > + # update-alternatives may need the links from commands > > added in > > the prerm step > > + # to operate. Make sure we can get to that path. > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > then > > + export PATH=$busybox_rmdir:$PATH > > + fi > > + done > > + > > + # Create a temporary directory for the busybox binary > > and the link lists > > + BUSYBOX=${base_bindir}/busybox > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > /tmp/busyboxpreinst- > > XXXXXX` > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > + > > + # Go through all the links and install an alternative > > that points to the > > temporary > > + # busybox binary. > > + for suffix in "" ".nosuid" ".suid"; do > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; then > > + $BUSYBOX cp ${sysconfdir}/busybox.links$suffix > > $BUSYBOX_TMP_DIR > > + while read link; do > > + update-alternatives --install $link > > $($BUSYBOX basename $link) > > $BUSYBOX_TMP_LOC 1 > > + done < $BUSYBOX_TMP_DIR/busybox.links$suffix > > + fi > > + done > > + fi > > +} > > + > > pkg_postrm_${PN} () { > > # Add path to remove dir in case we removed our only grep > > if [ "x$D" = "x" ] ; then > > -- > > 2.17.1
Andrej, Thanks for the response. This is an attempt to fix a problem I am having with automated firmware upgrades for my system. I am using opkg for a package manager; not sure if the same problem exists with other package managers. I run into problems whenever busybox is one of the packages that needs to get updated. I enact my distribution firmware upgrade by calling "opkg --download-only upgrade; opkg upgrade". What I see happen is: 1. In the busybox pkg_prerm stage sets up some soft links for some common applets in a temporary directory and exports a path to that directory. It might also setup a temporary alternative to /bin/sh if it is the last shell. 2. After the remove stage, the busybox binary is gone. The softlinks created in the prerm stage are useless since they point to binary that no longer exists. 3. opkg continues with upgrade on other packages which may depend on a command provided by busybox in a prerm, postrm, preinst or postinst script. These upgrades then fail since the commands are no longer available. 4. The busybox upgrade completes, which may or may not complete successfully. For what I am attempting, I am upgrading my system from the morty branch to dunfell. I have util-linux on my system which shares some alternatives with busybox. The util-linux upgrade fails because it needs some busybox applets during its upgrade process. Then the busybox upgrade fails because the final update-alternatives doesn’t work; some files still exist that util-linux couldn't remove during its upgrade that clash with busybox's alternatives. After trying several ways to get my upgrade to work, I landed on the approach below. I'm creating a temporary directory and copying the busybox binary and the busybox.links files to that directory. I then install an alternative for every applet for busybox listed in all of its busybox.links files that points to the temporary copy of the busybox binary. This means that any package that uses a busybox applet during its install process should still work. Then during the postinst step I am removing all the temporary alternative links. I use the temporary busybox.links files for removing the alternative links in case the upgraded busybox is now configured with a different set of applets. This is a heavy handed approach, and it does extend the upgrade process for me by a few minutes since it runs through update-alternatives for busybox two more times. But, the approach works for me and I think would be more resilient than past approaches. I tried to mimic the existing code in my additions. If a more widescale rewrite makes sense than that works for me also. Thanks, Bryan > -----Original Message----- > From: Valek, Andrej <andrej.valek@siemens.com> > Sent: Friday, January 21, 2022 9:01 AM > To: openembedded-core@lists.openembedded.org; Bryan Evenson > <bevenson@melinkcorp.com> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hi Bryan, > > Sorry, maybe I didn't fully understand the use-case. Are you trying to > upgrade the busybox on demand? If yes, that is not a good idea. > > I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" and > creating a custom locks is not a good at all. > > Cheers, > Andrej > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > All, > > > > Ping on this RFC. It works for me, but I have a feeling there is a > > better way to do this. It still seems a little messy and could > > probably be simplified for the same effect. > > > > Thanks, > > Bryan > > > > > -----Original Message----- > > > From: Bryan Evenson > > > Sent: Thursday, December 23, 2021 9:50 AM > > > To: openembedded-core@lists.openembedded.org > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > > > links during install > > > > > > Busybox upgrades sometimes fail, especially if there is a major > > > distribution upgrade and all packages need to be updated. Success > > > is highly dependent on the package upgrade order. > > > > > > Commit [1] attempts to ensure a shell is still present by adding an > > > alternative to /bin/sh if busybox is the only shell. However, if > > > busybox is not the only shell present and the other shells are > > > upgrading, it may then be possible that all shells will be removed > > > during the upgrade process. > > > > > > Commit [2] creates temporary symbolic links for all the busybox > > > links during busybox's postinst step. However, this is too late in > > > the process as some packages attempt to use 'rm' and 'sed' after > > > update-alternatives removes the old links and prior to when > > > busybox's postinst step runs. > > > > > > This fix is similar to [2] but runs during the preinst step. For > > > opkg, this is the first step that is guaranteed to run from the new > > > package (prerm is run from the old package) and will therefore be a > > > backwards-compatible fix for upgrading older systems. > > > > > > Copies the existing busybox binary and the busybox.links files to a > > > temporary directory and then creates alternative links for all > > > installed busybox commands. The temporary links and directory are > > > cleaned up during the postinst step. > > > > > > RFC: This works for me, but there may be room for improvement. I > > > don't know if the current pkg_prerm steps are necessary anymore. > > > However, in > > > my testing I did need the links for update-alternatives to work in > > > the preinst step. I am also not certain if the > > > populate_packages_updatealternatives_append > > > step is necessary anymore. I have also only tested this fix on > > > dunfell, as I don't have a working image based on master yet. It > > > may be more appropriate for this to go to master and then be > > > backported to dunfell, but I would need assistance in testing. > > > > > > [1] https://git.openembedded.org/openembedded- > > > core/commit/meta/recipes- > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > a > > > [2] https://git.openembedded.org/openembedded- > > > core/commit/meta/recipes- > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > d2 > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > --- > > > meta/recipes-core/busybox/busybox.inc | 57 > > > ++++++++++++++++++++++++++- > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes- > > > core/busybox/busybox.inc index e0522be729..c85402411b 100644 > > > --- a/meta/recipes-core/busybox/busybox.inc > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > pkg_postinst_${PN}_append () { > > > - # If busybox exists in the remove directory it is because > > > it was the only shell left. > > > if [ "x$D" = "x" ] ; then > > > + # If busybox exists in the remove directory it is > > > because it was the only > > > shell left. > > > if [ "x$BUSYBOX" != "x" ] ; then > > > update-alternatives --remove sh $BUSYBOX > > > - rm -f $BUSYBOX > > > fi > > > + # Remove the temporary alternatives > > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > > + if [ "$busybox_preinstdir" != '/tmp/busyboxpreinst- > > > *' ] ; then > > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > + if [ -e $BUSYBOX ] ; then > > > + for suffix in "" ".nosuid" ".suid"; do > > > + if [ -e > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > + while read link; do > > > + update-alternatives --remove > > > $($BUSYBOX basename > > > $link) $BUSYBOX > > > + done < > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > + fi > > > + done > > > + fi > > > + rm -rf $BUSYBOX_PREINST_DIR > > > + fi > > > + done > > > fi > > > } > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > fi > > > } > > > > > > +pkg_preinst_${PN} () { > > > + # Create a temporary copy the busybox binary and the links > > > files. Then, > > > + # install an alternative link for all the links. Other > > > packages use these > > > + # commands during their upgrade process. This ensures the > > > links are > > > available > > > + # to all the other packages. We do this in the preinst > > > step because it is > > > + # the first step guaranteed to be used from the new > > > package. The > > > prerm is > > > + # used from the old package. Placing this here ensures it > > > runs on > > > upgrade even > > > + # on older systems. > > > + > > > + if [ "x$D" = "x" ] ; then > > > + # update-alternatives may need the links from commands > > > added in > > > the prerm step > > > + # to operate. Make sure we can get to that path. > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > > then > > > + export PATH=$busybox_rmdir:$PATH > > > + fi > > > + done > > > + > > > + # Create a temporary directory for the busybox binary > > > and the link lists > > > + BUSYBOX=${base_bindir}/busybox > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > /tmp/busyboxpreinst- > > > XXXXXX` > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > + > > > + # Go through all the links and install an alternative > > > that points to the > > > temporary > > > + # busybox binary. > > > + for suffix in "" ".nosuid" ".suid"; do > > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; then > > > + $BUSYBOX cp ${sysconfdir}/busybox.links$suffix > > > $BUSYBOX_TMP_DIR > > > + while read link; do > > > + update-alternatives --install $link > > > $($BUSYBOX basename $link) > > > $BUSYBOX_TMP_LOC 1 > > > + done < $BUSYBOX_TMP_DIR/busybox.links$suffix > > > + fi > > > + done > > > + fi > > > +} > > > + > > > pkg_postrm_${PN} () { > > > # Add path to remove dir in case we removed our only grep > > > if [ "x$D" = "x" ] ; then > > > -- > > > 2.17.1
Hello again, Maybe a general question. Is it working in current master? I do not want to brake dunfell, just applying something, which will create a lot of divergence. Cheers, Andrej On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > Andrej, > > Thanks for the response. This is an attempt to fix a problem I am > having with automated firmware upgrades for my system. I am using > opkg for a package manager; not sure if the same problem exists with > other package managers. I run into problems whenever busybox is one > of the packages that needs to get updated. I enact my distribution > firmware upgrade by calling "opkg --download-only upgrade; opkg > upgrade". What I see happen is: > > 1. In the busybox pkg_prerm stage sets up some soft links for some > common applets in a temporary directory and exports a path to that > directory. It might also setup a temporary alternative to /bin/sh if > it is the last shell. > 2. After the remove stage, the busybox binary is gone. The softlinks > created in the prerm stage are useless since they point to binary > that no longer exists. > 3. opkg continues with upgrade on other packages which may depend on > a command provided by busybox in a prerm, postrm, preinst or postinst > script. These upgrades then fail since the commands are no longer > available. > 4. The busybox upgrade completes, which may or may not complete > successfully. For what I am attempting, I am upgrading my system > from the morty branch to dunfell. I have util-linux on my system > which shares some alternatives with busybox. The util-linux upgrade > fails because it needs some busybox applets during its upgrade > process. Then the busybox upgrade fails because the final update- > alternatives doesn’t work; some files still exist that util-linux > couldn't remove during its upgrade that clash with busybox's > alternatives. > > After trying several ways to get my upgrade to work, I landed on the > approach below. I'm creating a temporary directory and copying the > busybox binary and the busybox.links files to that directory. I then > install an alternative for every applet for busybox listed in all of > its busybox.links files that points to the temporary copy of the > busybox binary. This means that any package that uses a busybox > applet during its install process should still work. Then during the > postinst step I am removing all the temporary alternative links. I > use the temporary busybox.links files for removing the alternative > links in case the upgraded busybox is now configured with a different > set of applets. > > This is a heavy handed approach, and it does extend the upgrade > process for me by a few minutes since it runs through update- > alternatives for busybox two more times. But, the approach works for > me and I think would be more resilient than past approaches. I tried > to mimic the existing code in my additions. If a more widescale > rewrite makes sense than that works for me also. > > Thanks, > Bryan > > > > -----Original Message----- > > From: Valek, Andrej <andrej.valek@siemens.com> > > Sent: Friday, January 21, 2022 9:01 AM > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > <bevenson@melinkcorp.com> > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > busybox > > links during install > > > > Hi Bryan, > > > > Sorry, maybe I didn't fully understand the use-case. Are you trying > > to > > upgrade the busybox on demand? If yes, that is not a good idea. > > > > I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" > > and > > creating a custom locks is not a good at all. > > > > Cheers, > > Andrej > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > All, > > > > > > Ping on this RFC. It works for me, but I have a feeling there is > > > a > > > better way to do this. It still seems a little messy and could > > > probably be simplified for the same effect. > > > > > > Thanks, > > > Bryan > > > > > > > -----Original Message----- > > > > From: Bryan Evenson > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > To: openembedded-core@lists.openembedded.org > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > busybox > > > > links during install > > > > > > > > Busybox upgrades sometimes fail, especially if there is a major > > > > distribution upgrade and all packages need to be updated. > > > > Success > > > > is highly dependent on the package upgrade order. > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > adding an > > > > alternative to /bin/sh if busybox is the only shell. However, > > > > if > > > > busybox is not the only shell present and the other shells are > > > > upgrading, it may then be possible that all shells will be > > > > removed > > > > during the upgrade process. > > > > > > > > Commit [2] creates temporary symbolic links for all the busybox > > > > links during busybox's postinst step. However, this is too > > > > late in > > > > the process as some packages attempt to use 'rm' and 'sed' > > > > after > > > > update-alternatives removes the old links and prior to when > > > > busybox's postinst step runs. > > > > > > > > This fix is similar to [2] but runs during the preinst step. > > > > For > > > > opkg, this is the first step that is guaranteed to run from the > > > > new > > > > package (prerm is run from the old package) and will therefore > > > > be a > > > > backwards-compatible fix for upgrading older systems. > > > > > > > > Copies the existing busybox binary and the busybox.links files > > > > to a > > > > temporary directory and then creates alternative links for all > > > > installed busybox commands. The temporary links and directory > > > > are > > > > cleaned up during the postinst step. > > > > > > > > RFC: This works for me, but there may be room for improvement. > > > > I > > > > don't know if the current pkg_prerm steps are necessary > > > > anymore. > > > > However, in > > > > my testing I did need the links for update-alternatives to work > > > > in > > > > the preinst step. I am also not certain if the > > > > populate_packages_updatealternatives_append > > > > step is necessary anymore. I have also only tested this fix on > > > > dunfell, as I don't have a working image based on master yet. > > > > It > > > > may be more appropriate for this to go to master and then be > > > > backported to dunfell, but I would need assistance in testing. > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > core/commit/meta/recipes- > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > a > > > > [2] https://git.openembedded.org/openembedded- > > > > core/commit/meta/recipes- > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > d2 > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > --- > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > ++++++++++++++++++++++++++- > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > b/meta/recipes- > > > > core/busybox/busybox.inc index e0522be729..c85402411b 100644 > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > pkg_postinst_${PN}_append () { > > > > - # If busybox exists in the remove directory it is > > > > because > > > > it was the only shell left. > > > > if [ "x$D" = "x" ] ; then > > > > + # If busybox exists in the remove directory it is > > > > because it was the only > > > > shell left. > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > update-alternatives --remove sh $BUSYBOX > > > > - rm -f $BUSYBOX > > > > fi > > > > + # Remove the temporary alternatives > > > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > > > + if [ "$busybox_preinstdir" != > > > > '/tmp/busyboxpreinst- > > > > *' ] ; then > > > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > + if [ -e $BUSYBOX ] ; then > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > + if [ -e > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > + while read link; do > > > > + update-alternatives --remove > > > > $($BUSYBOX basename > > > > $link) $BUSYBOX > > > > + done < > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > + fi > > > > + done > > > > + fi > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > + fi > > > > + done > > > > fi > > > > } > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > fi > > > > } > > > > > > > > +pkg_preinst_${PN} () { > > > > + # Create a temporary copy the busybox binary and the > > > > links > > > > files. Then, > > > > + # install an alternative link for all the links. > > > > Other > > > > packages use these > > > > + # commands during their upgrade process. This ensures > > > > the > > > > links are > > > > available > > > > + # to all the other packages. We do this in the > > > > preinst > > > > step because it is > > > > + # the first step guaranteed to be used from the new > > > > package. The > > > > prerm is > > > > + # used from the old package. Placing this here > > > > ensures it > > > > runs on > > > > upgrade even > > > > + # on older systems. > > > > + > > > > + if [ "x$D" = "x" ] ; then > > > > + # update-alternatives may need the links from > > > > commands > > > > added in > > > > the prerm step > > > > + # to operate. Make sure we can get to that path. > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > > > then > > > > + export PATH=$busybox_rmdir:$PATH > > > > + fi > > > > + done > > > > + > > > > + # Create a temporary directory for the busybox > > > > binary > > > > and the link lists > > > > + BUSYBOX=${base_bindir}/busybox > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > /tmp/busyboxpreinst- > > > > XXXXXX` > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > + > > > > + # Go through all the links and install an > > > > alternative > > > > that points to the > > > > temporary > > > > + # busybox binary. > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; > > > > then > > > > + $BUSYBOX cp > > > > ${sysconfdir}/busybox.links$suffix > > > > $BUSYBOX_TMP_DIR > > > > + while read link; do > > > > + update-alternatives --install $link > > > > $($BUSYBOX basename $link) > > > > $BUSYBOX_TMP_LOC 1 > > > > + done < > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > + fi > > > > + done > > > > + fi > > > > +} > > > > + > > > > pkg_postrm_${PN} () { > > > > # Add path to remove dir in case we removed our only > > > > grep > > > > if [ "x$D" = "x" ] ; then > > > > -- > > > > 2.17.1 >
Andrej, I suspect it is still an issue in master, but I haven't been able to confirm. I'm using a third-party layer that hasn't been updated to support the new override syntax introduced in the honister release. I agree that it should be fixed in master and then backported as deemed necessary to other supported releases. I wasn’t able to do that yet but I wanted to give some visibility to the issue. I suspect that it is still a problem in master since I don't see any changes in busybox.inc between dunfell and master that I think could change this behavior. I'll continue to work on a consistent way for others to reproduce the problem. I suspect anyone could reproduce this problem by just adding 'util-linux' to core-image-minimal and attempting an upgrade between major releases. I'll work to confirm if this is true or not. Thanks, Bryan > -----Original Message----- > From: Valek, Andrej <andrej.valek@siemens.com> > Sent: Saturday, January 22, 2022 2:26 AM > To: openembedded-core@lists.openembedded.org; Bryan Evenson > <bevenson@melinkcorp.com> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello again, > > Maybe a general question. Is it working in current master? I do not want to > brake dunfell, just applying something, which will create a lot of divergence. > > Cheers, > Andrej > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > Andrej, > > > > Thanks for the response. This is an attempt to fix a problem I am > > having with automated firmware upgrades for my system. I am using > > opkg for a package manager; not sure if the same problem exists with > > other package managers. I run into problems whenever busybox is one > > of the packages that needs to get updated. I enact my distribution > > firmware upgrade by calling "opkg --download-only upgrade; opkg > > upgrade". What I see happen is: > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > common applets in a temporary directory and exports a path to that > > directory. It might also setup a temporary alternative to /bin/sh if > > it is the last shell. > > 2. After the remove stage, the busybox binary is gone. The softlinks > > created in the prerm stage are useless since they point to binary that > > no longer exists. > > 3. opkg continues with upgrade on other packages which may depend on a > > command provided by busybox in a prerm, postrm, preinst or postinst > > script. These upgrades then fail since the commands are no longer > > available. > > 4. The busybox upgrade completes, which may or may not complete > > successfully. For what I am attempting, I am upgrading my system from > > the morty branch to dunfell. I have util-linux on my system which > > shares some alternatives with busybox. The util-linux upgrade fails > > because it needs some busybox applets during its upgrade process. > > Then the busybox upgrade fails because the final update- alternatives > > doesn’t work; some files still exist that util-linux couldn't remove > > during its upgrade that clash with busybox's alternatives. > > > > After trying several ways to get my upgrade to work, I landed on the > > approach below. I'm creating a temporary directory and copying the > > busybox binary and the busybox.links files to that directory. I then > > install an alternative for every applet for busybox listed in all of > > its busybox.links files that points to the temporary copy of the > > busybox binary. This means that any package that uses a busybox > > applet during its install process should still work. Then during the > > postinst step I am removing all the temporary alternative links. I > > use the temporary busybox.links files for removing the alternative > > links in case the upgraded busybox is now configured with a different > > set of applets. > > > > This is a heavy handed approach, and it does extend the upgrade > > process for me by a few minutes since it runs through update- > > alternatives for busybox two more times. But, the approach works for > > me and I think would be more resilient than past approaches. I tried > > to mimic the existing code in my additions. If a more widescale > > rewrite makes sense than that works for me also. > > > > Thanks, > > Bryan > > > > > > > -----Original Message----- > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > Sent: Friday, January 21, 2022 9:01 AM > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > <bevenson@melinkcorp.com> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hi Bryan, > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you trying > > > to upgrade the busybox on demand? If yes, that is not a good idea. > > > > > > I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" > > > and > > > creating a custom locks is not a good at all. > > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > All, > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there is > > > > a better way to do this. It still seems a little messy and could > > > > probably be simplified for the same effect. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > -----Original Message----- > > > > > From: Bryan Evenson > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > To: openembedded-core@lists.openembedded.org > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a major > > > > > distribution upgrade and all packages need to be updated. > > > > > Success > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by adding > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > However, if busybox is not the only shell present and the other > > > > > shells are upgrading, it may then be possible that all shells > > > > > will be removed during the upgrade process. > > > > > > > > > > Commit [2] creates temporary symbolic links for all the busybox > > > > > links during busybox's postinst step. However, this is too late > > > > > in the process as some packages attempt to use 'rm' and 'sed' > > > > > after > > > > > update-alternatives removes the old links and prior to when > > > > > busybox's postinst step runs. > > > > > > > > > > This fix is similar to [2] but runs during the preinst step. For > > > > > opkg, this is the first step that is guaranteed to run from the > > > > > new package (prerm is run from the old package) and will > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > systems. > > > > > > > > > > Copies the existing busybox binary and the busybox.links files > > > > > to a temporary directory and then creates alternative links for > > > > > all installed busybox commands. The temporary links and > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > RFC: This works for me, but there may be room for improvement. I > > > > > don't know if the current pkg_prerm steps are necessary anymore. > > > > > However, in > > > > > my testing I did need the links for update-alternatives to work > > > > > in the preinst step. I am also not certain if the > > > > > populate_packages_updatealternatives_append > > > > > step is necessary anymore. I have also only tested this fix on > > > > > dunfell, as I don't have a working image based on master yet. > > > > > It > > > > > may be more appropriate for this to go to master and then be > > > > > backported to dunfell, but I would need assistance in testing. > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > a > > > > > [2] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > d2 > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > > --- > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > ++++++++++++++++++++++++++- > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > b/meta/recipes- > > > > > core/busybox/busybox.inc index e0522be729..c85402411b 100644 > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > - # If busybox exists in the remove directory it is > > > > > because it was the only shell left. > > > > > if [ "x$D" = "x" ] ; then > > > > > + # If busybox exists in the remove directory it is > > > > > because it was the only > > > > > shell left. > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > update-alternatives --remove sh $BUSYBOX > > > > > - rm -f $BUSYBOX > > > > > fi > > > > > + # Remove the temporary alternatives > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > > > > + if [ "$busybox_preinstdir" != > > > > > '/tmp/busyboxpreinst- > > > > > *' ] ; then > > > > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > + if [ -e $BUSYBOX ] ; then > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > + while read link; do > > > > > + update-alternatives --remove > > > > > $($BUSYBOX basename > > > > > $link) $BUSYBOX > > > > > + done < > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > + fi > > > > > + done > > > > > fi > > > > > } > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > fi > > > > > } > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > + # Create a temporary copy the busybox binary and the > > > > > links > > > > > files. Then, > > > > > + # install an alternative link for all the links. > > > > > Other > > > > > packages use these > > > > > + # commands during their upgrade process. This ensures > > > > > the > > > > > links are > > > > > available > > > > > + # to all the other packages. We do this in the > > > > > preinst > > > > > step because it is > > > > > + # the first step guaranteed to be used from the new > > > > > package. The > > > > > prerm is > > > > > + # used from the old package. Placing this here > > > > > ensures it > > > > > runs on > > > > > upgrade even > > > > > + # on older systems. > > > > > + > > > > > + if [ "x$D" = "x" ] ; then > > > > > + # update-alternatives may need the links from > > > > > commands > > > > > added in > > > > > the prerm step > > > > > + # to operate. Make sure we can get to that path. > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > > > > then > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > + fi > > > > > + done > > > > > + > > > > > + # Create a temporary directory for the busybox > > > > > binary > > > > > and the link lists > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > /tmp/busyboxpreinst- > > > > > XXXXXX` > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > + > > > > > + # Go through all the links and install an > > > > > alternative > > > > > that points to the > > > > > temporary > > > > > + # busybox binary. > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; > > > > > then > > > > > + $BUSYBOX cp > > > > > ${sysconfdir}/busybox.links$suffix > > > > > $BUSYBOX_TMP_DIR > > > > > + while read link; do > > > > > + update-alternatives --install $link > > > > > $($BUSYBOX basename $link) > > > > > $BUSYBOX_TMP_LOC 1 > > > > > + done < > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > +} > > > > > + > > > > > pkg_postrm_${PN} () { > > > > > # Add path to remove dir in case we removed our only > > > > > grep > > > > > if [ "x$D" = "x" ] ; then > > > > > -- > > > > > 2.17.1 > >
Andrej, > -----Original Message----- > From: Valek, Andrej <andrej.valek@siemens.com> > Sent: Saturday, January 22, 2022 2:26 AM > To: openembedded-core@lists.openembedded.org; Bryan Evenson > <bevenson@melinkcorp.com> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello again, > > Maybe a general question. Is it working in current master? I do not want to > brake dunfell, just applying something, which will create a lot of divergence. > I wasn't able to build an image based on the latest master as of Monday. From the error messages I think it had something to do with the addition of setuptools3-base that wasn't complete yet. I was able to build and run an image based on the latest honister branch. The master branch has four commits for busybox that honsiter does not have. In those four commits, I see no changes to any of the installation steps. Based on this information, I'd say its safe to say that testing on the latest honister branch should confirm whether the problem is still present or not. To test, I first built my image and directly programmed it on my hardware. My image is based upon core-image-minimal, uses sysvinit for an init system and opkg for a package manager. I have also added bash, cronie, dhcpcd and util-linux to my image; I've added others but these seem to produce the most obvious errors. After programming the image on my hardware, I added PR="r1" to the gcc recipe. This forced all the packages to get rebuilt and increment the version number (if anyone knows a simpler way to accomplish this, let me know). I then created a package feed with this update and attempted an upgrade. To upgrade, I did: opkg update opkg --download-only upgrade opkg upgrade > full_upgrade.txt 2>&1 & I download all the packages to be upgraded first. I then sent the upgrade process to the background so I could check other things during the upgrade process. During the upgrade, I would occasionally execute "ls -l" to verify that the file full_upgrade.txt existed and was still growing. About a minute into the upgrade process when I executed "ls -l" I got the error message: ls: not found From that point on until upgrade was complete I had to execute "busybox ls -l" to see the current directory file list. After upgrade was complete, I checked full_upgrade.txt for error messages. I saw the following: /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 times) /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not found /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found /etc/init.d/udev: line 74: start-stop-daemon: not found /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found In this case, a lot of alternatives and init scripts may not be setup correctly because either the old ones were not removed or the new ones were not installed. Prior to my patch below, I saw similar errors when upgrading my image from a build based off the morty branch to one based off dunfell (and my system was not operational after the upgrade). After my patch, I performed the same upgrade with no errors and my system was fully operational. I'm up for any thoughts on a different approach that accomplishes the same goal. Otherwise, I'm going to work with what I have and update my changes so they patch cleanly on master. Thanks, Bryan > Cheers, > Andrej > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > Andrej, > > > > Thanks for the response. This is an attempt to fix a problem I am > > having with automated firmware upgrades for my system. I am using > > opkg for a package manager; not sure if the same problem exists with > > other package managers. I run into problems whenever busybox is one > > of the packages that needs to get updated. I enact my distribution > > firmware upgrade by calling "opkg --download-only upgrade; opkg > > upgrade". What I see happen is: > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > common applets in a temporary directory and exports a path to that > > directory. It might also setup a temporary alternative to /bin/sh if > > it is the last shell. > > 2. After the remove stage, the busybox binary is gone. The softlinks > > created in the prerm stage are useless since they point to binary that > > no longer exists. > > 3. opkg continues with upgrade on other packages which may depend on a > > command provided by busybox in a prerm, postrm, preinst or postinst > > script. These upgrades then fail since the commands are no longer > > available. > > 4. The busybox upgrade completes, which may or may not complete > > successfully. For what I am attempting, I am upgrading my system from > > the morty branch to dunfell. I have util-linux on my system which > > shares some alternatives with busybox. The util-linux upgrade fails > > because it needs some busybox applets during its upgrade process. > > Then the busybox upgrade fails because the final update- alternatives > > doesn’t work; some files still exist that util-linux couldn't remove > > during its upgrade that clash with busybox's alternatives. > > > > After trying several ways to get my upgrade to work, I landed on the > > approach below. I'm creating a temporary directory and copying the > > busybox binary and the busybox.links files to that directory. I then > > install an alternative for every applet for busybox listed in all of > > its busybox.links files that points to the temporary copy of the > > busybox binary. This means that any package that uses a busybox > > applet during its install process should still work. Then during the > > postinst step I am removing all the temporary alternative links. I > > use the temporary busybox.links files for removing the alternative > > links in case the upgraded busybox is now configured with a different > > set of applets. > > > > This is a heavy handed approach, and it does extend the upgrade > > process for me by a few minutes since it runs through update- > > alternatives for busybox two more times. But, the approach works for > > me and I think would be more resilient than past approaches. I tried > > to mimic the existing code in my additions. If a more widescale > > rewrite makes sense than that works for me also. > > > > Thanks, > > Bryan > > > > > > > -----Original Message----- > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > Sent: Friday, January 21, 2022 9:01 AM > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > <bevenson@melinkcorp.com> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hi Bryan, > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you trying > > > to upgrade the busybox on demand? If yes, that is not a good idea. > > > > > > I'm little bit scary about doing "export PATH=$busybox_rmdir:$PATH" > > > and > > > creating a custom locks is not a good at all. > > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > All, > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there is > > > > a better way to do this. It still seems a little messy and could > > > > probably be simplified for the same effect. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > -----Original Message----- > > > > > From: Bryan Evenson > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > To: openembedded-core@lists.openembedded.org > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a major > > > > > distribution upgrade and all packages need to be updated. > > > > > Success > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by adding > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > However, if busybox is not the only shell present and the other > > > > > shells are upgrading, it may then be possible that all shells > > > > > will be removed during the upgrade process. > > > > > > > > > > Commit [2] creates temporary symbolic links for all the busybox > > > > > links during busybox's postinst step. However, this is too late > > > > > in the process as some packages attempt to use 'rm' and 'sed' > > > > > after > > > > > update-alternatives removes the old links and prior to when > > > > > busybox's postinst step runs. > > > > > > > > > > This fix is similar to [2] but runs during the preinst step. For > > > > > opkg, this is the first step that is guaranteed to run from the > > > > > new package (prerm is run from the old package) and will > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > systems. > > > > > > > > > > Copies the existing busybox binary and the busybox.links files > > > > > to a temporary directory and then creates alternative links for > > > > > all installed busybox commands. The temporary links and > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > RFC: This works for me, but there may be room for improvement. I > > > > > don't know if the current pkg_prerm steps are necessary anymore. > > > > > However, in > > > > > my testing I did need the links for update-alternatives to work > > > > > in the preinst step. I am also not certain if the > > > > > populate_packages_updatealternatives_append > > > > > step is necessary anymore. I have also only tested this fix on > > > > > dunfell, as I don't have a working image based on master yet. > > > > > It > > > > > may be more appropriate for this to go to master and then be > > > > > backported to dunfell, but I would need assistance in testing. > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > a > > > > > [2] https://git.openembedded.org/openembedded- > > > > > core/commit/meta/recipes- > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > d2 > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > > --- > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > ++++++++++++++++++++++++++- > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > b/meta/recipes- > > > > > core/busybox/busybox.inc index e0522be729..c85402411b 100644 > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > - # If busybox exists in the remove directory it is > > > > > because it was the only shell left. > > > > > if [ "x$D" = "x" ] ; then > > > > > + # If busybox exists in the remove directory it is > > > > > because it was the only > > > > > shell left. > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > update-alternatives --remove sh $BUSYBOX > > > > > - rm -f $BUSYBOX > > > > > fi > > > > > + # Remove the temporary alternatives > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst-*; do > > > > > + if [ "$busybox_preinstdir" != > > > > > '/tmp/busyboxpreinst- > > > > > *' ] ; then > > > > > + BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > + if [ -e $BUSYBOX ] ; then > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > + while read link; do > > > > > + update-alternatives --remove > > > > > $($BUSYBOX basename > > > > > $link) $BUSYBOX > > > > > + done < > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > + fi > > > > > + done > > > > > fi > > > > > } > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > fi > > > > > } > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > + # Create a temporary copy the busybox binary and the > > > > > links > > > > > files. Then, > > > > > + # install an alternative link for all the links. > > > > > Other > > > > > packages use these > > > > > + # commands during their upgrade process. This ensures > > > > > the > > > > > links are > > > > > available > > > > > + # to all the other packages. We do this in the > > > > > preinst > > > > > step because it is > > > > > + # the first step guaranteed to be used from the new > > > > > package. The > > > > > prerm is > > > > > + # used from the old package. Placing this here > > > > > ensures it > > > > > runs on > > > > > upgrade even > > > > > + # on older systems. > > > > > + > > > > > + if [ "x$D" = "x" ] ; then > > > > > + # update-alternatives may need the links from > > > > > commands > > > > > added in > > > > > the prerm step > > > > > + # to operate. Make sure we can get to that path. > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; > > > > > then > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > + fi > > > > > + done > > > > > + > > > > > + # Create a temporary directory for the busybox > > > > > binary > > > > > and the link lists > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > /tmp/busyboxpreinst- > > > > > XXXXXX` > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > + > > > > > + # Go through all the links and install an > > > > > alternative > > > > > that points to the > > > > > temporary > > > > > + # busybox binary. > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix ] ; > > > > > then > > > > > + $BUSYBOX cp > > > > > ${sysconfdir}/busybox.links$suffix > > > > > $BUSYBOX_TMP_DIR > > > > > + while read link; do > > > > > + update-alternatives --install $link > > > > > $($BUSYBOX basename $link) > > > > > $BUSYBOX_TMP_LOC 1 > > > > > + done < > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > + fi > > > > > + done > > > > > + fi > > > > > +} > > > > > + > > > > > pkg_postrm_${PN} () { > > > > > # Add path to remove dir in case we removed our only > > > > > grep > > > > > if [ "x$D" = "x" ] ; then > > > > > -- > > > > > 2.17.1 > >
Hello Bryan, So looks like, there is some kind of problem. Was you able to run the busybox command after upgrade like, "busybox ls /" ? - If yes, there is a problem, that update alternatives hasn't been processed correctly. Try direct command after reboot, if possible - If no, lets continue with patch creation based on the master branch Cheers, Andrej On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > Andrej, > > > -----Original Message----- > > From: Valek, Andrej <andrej.valek@siemens.com> > > Sent: Saturday, January 22, 2022 2:26 AM > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > <bevenson@melinkcorp.com> > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > busybox > > links during install > > > > Hello again, > > > > Maybe a general question. Is it working in current master? I do not > > want to > > brake dunfell, just applying something, which will create a lot of > > divergence. > > > > I wasn't able to build an image based on the latest master as of > Monday. From the error messages I think it had something to do with > the addition of setuptools3-base that wasn't complete yet. I was > able to build and run an image based on the latest honister branch. > The master branch has four commits for busybox that honsiter does not > have. In those four commits, I see no changes to any of the > installation steps. Based on this information, I'd say its safe to > say that testing on the latest honister branch should confirm whether > the problem is still present or not. > > To test, I first built my image and directly programmed it on my > hardware. My image is based upon core-image-minimal, uses sysvinit > for an init system and opkg for a package manager. I have also added > bash, cronie, dhcpcd and util-linux to my image; I've added others > but these seem to produce the most obvious errors. > > After programming the image on my hardware, I added PR="r1" to the > gcc recipe. This forced all the packages to get rebuilt and > increment the version number (if anyone knows a simpler way to > accomplish this, let me know). I then created a package feed with > this update and attempted an upgrade. > > To upgrade, I did: > opkg update > opkg --download-only upgrade > opkg upgrade > full_upgrade.txt 2>&1 & > > I download all the packages to be upgraded first. I then sent the > upgrade process to the background so I could check other things > during the upgrade process. > > During the upgrade, I would occasionally execute "ls -l" to verify > that the file full_upgrade.txt existed and was still growing. About > a minute into the upgrade process when I executed "ls -l" I got the > error message: > > ls: not found > > From that point on until upgrade was complete I had to execute > "busybox ls -l" to see the current directory file list. After > upgrade was complete, I checked full_upgrade.txt for error messages. > I saw the following: > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > times) > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > found > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > /etc/init.d/udev: line 74: start-stop-daemon: not found > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > In this case, a lot of alternatives and init scripts may not be setup > correctly because either the old ones were not removed or the new > ones were not installed. Prior to my patch below, I saw similar > errors when upgrading my image from a build based off the morty > branch to one based off dunfell (and my system was not operational > after the upgrade). After my patch, I performed the same upgrade > with no errors and my system was fully operational. > > I'm up for any thoughts on a different approach that accomplishes the > same goal. Otherwise, I'm going to work with what I have and update > my changes so they patch cleanly on master. > > Thanks, > Bryan > > > Cheers, > > Andrej > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > Andrej, > > > > > > Thanks for the response. This is an attempt to fix a problem I > > > am > > > having with automated firmware upgrades for my system. I am > > > using > > > opkg for a package manager; not sure if the same problem exists > > > with > > > other package managers. I run into problems whenever busybox is > > > one > > > of the packages that needs to get updated. I enact my > > > distribution > > > firmware upgrade by calling "opkg --download-only upgrade; opkg > > > upgrade". What I see happen is: > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for > > > some > > > common applets in a temporary directory and exports a path to > > > that > > > directory. It might also setup a temporary alternative to > > > /bin/sh if > > > it is the last shell. > > > 2. After the remove stage, the busybox binary is gone. The > > > softlinks > > > created in the prerm stage are useless since they point to binary > > > that > > > no longer exists. > > > 3. opkg continues with upgrade on other packages which may depend > > > on a > > > command provided by busybox in a prerm, postrm, preinst or > > > postinst > > > script. These upgrades then fail since the commands are no > > > longer > > > available. > > > 4. The busybox upgrade completes, which may or may not complete > > > successfully. For what I am attempting, I am upgrading my system > > > from > > > the morty branch to dunfell. I have util-linux on my system > > > which > > > shares some alternatives with busybox. The util-linux upgrade > > > fails > > > because it needs some busybox applets during its upgrade process. > > > Then the busybox upgrade fails because the final update- > > > alternatives > > > doesn’t work; some files still exist that util-linux couldn't > > > remove > > > during its upgrade that clash with busybox's alternatives. > > > > > > After trying several ways to get my upgrade to work, I landed on > > > the > > > approach below. I'm creating a temporary directory and copying > > > the > > > busybox binary and the busybox.links files to that directory. I > > > then > > > install an alternative for every applet for busybox listed in all > > > of > > > its busybox.links files that points to the temporary copy of the > > > busybox binary. This means that any package that uses a busybox > > > applet during its install process should still work. Then during > > > the > > > postinst step I am removing all the temporary alternative links. > > > I > > > use the temporary busybox.links files for removing the > > > alternative > > > links in case the upgraded busybox is now configured with a > > > different > > > set of applets. > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > process for me by a few minutes since it runs through update- > > > alternatives for busybox two more times. But, the approach works > > > for > > > me and I think would be more resilient than past approaches. I > > > tried > > > to mimic the existing code in my additions. If a more widescale > > > rewrite makes sense than that works for me also. > > > > > > Thanks, > > > Bryan > > > > > > > > > > -----Original Message----- > > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > <bevenson@melinkcorp.com> > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > busybox links during install > > > > > > > > Hi Bryan, > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > trying > > > > to upgrade the busybox on demand? If yes, that is not a good > > > > idea. > > > > > > > > I'm little bit scary about doing "export > > > > PATH=$busybox_rmdir:$PATH" > > > > and > > > > creating a custom locks is not a good at all. > > > > > > > > Cheers, > > > > Andrej > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > All, > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling > > > > > there is > > > > > a better way to do this. It still seems a little messy and > > > > > could > > > > > probably be simplified for the same effect. > > > > > > > > > > Thanks, > > > > > Bryan > > > > > > > > > > > -----Original Message----- > > > > > > From: Bryan Evenson > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > To: openembedded-core@lists.openembedded.org > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > busybox links during install > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > major > > > > > > distribution upgrade and all packages need to be updated. > > > > > > Success > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > adding > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > However, if busybox is not the only shell present and the > > > > > > other > > > > > > shells are upgrading, it may then be possible that all > > > > > > shells > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > busybox > > > > > > links during busybox's postinst step. However, this is too > > > > > > late > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > 'sed' > > > > > > after > > > > > > update-alternatives removes the old links and prior to when > > > > > > busybox's postinst step runs. > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > step. For > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > the > > > > > > new package (prerm is run from the old package) and will > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > systems. > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > files > > > > > > to a temporary directory and then creates alternative links > > > > > > for > > > > > > all installed busybox commands. The temporary links and > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > improvement. I > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > anymore. > > > > > > However, in > > > > > > my testing I did need the links for update-alternatives to > > > > > > work > > > > > > in the preinst step. I am also not certain if the > > > > > > populate_packages_updatealternatives_append > > > > > > step is necessary anymore. I have also only tested this > > > > > > fix on > > > > > > dunfell, as I don't have a working image based on master > > > > > > yet. > > > > > > It > > > > > > may be more appropriate for this to go to master and then > > > > > > be > > > > > > backported to dunfell, but I would need assistance in > > > > > > testing. > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > a > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > d2 > > > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > > > --- > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > ++++++++++++++++++++++++++- > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > b/meta/recipes- > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > 100644 > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > - # If busybox exists in the remove directory it is > > > > > > because it was the only shell left. > > > > > > if [ "x$D" = "x" ] ; then > > > > > > + # If busybox exists in the remove directory it > > > > > > is > > > > > > because it was the only > > > > > > shell left. > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > - rm -f $BUSYBOX > > > > > > fi > > > > > > + # Remove the temporary alternatives > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > *; do > > > > > > + if [ "$busybox_preinstdir" != > > > > > > '/tmp/busyboxpreinst- > > > > > > *' ] ; then > > > > > > + > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > do > > > > > > + if [ -e > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > + while read link; do > > > > > > + update-alternatives -- > > > > > > remove > > > > > > $($BUSYBOX basename > > > > > > $link) $BUSYBOX > > > > > > + done < > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > + fi > > > > > > + done > > > > > > + fi > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > + fi > > > > > > + done > > > > > > fi > > > > > > } > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > fi > > > > > > } > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > + # Create a temporary copy the busybox binary and > > > > > > the > > > > > > links > > > > > > files. Then, > > > > > > + # install an alternative link for all the links. > > > > > > Other > > > > > > packages use these > > > > > > + # commands during their upgrade process. This > > > > > > ensures > > > > > > the > > > > > > links are > > > > > > available > > > > > > + # to all the other packages. We do this in the > > > > > > preinst > > > > > > step because it is > > > > > > + # the first step guaranteed to be used from the > > > > > > new > > > > > > package. The > > > > > > prerm is > > > > > > + # used from the old package. Placing this here > > > > > > ensures it > > > > > > runs on > > > > > > upgrade even > > > > > > + # on older systems. > > > > > > + > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > + # update-alternatives may need the links from > > > > > > commands > > > > > > added in > > > > > > the prerm step > > > > > > + # to operate. Make sure we can get to that > > > > > > path. > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > ] ; > > > > > > then > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > + fi > > > > > > + done > > > > > > + > > > > > > + # Create a temporary directory for the busybox > > > > > > binary > > > > > > and the link lists > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > /tmp/busyboxpreinst- > > > > > > XXXXXX` > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > + > > > > > > + # Go through all the links and install an > > > > > > alternative > > > > > > that points to the > > > > > > temporary > > > > > > + # busybox binary. > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > ] ; > > > > > > then > > > > > > + $BUSYBOX cp > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > $BUSYBOX_TMP_DIR > > > > > > + while read link; do > > > > > > + update-alternatives --install $link > > > > > > $($BUSYBOX basename $link) > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > + done < > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > + fi > > > > > > + done > > > > > > + fi > > > > > > +} > > > > > > + > > > > > > pkg_postrm_${PN} () { > > > > > > # Add path to remove dir in case we removed our > > > > > > only > > > > > > grep > > > > > > if [ "x$D" = "x" ] ; then > > > > > > -- > > > > > > 2.17.1 > > > >
Andrej, > -----Original Message----- > From: Valek, Andrej <andrej.valek@siemens.com> > Sent: Friday, January 28, 2022 6:39 AM > To: openembedded-core@lists.openembedded.org; Bryan Evenson > <bevenson@melinkcorp.com> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello Bryan, > > So looks like, there is some kind of problem. > > Was you able to run the busybox command after upgrade like, "busybox ls /" > ? In this particular case, after upgrade both "busybox ls" and "ls" worked. However, that was partly because for this particular case the packages were identical but I just forced a version number change. None of the files were actually different after the upgrade. Failure to replace files didn't affect operation. When I was upgrading my image from the morty release to the dunfell release, there was a change in util-linux that conflicted with busybox if they both didn't upgrade cleanly and the update-alternatives installation step would fail for both. In that case "busybox ls" would still work but "ls" would not. Repeated upgrade attempts didn't fix it. I had to manually delete certain files that were not deleted during the initial upgrade attempt and then upgrade packages manually in a particular order to get the system fully operational again. > - If yes, there is a problem, that update alternatives hasn't been processed > correctly. Try direct command after reboot, if possible > - If no, lets continue with patch creation based on the master branch I will continue work on a patch based on master. I'm going to try testing a few other cases so I don't anticipate having a patch ready until early next week. Thanks, Bryan > > Cheers, > Andrej > > On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > > Andrej, > > > > > -----Original Message----- > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > Sent: Saturday, January 22, 2022 2:26 AM > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > <bevenson@melinkcorp.com> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hello again, > > > > > > Maybe a general question. Is it working in current master? I do not > > > want to brake dunfell, just applying something, which will create a > > > lot of divergence. > > > > > > > I wasn't able to build an image based on the latest master as of > > Monday. From the error messages I think it had something to do with > > the addition of setuptools3-base that wasn't complete yet. I was able > > to build and run an image based on the latest honister branch. > > The master branch has four commits for busybox that honsiter does not > > have. In those four commits, I see no changes to any of the > > installation steps. Based on this information, I'd say its safe to > > say that testing on the latest honister branch should confirm whether > > the problem is still present or not. > > > > To test, I first built my image and directly programmed it on my > > hardware. My image is based upon core-image-minimal, uses sysvinit > > for an init system and opkg for a package manager. I have also added > > bash, cronie, dhcpcd and util-linux to my image; I've added others but > > these seem to produce the most obvious errors. > > > > After programming the image on my hardware, I added PR="r1" to the gcc > > recipe. This forced all the packages to get rebuilt and increment the > > version number (if anyone knows a simpler way to accomplish this, let > > me know). I then created a package feed with this update and > > attempted an upgrade. > > > > To upgrade, I did: > > opkg update > > opkg --download-only upgrade > > opkg upgrade > full_upgrade.txt 2>&1 & > > > > I download all the packages to be upgraded first. I then sent the > > upgrade process to the background so I could check other things during > > the upgrade process. > > > > During the upgrade, I would occasionally execute "ls -l" to verify > > that the file full_upgrade.txt existed and was still growing. About a > > minute into the upgrade process when I executed "ls -l" I got the > > error message: > > > > ls: not found > > > > From that point on until upgrade was complete I had to execute > > "busybox ls -l" to see the current directory file list. After upgrade > > was complete, I checked full_upgrade.txt for error messages. > > I saw the following: > > > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > > times) > > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > > found > > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > > /etc/init.d/udev: line 74: start-stop-daemon: not found > > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > > > In this case, a lot of alternatives and init scripts may not be setup > > correctly because either the old ones were not removed or the new ones > > were not installed. Prior to my patch below, I saw similar errors > > when upgrading my image from a build based off the morty branch to one > > based off dunfell (and my system was not operational after the > > upgrade). After my patch, I performed the same upgrade with no errors > > and my system was fully operational. > > > > I'm up for any thoughts on a different approach that accomplishes the > > same goal. Otherwise, I'm going to work with what I have and update > > my changes so they patch cleanly on master. > > > > Thanks, > > Bryan > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > > Andrej, > > > > > > > > Thanks for the response. This is an attempt to fix a problem I am > > > > having with automated firmware upgrades for my system. I am using > > > > opkg for a package manager; not sure if the same problem exists > > > > with other package managers. I run into problems whenever busybox > > > > is one of the packages that needs to get updated. I enact my > > > > distribution firmware upgrade by calling "opkg --download-only > > > > upgrade; opkg upgrade". What I see happen is: > > > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > > > common applets in a temporary directory and exports a path to that > > > > directory. It might also setup a temporary alternative to /bin/sh > > > > if it is the last shell. > > > > 2. After the remove stage, the busybox binary is gone. The > > > > softlinks created in the prerm stage are useless since they point > > > > to binary that no longer exists. > > > > 3. opkg continues with upgrade on other packages which may depend > > > > on a command provided by busybox in a prerm, postrm, preinst or > > > > postinst script. These upgrades then fail since the commands are > > > > no longer available. > > > > 4. The busybox upgrade completes, which may or may not complete > > > > successfully. For what I am attempting, I am upgrading my system > > > > from the morty branch to dunfell. I have util-linux on my system > > > > which shares some alternatives with busybox. The util-linux > > > > upgrade fails because it needs some busybox applets during its > > > > upgrade process. > > > > Then the busybox upgrade fails because the final update- > > > > alternatives doesn’t work; some files still exist that util-linux > > > > couldn't remove during its upgrade that clash with busybox's > > > > alternatives. > > > > > > > > After trying several ways to get my upgrade to work, I landed on > > > > the approach below. I'm creating a temporary directory and > > > > copying the busybox binary and the busybox.links files to that > > > > directory. I then install an alternative for every applet for > > > > busybox listed in all of its busybox.links files that points to > > > > the temporary copy of the busybox binary. This means that any > > > > package that uses a busybox applet during its install process > > > > should still work. Then during the postinst step I am removing > > > > all the temporary alternative links. > > > > I > > > > use the temporary busybox.links files for removing the alternative > > > > links in case the upgraded busybox is now configured with a > > > > different set of applets. > > > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > > process for me by a few minutes since it runs through update- > > > > alternatives for busybox two more times. But, the approach works > > > > for me and I think would be more resilient than past approaches. > > > > I tried to mimic the existing code in my additions. If a more > > > > widescale rewrite makes sense than that works for me also. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > > <bevenson@melinkcorp.com> > > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Hi Bryan, > > > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > > trying to upgrade the busybox on demand? If yes, that is not a > > > > > good idea. > > > > > > > > > > I'm little bit scary about doing "export > > > > > PATH=$busybox_rmdir:$PATH" > > > > > and > > > > > creating a custom locks is not a good at all. > > > > > > > > > > Cheers, > > > > > Andrej > > > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > > All, > > > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there > > > > > > is a better way to do this. It still seems a little messy and > > > > > > could probably be simplified for the same effect. > > > > > > > > > > > > Thanks, > > > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Bryan Evenson > > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > > To: openembedded-core@lists.openembedded.org > > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > > busybox links during install > > > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > > major > > > > > > > distribution upgrade and all packages need to be updated. > > > > > > > Success > > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > > adding > > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > > However, if busybox is not the only shell present and the > > > > > > > other > > > > > > > shells are upgrading, it may then be possible that all > > > > > > > shells > > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > > busybox > > > > > > > links during busybox's postinst step. However, this is too > > > > > > > late > > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > > 'sed' > > > > > > > after > > > > > > > update-alternatives removes the old links and prior to when > > > > > > > busybox's postinst step runs. > > > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > > step. For > > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > > the > > > > > > > new package (prerm is run from the old package) and will > > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > > systems. > > > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > > files > > > > > > > to a temporary directory and then creates alternative links > > > > > > > for > > > > > > > all installed busybox commands. The temporary links and > > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > > improvement. I > > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > > anymore. > > > > > > > However, in > > > > > > > my testing I did need the links for update-alternatives to > > > > > > > work > > > > > > > in the preinst step. I am also not certain if the > > > > > > > populate_packages_updatealternatives_append > > > > > > > step is necessary anymore. I have also only tested this > > > > > > > fix on > > > > > > > dunfell, as I don't have a working image based on master > > > > > > > yet. > > > > > > > It > > > > > > > may be more appropriate for this to go to master and then > > > > > > > be > > > > > > > backported to dunfell, but I would need assistance in > > > > > > > testing. > > > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > > a > > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > > d2 > > > > > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > > > > --- > > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > > ++++++++++++++++++++++++++- > > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > > b/meta/recipes- > > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > > 100644 > > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > > - # If busybox exists in the remove directory it is > > > > > > > because it was the only shell left. > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > + # If busybox exists in the remove directory it > > > > > > > is > > > > > > > because it was the only > > > > > > > shell left. > > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > > - rm -f $BUSYBOX > > > > > > > fi > > > > > > > + # Remove the temporary alternatives > > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > > *; do > > > > > > > + if [ "$busybox_preinstdir" != > > > > > > > '/tmp/busyboxpreinst- > > > > > > > *' ] ; then > > > > > > > + > > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > > do > > > > > > > + if [ -e > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > > + while read link; do > > > > > > > + update-alternatives -- > > > > > > > remove > > > > > > > $($BUSYBOX basename > > > > > > > $link) $BUSYBOX > > > > > > > + done < > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > > + fi > > > > > > > + done > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > > + # Create a temporary copy the busybox binary and > > > > > > > the > > > > > > > links > > > > > > > files. Then, > > > > > > > + # install an alternative link for all the links. > > > > > > > Other > > > > > > > packages use these > > > > > > > + # commands during their upgrade process. This > > > > > > > ensures > > > > > > > the > > > > > > > links are > > > > > > > available > > > > > > > + # to all the other packages. We do this in the > > > > > > > preinst > > > > > > > step because it is > > > > > > > + # the first step guaranteed to be used from the > > > > > > > new > > > > > > > package. The > > > > > > > prerm is > > > > > > > + # used from the old package. Placing this here > > > > > > > ensures it > > > > > > > runs on > > > > > > > upgrade even > > > > > > > + # on older systems. > > > > > > > + > > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > > + # update-alternatives may need the links from > > > > > > > commands > > > > > > > added in > > > > > > > the prerm step > > > > > > > + # to operate. Make sure we can get to that > > > > > > > path. > > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > > ] ; > > > > > > > then > > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > > + fi > > > > > > > + done > > > > > > > + > > > > > > > + # Create a temporary directory for the busybox > > > > > > > binary > > > > > > > and the link lists > > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > > /tmp/busyboxpreinst- > > > > > > > XXXXXX` > > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > > + > > > > > > > + # Go through all the links and install an > > > > > > > alternative > > > > > > > that points to the > > > > > > > temporary > > > > > > > + # busybox binary. > > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > > ] ; > > > > > > > then > > > > > > > + $BUSYBOX cp > > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > > $BUSYBOX_TMP_DIR > > > > > > > + while read link; do > > > > > > > + update-alternatives --install $link > > > > > > > $($BUSYBOX basename $link) > > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > > + done < > > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > +} > > > > > > > + > > > > > > > pkg_postrm_${PN} () { > > > > > > > # Add path to remove dir in case we removed our > > > > > > > only > > > > > > > grep > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > -- > > > > > > > 2.17.1 > > > > > >
I have performed a major distribution upgrade in the past. This was with rpm which requires bash. I did not have issues with utilities required by the upgrade disappearing. I believe rpm installs the new file, and then removes old files. Does opkg remove the old binary before installing the new version? -Jate On Fri, Jan 28, 2022 at 8:12 AM Bryan Evenson <bevenson@melinkcorp.com> wrote: > Andrej, > > > -----Original Message----- > > From: Valek, Andrej <andrej.valek@siemens.com> > > Sent: Friday, January 28, 2022 6:39 AM > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > <bevenson@melinkcorp.com> > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > > links during install > > > > Hello Bryan, > > > > So looks like, there is some kind of problem. > > > > Was you able to run the busybox command after upgrade like, "busybox ls > /" > > ? > > In this particular case, after upgrade both "busybox ls" and "ls" worked. > However, that was partly because for this particular case the packages were > identical but I just forced a version number change. None of the files > were actually different after the upgrade. Failure to replace files didn't > affect operation. When I was upgrading my image from the morty release to > the dunfell release, there was a change in util-linux that conflicted with > busybox if they both didn't upgrade cleanly and the update-alternatives > installation step would fail for both. In that case "busybox ls" would > still work but "ls" would not. Repeated upgrade attempts didn't fix it. I > had to manually delete certain files that were not deleted during the > initial upgrade attempt and then upgrade packages manually in a particular > order to get the system fully operational again. > > > - If yes, there is a problem, that update alternatives hasn't been > processed > > correctly. Try direct command after reboot, if possible > > - If no, lets continue with patch creation based on the master branch > > I will continue work on a patch based on master. I'm going to try testing > a few other cases so I don't anticipate having a patch ready until early > next week. > > Thanks, > Bryan > > > > > Cheers, > > Andrej > > > > On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > > > Andrej, > > > > > > > -----Original Message----- > > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > > Sent: Saturday, January 22, 2022 2:26 AM > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > <bevenson@melinkcorp.com> > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > busybox links during install > > > > > > > > Hello again, > > > > > > > > Maybe a general question. Is it working in current master? I do not > > > > want to brake dunfell, just applying something, which will create a > > > > lot of divergence. > > > > > > > > > > I wasn't able to build an image based on the latest master as of > > > Monday. From the error messages I think it had something to do with > > > the addition of setuptools3-base that wasn't complete yet. I was able > > > to build and run an image based on the latest honister branch. > > > The master branch has four commits for busybox that honsiter does not > > > have. In those four commits, I see no changes to any of the > > > installation steps. Based on this information, I'd say its safe to > > > say that testing on the latest honister branch should confirm whether > > > the problem is still present or not. > > > > > > To test, I first built my image and directly programmed it on my > > > hardware. My image is based upon core-image-minimal, uses sysvinit > > > for an init system and opkg for a package manager. I have also added > > > bash, cronie, dhcpcd and util-linux to my image; I've added others but > > > these seem to produce the most obvious errors. > > > > > > After programming the image on my hardware, I added PR="r1" to the gcc > > > recipe. This forced all the packages to get rebuilt and increment the > > > version number (if anyone knows a simpler way to accomplish this, let > > > me know). I then created a package feed with this update and > > > attempted an upgrade. > > > > > > To upgrade, I did: > > > opkg update > > > opkg --download-only upgrade > > > opkg upgrade > full_upgrade.txt 2>&1 & > > > > > > I download all the packages to be upgraded first. I then sent the > > > upgrade process to the background so I could check other things during > > > the upgrade process. > > > > > > During the upgrade, I would occasionally execute "ls -l" to verify > > > that the file full_upgrade.txt existed and was still growing. About a > > > minute into the upgrade process when I executed "ls -l" I got the > > > error message: > > > > > > ls: not found > > > > > > From that point on until upgrade was complete I had to execute > > > "busybox ls -l" to see the current directory file list. After upgrade > > > was complete, I checked full_upgrade.txt for error messages. > > > I saw the following: > > > > > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > > > times) > > > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > > > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > > > found > > > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > > > /etc/init.d/udev: line 74: start-stop-daemon: not found > > > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > > > > > In this case, a lot of alternatives and init scripts may not be setup > > > correctly because either the old ones were not removed or the new ones > > > were not installed. Prior to my patch below, I saw similar errors > > > when upgrading my image from a build based off the morty branch to one > > > based off dunfell (and my system was not operational after the > > > upgrade). After my patch, I performed the same upgrade with no errors > > > and my system was fully operational. > > > > > > I'm up for any thoughts on a different approach that accomplishes the > > > same goal. Otherwise, I'm going to work with what I have and update > > > my changes so they patch cleanly on master. > > > > > > Thanks, > > > Bryan > > > > > > > Cheers, > > > > Andrej > > > > > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > > > Andrej, > > > > > > > > > > Thanks for the response. This is an attempt to fix a problem I am > > > > > having with automated firmware upgrades for my system. I am using > > > > > opkg for a package manager; not sure if the same problem exists > > > > > with other package managers. I run into problems whenever busybox > > > > > is one of the packages that needs to get updated. I enact my > > > > > distribution firmware upgrade by calling "opkg --download-only > > > > > upgrade; opkg upgrade". What I see happen is: > > > > > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > > > > common applets in a temporary directory and exports a path to that > > > > > directory. It might also setup a temporary alternative to /bin/sh > > > > > if it is the last shell. > > > > > 2. After the remove stage, the busybox binary is gone. The > > > > > softlinks created in the prerm stage are useless since they point > > > > > to binary that no longer exists. > > > > > 3. opkg continues with upgrade on other packages which may depend > > > > > on a command provided by busybox in a prerm, postrm, preinst or > > > > > postinst script. These upgrades then fail since the commands are > > > > > no longer available. > > > > > 4. The busybox upgrade completes, which may or may not complete > > > > > successfully. For what I am attempting, I am upgrading my system > > > > > from the morty branch to dunfell. I have util-linux on my system > > > > > which shares some alternatives with busybox. The util-linux > > > > > upgrade fails because it needs some busybox applets during its > > > > > upgrade process. > > > > > Then the busybox upgrade fails because the final update- > > > > > alternatives doesn’t work; some files still exist that util-linux > > > > > couldn't remove during its upgrade that clash with busybox's > > > > > alternatives. > > > > > > > > > > After trying several ways to get my upgrade to work, I landed on > > > > > the approach below. I'm creating a temporary directory and > > > > > copying the busybox binary and the busybox.links files to that > > > > > directory. I then install an alternative for every applet for > > > > > busybox listed in all of its busybox.links files that points to > > > > > the temporary copy of the busybox binary. This means that any > > > > > package that uses a busybox applet during its install process > > > > > should still work. Then during the postinst step I am removing > > > > > all the temporary alternative links. > > > > > I > > > > > use the temporary busybox.links files for removing the alternative > > > > > links in case the upgraded busybox is now configured with a > > > > > different set of applets. > > > > > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > > > process for me by a few minutes since it runs through update- > > > > > alternatives for busybox two more times. But, the approach works > > > > > for me and I think would be more resilient than past approaches. > > > > > I tried to mimic the existing code in my additions. If a more > > > > > widescale rewrite makes sense than that works for me also. > > > > > > > > > > Thanks, > > > > > Bryan > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Valek, Andrej <andrej.valek@siemens.com> > > > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > > > To: openembedded-core@lists.openembedded.org; Bryan Evenson > > > > > > <bevenson@melinkcorp.com> > > > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > busybox links during install > > > > > > > > > > > > Hi Bryan, > > > > > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > > > trying to upgrade the busybox on demand? If yes, that is not a > > > > > > good idea. > > > > > > > > > > > > I'm little bit scary about doing "export > > > > > > PATH=$busybox_rmdir:$PATH" > > > > > > and > > > > > > creating a custom locks is not a good at all. > > > > > > > > > > > > Cheers, > > > > > > Andrej > > > > > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > > > All, > > > > > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there > > > > > > > is a better way to do this. It still seems a little messy and > > > > > > > could probably be simplified for the same effect. > > > > > > > > > > > > > > Thanks, > > > > > > > Bryan > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Bryan Evenson > > > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > > > To: openembedded-core@lists.openembedded.org > > > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > > > busybox links during install > > > > > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > > > major > > > > > > > > distribution upgrade and all packages need to be updated. > > > > > > > > Success > > > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > > > adding > > > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > > > However, if busybox is not the only shell present and the > > > > > > > > other > > > > > > > > shells are upgrading, it may then be possible that all > > > > > > > > shells > > > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > > > busybox > > > > > > > > links during busybox's postinst step. However, this is too > > > > > > > > late > > > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > > > 'sed' > > > > > > > > after > > > > > > > > update-alternatives removes the old links and prior to when > > > > > > > > busybox's postinst step runs. > > > > > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > > > step. For > > > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > > > the > > > > > > > > new package (prerm is run from the old package) and will > > > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > > > systems. > > > > > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > > > files > > > > > > > > to a temporary directory and then creates alternative links > > > > > > > > for > > > > > > > > all installed busybox commands. The temporary links and > > > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > > > improvement. I > > > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > > > anymore. > > > > > > > > However, in > > > > > > > > my testing I did need the links for update-alternatives to > > > > > > > > work > > > > > > > > in the preinst step. I am also not certain if the > > > > > > > > populate_packages_updatealternatives_append > > > > > > > > step is necessary anymore. I have also only tested this > > > > > > > > fix on > > > > > > > > dunfell, as I don't have a working image based on master > > > > > > > > yet. > > > > > > > > It > > > > > > > > may be more appropriate for this to go to master and then > > > > > > > > be > > > > > > > > backported to dunfell, but I would need assistance in > > > > > > > > testing. > > > > > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > > > a > > > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > > > d2 > > > > > > > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> > > > > > > > > --- > > > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > > > ++++++++++++++++++++++++++- > > > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > > > b/meta/recipes- > > > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > > > 100644 > > > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > > > - # If busybox exists in the remove directory it is > > > > > > > > because it was the only shell left. > > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > > + # If busybox exists in the remove directory it > > > > > > > > is > > > > > > > > because it was the only > > > > > > > > shell left. > > > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > > > - rm -f $BUSYBOX > > > > > > > > fi > > > > > > > > + # Remove the temporary alternatives > > > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > > > *; do > > > > > > > > + if [ "$busybox_preinstdir" != > > > > > > > > '/tmp/busyboxpreinst- > > > > > > > > *' ] ; then > > > > > > > > + > > > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > > > do > > > > > > > > + if [ -e > > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > > > + while read link; do > > > > > > > > + update-alternatives -- > > > > > > > > remove > > > > > > > > $($BUSYBOX basename > > > > > > > > $link) $BUSYBOX > > > > > > > > + done < > > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + fi > > > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > > > + fi > > > > > > > > + done > > > > > > > > fi > > > > > > > > } > > > > > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > > > fi > > > > > > > > } > > > > > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > > > + # Create a temporary copy the busybox binary and > > > > > > > > the > > > > > > > > links > > > > > > > > files. Then, > > > > > > > > + # install an alternative link for all the links. > > > > > > > > Other > > > > > > > > packages use these > > > > > > > > + # commands during their upgrade process. This > > > > > > > > ensures > > > > > > > > the > > > > > > > > links are > > > > > > > > available > > > > > > > > + # to all the other packages. We do this in the > > > > > > > > preinst > > > > > > > > step because it is > > > > > > > > + # the first step guaranteed to be used from the > > > > > > > > new > > > > > > > > package. The > > > > > > > > prerm is > > > > > > > > + # used from the old package. Placing this here > > > > > > > > ensures it > > > > > > > > runs on > > > > > > > > upgrade even > > > > > > > > + # on older systems. > > > > > > > > + > > > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > > > + # update-alternatives may need the links from > > > > > > > > commands > > > > > > > > added in > > > > > > > > the prerm step > > > > > > > > + # to operate. Make sure we can get to that > > > > > > > > path. > > > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > > > ] ; > > > > > > > > then > > > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + > > > > > > > > + # Create a temporary directory for the busybox > > > > > > > > binary > > > > > > > > and the link lists > > > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > > > /tmp/busyboxpreinst- > > > > > > > > XXXXXX` > > > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > > > + > > > > > > > > + # Go through all the links and install an > > > > > > > > alternative > > > > > > > > that points to the > > > > > > > > temporary > > > > > > > > + # busybox binary. > > > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > > > ] ; > > > > > > > > then > > > > > > > > + $BUSYBOX cp > > > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > > > $BUSYBOX_TMP_DIR > > > > > > > > + while read link; do > > > > > > > > + update-alternatives --install $link > > > > > > > > $($BUSYBOX basename $link) > > > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > > > + done < > > > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > > > + fi > > > > > > > > + done > > > > > > > > + fi > > > > > > > > +} > > > > > > > > + > > > > > > > > pkg_postrm_${PN} () { > > > > > > > > # Add path to remove dir in case we removed our > > > > > > > > only > > > > > > > > grep > > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > > -- > > > > > > > > 2.17.1 > > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#161078): > https://lists.openembedded.org/g/openembedded-core/message/161078 > Mute This Topic: https://lists.openembedded.org/mt/87919018/3616919 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > jatedev@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Jate, I don’t know when the binary is removed, but I see update-alternatives –remove for all the busybox links is in the prerm script and update-alternatives –install is in the postinst script. For opkg, I am witnessing the prerm stage run for busybox and other packages performing some stages of their upgrade prior to busybox completing its upgrade. It is possible that rpm operates differently. I have a patch that I will be sending out shortly. I’ve tested on honister by forcing all package versions to increment. I’ve also tested upgrading from an image based on the morty release up to the latest on honister. Both ways I upgrade cleanly with my patch. -Bryan From: Jate Sujjavanich <jatedev@gmail.com> Sent: Wednesday, February 2, 2022 10:59 AM To: Bryan Evenson <bevenson@melinkcorp.com> Cc: Valek, Andrej <andrej.valek@siemens.com>; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [dunfell][PATCH RFC] busybox.inc: Create temporary busybox links during install I have performed a major distribution upgrade in the past. This was with rpm which requires bash. I did not have issues with utilities required by the upgrade disappearing. I believe rpm installs the new file, and then removes old files. Does opkg remove the old binary before installing the new version? -Jate On Fri, Jan 28, 2022 at 8:12 AM Bryan Evenson <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> wrote: Andrej, > -----Original Message----- > From: Valek, Andrej <andrej.valek@siemens.com<mailto:andrej.valek@siemens.com>> > Sent: Friday, January 28, 2022 6:39 AM > To: openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; Bryan Evenson > <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary busybox > links during install > > Hello Bryan, > > So looks like, there is some kind of problem. > > Was you able to run the busybox command after upgrade like, "busybox ls /" > ? In this particular case, after upgrade both "busybox ls" and "ls" worked. However, that was partly because for this particular case the packages were identical but I just forced a version number change. None of the files were actually different after the upgrade. Failure to replace files didn't affect operation. When I was upgrading my image from the morty release to the dunfell release, there was a change in util-linux that conflicted with busybox if they both didn't upgrade cleanly and the update-alternatives installation step would fail for both. In that case "busybox ls" would still work but "ls" would not. Repeated upgrade attempts didn't fix it. I had to manually delete certain files that were not deleted during the initial upgrade attempt and then upgrade packages manually in a particular order to get the system fully operational again. > - If yes, there is a problem, that update alternatives hasn't been processed > correctly. Try direct command after reboot, if possible > - If no, lets continue with patch creation based on the master branch I will continue work on a patch based on master. I'm going to try testing a few other cases so I don't anticipate having a patch ready until early next week. Thanks, Bryan > > Cheers, > Andrej > > On Thu, 2022-01-27 at 14:39 +0000, Bryan Evenson wrote: > > Andrej, > > > > > -----Original Message----- > > > From: Valek, Andrej <andrej.valek@siemens.com<mailto:andrej.valek@siemens.com>> > > > Sent: Saturday, January 22, 2022 2:26 AM > > > To: openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; Bryan Evenson > > > <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > busybox links during install > > > > > > Hello again, > > > > > > Maybe a general question. Is it working in current master? I do not > > > want to brake dunfell, just applying something, which will create a > > > lot of divergence. > > > > > > > I wasn't able to build an image based on the latest master as of > > Monday. From the error messages I think it had something to do with > > the addition of setuptools3-base that wasn't complete yet. I was able > > to build and run an image based on the latest honister branch. > > The master branch has four commits for busybox that honsiter does not > > have. In those four commits, I see no changes to any of the > > installation steps. Based on this information, I'd say its safe to > > say that testing on the latest honister branch should confirm whether > > the problem is still present or not. > > > > To test, I first built my image and directly programmed it on my > > hardware. My image is based upon core-image-minimal, uses sysvinit > > for an init system and opkg for a package manager. I have also added > > bash, cronie, dhcpcd and util-linux to my image; I've added others but > > these seem to produce the most obvious errors. > > > > After programming the image on my hardware, I added PR="r1" to the gcc > > recipe. This forced all the packages to get rebuilt and increment the > > version number (if anyone knows a simpler way to accomplish this, let > > me know). I then created a package feed with this update and > > attempted an upgrade. > > > > To upgrade, I did: > > opkg update > > opkg --download-only upgrade > > opkg upgrade > full_upgrade.txt 2>&1 & > > > > I download all the packages to be upgraded first. I then sent the > > upgrade process to the background so I could check other things during > > the upgrade process. > > > > During the upgrade, I would occasionally execute "ls -l" to verify > > that the file full_upgrade.txt existed and was still growing. About a > > minute into the upgrade process when I executed "ls -l" I got the > > error message: > > > > ls: not found > > > > From that point on until upgrade was complete I had to execute > > "busybox ls -l" to see the current directory file list. After upgrade > > was complete, I checked full_upgrade.txt for error messages. > > I saw the following: > > > > /usr/bin/update-alternatives: line 1: sed: not found (occurred 102 > > times) > > /usr/sbin/update-rc.d: line 54: rm: not found (occurred 36 times) > > Stopping crond: /etc/init.d/crond: line 37: start-stop-daemon: not > > found > > /tmp/opkg-yBH1Ta/cronie-zHpym4/preinst: line 1: tr: not found > > /etc/init.d/udev: line 74: start-stop-daemon: not found > > /tmp/opkg-yBH1Ta/dhcpcd-c86ca9/preinst: line 1: tr: not found > > > > In this case, a lot of alternatives and init scripts may not be setup > > correctly because either the old ones were not removed or the new ones > > were not installed. Prior to my patch below, I saw similar errors > > when upgrading my image from a build based off the morty branch to one > > based off dunfell (and my system was not operational after the > > upgrade). After my patch, I performed the same upgrade with no errors > > and my system was fully operational. > > > > I'm up for any thoughts on a different approach that accomplishes the > > same goal. Otherwise, I'm going to work with what I have and update > > my changes so they patch cleanly on master. > > > > Thanks, > > Bryan > > > > > Cheers, > > > Andrej > > > > > > On Fri, 2022-01-21 at 15:02 +0000, Bryan Evenson wrote: > > > > Andrej, > > > > > > > > Thanks for the response. This is an attempt to fix a problem I am > > > > having with automated firmware upgrades for my system. I am using > > > > opkg for a package manager; not sure if the same problem exists > > > > with other package managers. I run into problems whenever busybox > > > > is one of the packages that needs to get updated. I enact my > > > > distribution firmware upgrade by calling "opkg --download-only > > > > upgrade; opkg upgrade". What I see happen is: > > > > > > > > 1. In the busybox pkg_prerm stage sets up some soft links for some > > > > common applets in a temporary directory and exports a path to that > > > > directory. It might also setup a temporary alternative to /bin/sh > > > > if it is the last shell. > > > > 2. After the remove stage, the busybox binary is gone. The > > > > softlinks created in the prerm stage are useless since they point > > > > to binary that no longer exists. > > > > 3. opkg continues with upgrade on other packages which may depend > > > > on a command provided by busybox in a prerm, postrm, preinst or > > > > postinst script. These upgrades then fail since the commands are > > > > no longer available. > > > > 4. The busybox upgrade completes, which may or may not complete > > > > successfully. For what I am attempting, I am upgrading my system > > > > from the morty branch to dunfell. I have util-linux on my system > > > > which shares some alternatives with busybox. The util-linux > > > > upgrade fails because it needs some busybox applets during its > > > > upgrade process. > > > > Then the busybox upgrade fails because the final update- > > > > alternatives doesn’t work; some files still exist that util-linux > > > > couldn't remove during its upgrade that clash with busybox's > > > > alternatives. > > > > > > > > After trying several ways to get my upgrade to work, I landed on > > > > the approach below. I'm creating a temporary directory and > > > > copying the busybox binary and the busybox.links files to that > > > > directory. I then install an alternative for every applet for > > > > busybox listed in all of its busybox.links files that points to > > > > the temporary copy of the busybox binary. This means that any > > > > package that uses a busybox applet during its install process > > > > should still work. Then during the postinst step I am removing > > > > all the temporary alternative links. > > > > I > > > > use the temporary busybox.links files for removing the alternative > > > > links in case the upgraded busybox is now configured with a > > > > different set of applets. > > > > > > > > This is a heavy handed approach, and it does extend the upgrade > > > > process for me by a few minutes since it runs through update- > > > > alternatives for busybox two more times. But, the approach works > > > > for me and I think would be more resilient than past approaches. > > > > I tried to mimic the existing code in my additions. If a more > > > > widescale rewrite makes sense than that works for me also. > > > > > > > > Thanks, > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > From: Valek, Andrej <andrej.valek@siemens.com<mailto:andrej.valek@siemens.com>> > > > > > Sent: Friday, January 21, 2022 9:01 AM > > > > > To: openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>; Bryan Evenson > > > > > <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> > > > > > Subject: Re: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > busybox links during install > > > > > > > > > > Hi Bryan, > > > > > > > > > > Sorry, maybe I didn't fully understand the use-case. Are you > > > > > trying to upgrade the busybox on demand? If yes, that is not a > > > > > good idea. > > > > > > > > > > I'm little bit scary about doing "export > > > > > PATH=$busybox_rmdir:$PATH" > > > > > and > > > > > creating a custom locks is not a good at all. > > > > > > > > > > Cheers, > > > > > Andrej > > > > > > > > > > On Fri, 2022-01-21 at 13:29 +0000, Bryan Evenson wrote: > > > > > > All, > > > > > > > > > > > > Ping on this RFC. It works for me, but I have a feeling there > > > > > > is a better way to do this. It still seems a little messy and > > > > > > could probably be simplified for the same effect. > > > > > > > > > > > > Thanks, > > > > > > Bryan > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Bryan Evenson > > > > > > > Sent: Thursday, December 23, 2021 9:50 AM > > > > > > > To: openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org> > > > > > > > Subject: [dunfell][PATCH RFC] busybox.inc: Create temporary > > > > > > > busybox links during install > > > > > > > > > > > > > > Busybox upgrades sometimes fail, especially if there is a > > > > > > > major > > > > > > > distribution upgrade and all packages need to be updated. > > > > > > > Success > > > > > > > is highly dependent on the package upgrade order. > > > > > > > > > > > > > > Commit [1] attempts to ensure a shell is still present by > > > > > > > adding > > > > > > > an alternative to /bin/sh if busybox is the only shell. > > > > > > > However, if busybox is not the only shell present and the > > > > > > > other > > > > > > > shells are upgrading, it may then be possible that all > > > > > > > shells > > > > > > > will be removed during the upgrade process. > > > > > > > > > > > > > > Commit [2] creates temporary symbolic links for all the > > > > > > > busybox > > > > > > > links during busybox's postinst step. However, this is too > > > > > > > late > > > > > > > in the process as some packages attempt to use 'rm' and > > > > > > > 'sed' > > > > > > > after > > > > > > > update-alternatives removes the old links and prior to when > > > > > > > busybox's postinst step runs. > > > > > > > > > > > > > > This fix is similar to [2] but runs during the preinst > > > > > > > step. For > > > > > > > opkg, this is the first step that is guaranteed to run from > > > > > > > the > > > > > > > new package (prerm is run from the old package) and will > > > > > > > therefore be a backwards-compatible fix for upgrading older > > > > > > > systems. > > > > > > > > > > > > > > Copies the existing busybox binary and the busybox.links > > > > > > > files > > > > > > > to a temporary directory and then creates alternative links > > > > > > > for > > > > > > > all installed busybox commands. The temporary links and > > > > > > > directory are cleaned up during the postinst step. > > > > > > > > > > > > > > RFC: This works for me, but there may be room for > > > > > > > improvement. I > > > > > > > don't know if the current pkg_prerm steps are necessary > > > > > > > anymore. > > > > > > > However, in > > > > > > > my testing I did need the links for update-alternatives to > > > > > > > work > > > > > > > in the preinst step. I am also not certain if the > > > > > > > populate_packages_updatealternatives_append > > > > > > > step is necessary anymore. I have also only tested this > > > > > > > fix on > > > > > > > dunfell, as I don't have a working image based on master > > > > > > > yet. > > > > > > > It > > > > > > > may be more appropriate for this to go to master and then > > > > > > > be > > > > > > > backported to dunfell, but I would need assistance in > > > > > > > testing. > > > > > > > > > > > > > > [1] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23 > > > > > > > a > > > > > > > [2] https://git.openembedded.org/openembedded- > > > > > > > core/commit/meta/recipes- > > > > > > > > > > > > > > > > core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727 > > > > > > > d2 > > > > > > > > > > > > > > Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com<mailto:bevenson@melinkcorp.com>> > > > > > > > --- > > > > > > > meta/recipes-core/busybox/busybox.inc | 57 > > > > > > > ++++++++++++++++++++++++++- > > > > > > > 1 file changed, 55 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > diff --git a/meta/recipes-core/busybox/busybox.inc > > > > > > > b/meta/recipes- > > > > > > > core/busybox/busybox.inc index e0522be729..c85402411b > > > > > > > 100644 > > > > > > > --- a/meta/recipes-core/busybox/busybox.inc > > > > > > > +++ b/meta/recipes-core/busybox/busybox.inc > > > > > > > @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } > > > > > > > > > > > > > > pkg_postinst_${PN}_append () { > > > > > > > - # If busybox exists in the remove directory it is > > > > > > > because it was the only shell left. > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > + # If busybox exists in the remove directory it > > > > > > > is > > > > > > > because it was the only > > > > > > > shell left. > > > > > > > if [ "x$BUSYBOX" != "x" ] ; then > > > > > > > update-alternatives --remove sh $BUSYBOX > > > > > > > - rm -f $BUSYBOX > > > > > > > fi > > > > > > > + # Remove the temporary alternatives > > > > > > > + for busybox_preinstdir in /tmp/busyboxpreinst- > > > > > > > *; do > > > > > > > + if [ "$busybox_preinstdir" != > > > > > > > '/tmp/busyboxpreinst- > > > > > > > *' ] ; then > > > > > > > + > > > > > > > BUSYBOX_PREINST_DIR="$busybox_preinstdir" > > > > > > > + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" > > > > > > > + if [ -e $BUSYBOX ] ; then > > > > > > > + for suffix in "" ".nosuid" ".suid"; > > > > > > > do > > > > > > > + if [ -e > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then > > > > > > > + while read link; do > > > > > > > + update-alternatives -- > > > > > > > remove > > > > > > > $($BUSYBOX basename > > > > > > > $link) $BUSYBOX > > > > > > > + done < > > > > > > > $BUSYBOX_PREINST_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > + rm -rf $BUSYBOX_PREINST_DIR > > > > > > > + fi > > > > > > > + done > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { > > > > > > > fi > > > > > > > } > > > > > > > > > > > > > > +pkg_preinst_${PN} () { > > > > > > > + # Create a temporary copy the busybox binary and > > > > > > > the > > > > > > > links > > > > > > > files. Then, > > > > > > > + # install an alternative link for all the links. > > > > > > > Other > > > > > > > packages use these > > > > > > > + # commands during their upgrade process. This > > > > > > > ensures > > > > > > > the > > > > > > > links are > > > > > > > available > > > > > > > + # to all the other packages. We do this in the > > > > > > > preinst > > > > > > > step because it is > > > > > > > + # the first step guaranteed to be used from the > > > > > > > new > > > > > > > package. The > > > > > > > prerm is > > > > > > > + # used from the old package. Placing this here > > > > > > > ensures it > > > > > > > runs on > > > > > > > upgrade even > > > > > > > + # on older systems. > > > > > > > + > > > > > > > + if [ "x$D" = "x" ] ; then > > > > > > > + # update-alternatives may need the links from > > > > > > > commands > > > > > > > added in > > > > > > > the prerm step > > > > > > > + # to operate. Make sure we can get to that > > > > > > > path. > > > > > > > + for busybox_rmdir in /tmp/busyboxrm-*; do > > > > > > > + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' > > > > > > > ] ; > > > > > > > then > > > > > > > + export PATH=$busybox_rmdir:$PATH > > > > > > > + fi > > > > > > > + done > > > > > > > + > > > > > > > + # Create a temporary directory for the busybox > > > > > > > binary > > > > > > > and the link lists > > > > > > > + BUSYBOX=${base_bindir}/busybox > > > > > > > + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d > > > > > > > /tmp/busyboxpreinst- > > > > > > > XXXXXX` > > > > > > > + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" > > > > > > > + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC > > > > > > > + > > > > > > > + # Go through all the links and install an > > > > > > > alternative > > > > > > > that points to the > > > > > > > temporary > > > > > > > + # busybox binary. > > > > > > > + for suffix in "" ".nosuid" ".suid"; do > > > > > > > + if [ -e ${sysconfdir}/busybox.links$suffix > > > > > > > ] ; > > > > > > > then > > > > > > > + $BUSYBOX cp > > > > > > > ${sysconfdir}/busybox.links$suffix > > > > > > > $BUSYBOX_TMP_DIR > > > > > > > + while read link; do > > > > > > > + update-alternatives --install $link > > > > > > > $($BUSYBOX basename $link) > > > > > > > $BUSYBOX_TMP_LOC 1 > > > > > > > + done < > > > > > > > $BUSYBOX_TMP_DIR/busybox.links$suffix > > > > > > > + fi > > > > > > > + done > > > > > > > + fi > > > > > > > +} > > > > > > > + > > > > > > > pkg_postrm_${PN} () { > > > > > > > # Add path to remove dir in case we removed our > > > > > > > only > > > > > > > grep > > > > > > > if [ "x$D" = "x" ] ; then > > > > > > > -- > > > > > > > 2.17.1 > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161078): https://lists.openembedded.org/g/openembedded-core/message/161078 Mute This Topic: https://lists.openembedded.org/mt/87919018/3616919 Group Owner: openembedded-core+owner@lists.openembedded.org<mailto:openembedded-core%2Bowner@lists.openembedded.org> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [jatedev@gmail.com<mailto:jatedev@gmail.com>] -=-=-=-=-=-=-=-=-=-=-=-
diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc index e0522be729..c85402411b 100644 --- a/meta/recipes-core/busybox/busybox.inc +++ b/meta/recipes-core/busybox/busybox.inc @@ -441,12 +441,28 @@ pkg_postinst_${PN}_prepend () { } pkg_postinst_${PN}_append () { - # If busybox exists in the remove directory it is because it was the only shell left. if [ "x$D" = "x" ] ; then + # If busybox exists in the remove directory it is because it was the only shell left. if [ "x$BUSYBOX" != "x" ] ; then update-alternatives --remove sh $BUSYBOX - rm -f $BUSYBOX fi + # Remove the temporary alternatives + for busybox_preinstdir in /tmp/busyboxpreinst-*; do + if [ "$busybox_preinstdir" != '/tmp/busyboxpreinst-*' ] ; then + BUSYBOX_PREINST_DIR="$busybox_preinstdir" + BUSYBOX="$BUSYBOX_PREINST_DIR/busybox" + if [ -e $BUSYBOX ] ; then + for suffix in "" ".nosuid" ".suid"; do + if [ -e $BUSYBOX_PREINST_DIR/busybox.links$suffix ] ; then + while read link; do + update-alternatives --remove $($BUSYBOX basename $link) $BUSYBOX + done < $BUSYBOX_PREINST_DIR/busybox.links$suffix + fi + done + fi + rm -rf $BUSYBOX_PREINST_DIR + fi + done fi } @@ -480,6 +496,43 @@ pkg_prerm_${PN} () { fi } +pkg_preinst_${PN} () { + # Create a temporary copy the busybox binary and the links files. Then, + # install an alternative link for all the links. Other packages use these + # commands during their upgrade process. This ensures the links are available + # to all the other packages. We do this in the preinst step because it is + # the first step guaranteed to be used from the new package. The prerm is + # used from the old package. Placing this here ensures it runs on upgrade even + # on older systems. + + if [ "x$D" = "x" ] ; then + # update-alternatives may need the links from commands added in the prerm step + # to operate. Make sure we can get to that path. + for busybox_rmdir in /tmp/busyboxrm-*; do + if [ "$busybox_rmdir" != '/tmp/busyboxrm-*' ] ; then + export PATH=$busybox_rmdir:$PATH + fi + done + + # Create a temporary directory for the busybox binary and the link lists + BUSYBOX=${base_bindir}/busybox + BUSYBOX_TMP_DIR=`$BUSYBOX mktemp -d /tmp/busyboxpreinst-XXXXXX` + BUSYBOX_TMP_LOC="$BUSYBOX_TMP_DIR/busybox" + $BUSYBOX cp $BUSYBOX $BUSYBOX_TMP_LOC + + # Go through all the links and install an alternative that points to the temporary + # busybox binary. + for suffix in "" ".nosuid" ".suid"; do + if [ -e ${sysconfdir}/busybox.links$suffix ] ; then + $BUSYBOX cp ${sysconfdir}/busybox.links$suffix $BUSYBOX_TMP_DIR + while read link; do + update-alternatives --install $link $($BUSYBOX basename $link) $BUSYBOX_TMP_LOC 1 + done < $BUSYBOX_TMP_DIR/busybox.links$suffix + fi + done + fi +} + pkg_postrm_${PN} () { # Add path to remove dir in case we removed our only grep if [ "x$D" = "x" ] ; then
Busybox upgrades sometimes fail, especially if there is a major distribution upgrade and all packages need to be updated. Success is highly dependent on the package upgrade order. Commit [1] attempts to ensure a shell is still present by adding an alternative to /bin/sh if busybox is the only shell. However, if busybox is not the only shell present and the other shells are upgrading, it may then be possible that all shells will be removed during the upgrade process. Commit [2] creates temporary symbolic links for all the busybox links during busybox's postinst step. However, this is too late in the process as some packages attempt to use 'rm' and 'sed' after update-alternatives removes the old links and prior to when busybox's postinst step runs. This fix is similar to [2] but runs during the preinst step. For opkg, this is the first step that is guaranteed to run from the new package (prerm is run from the old package) and will therefore be a backwards-compatible fix for upgrading older systems. Copies the existing busybox binary and the busybox.links files to a temporary directory and then creates alternative links for all installed busybox commands. The temporary links and directory are cleaned up during the postinst step. RFC: This works for me, but there may be room for improvement. I don't know if the current pkg_prerm steps are necessary anymore. However, in my testing I did need the links for update-alternatives to work in the preinst step. I am also not certain if the populate_packages_updatealternatives_append step is necessary anymore. I have also only tested this fix on dunfell, as I don't have a working image based on master yet. It may be more appropriate for this to go to master and then be backported to dunfell, but I would need assistance in testing. [1] https://git.openembedded.org/openembedded-core/commit/meta/recipes-core/busybox/busybox.inc?id=a9d2af8f5b3da8239cf00a52883ca596a19ea23a [2] https://git.openembedded.org/openembedded-core/commit/meta/recipes-core/busybox/busybox.inc?id=3a035bd0a06a6ded4d0ce7e35a3bce42245727d2 Signed-off-by: Bryan Evenson <bevenson@melinkcorp.com> --- meta/recipes-core/busybox/busybox.inc | 57 ++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 2 deletions(-)