| Message ID | 20230531051408.2518859-1-xiangyu.chen@eng.windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | [kirkstone] perf: fix argument list too long during compile | expand |
On Wed, May 31, 2023 at 1:14 AM Xiangyu Chen <xiangyu.chen@eng.windriver.com> wrote: > > From: Xiangyu Chen <xiangyu.chen@windriver.com> > > The perf would fail to build when BUILDDIR is longer than 190, The "O = ${B}" > makes it use absolute path which makes the argument list too long, remove > "O = ${B}" will make it use relative path to build will fix the problem. > Can you elaborate on how the relative path use is happening when an explicit output directory isn't passed ? I haven't checked the source, but I'd assume that it will simply build directly in the source directory if we don't pass O to the build. Also, is this fixed in a different way in master ? Bruce > Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com> > --- > meta/recipes-kernel/perf/perf.bb | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb > index a4ce3169d3..e213c3632e 100644 > --- a/meta/recipes-kernel/perf/perf.bb > +++ b/meta/recipes-kernel/perf/perf.bb > @@ -72,7 +72,6 @@ LDFLAGS="-ldl -lutil" > EXTRA_OEMAKE = '\ > V=1 \ > -C ${S}/tools/perf \ > - O=${B} \ > CROSS_COMPILE=${TARGET_PREFIX} \ > ARCH=${ARCH} \ > CC="${CC}" \ > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#181989): https://lists.openembedded.org/g/openembedded-core/message/181989 > Mute This Topic: https://lists.openembedded.org/mt/99236216/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hi Bruce, Sorry for being late, update some information I observed in my local setup; On 5/31/23 20:23, Bruce Ashfield wrote: > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Wed, May 31, 2023 at 1:14 AM Xiangyu Chen > <xiangyu.chen@eng.windriver.com> wrote: >> From: Xiangyu Chen <xiangyu.chen@windriver.com> >> >> The perf would fail to build when BUILDDIR is longer than 190, The "O = ${B}" >> makes it use absolute path which makes the argument list too long, remove >> "O = ${B}" will make it use relative path to build will fix the problem. >> > Can you elaborate on how the relative path use is happening when an > explicit output directory isn't passed ? When I using a long path/long folder name as below[1] , the perf package would report an error "| make[4]: /bin/sh: Argument list too long" > > I haven't checked the source, but I'd assume that it will simply build > directly in the source directory if we don't pass O to the build. > > Also, is this fixed in a different way in master ? Indeed, simply drop the "O" is a workaround solution, just put the compile object to source folder. This behavior cannot find after mickledore branch, after comparing mickledore and kirkstone code, I found that it related to make. The kirkstone's make using 4.3, the mickledore using 4.4.1. There is a commit [2] already fix the issue, but this fix depends lots of other commits(and not sure those commit has alternative impaction), so looks upgrade package directly is lower risky than backport patches, but i am not sure bump the make package (4.3->4.4.x) is meet the policy, could you give some suggestion? Thanks. > > Bruce Ref: [1] e.g. build_test_logname_xxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxx_dir [2] https://git.savannah.gnu.org/cgit/make.git/commit/?id=dc2d963989b96161472b2cd38cef5d1f4851ea34 Thanks, Xiangyu >> Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com> >> --- >> meta/recipes-kernel/perf/perf.bb | 1 - >> 1 file changed, 1 deletion(-) >> >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb >> index a4ce3169d3..e213c3632e 100644 >> --- a/meta/recipes-kernel/perf/perf.bb >> +++ b/meta/recipes-kernel/perf/perf.bb >> @@ -72,7 +72,6 @@ LDFLAGS="-ldl -lutil" >> EXTRA_OEMAKE = '\ >> V=1 \ >> -C ${S}/tools/perf \ >> - O=${B} \ >> CROSS_COMPILE=${TARGET_PREFIX} \ >> ARCH=${ARCH} \ >> CC="${CC}" \ >> -- >> 2.34.1 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#181989): https://lists.openembedded.org/g/openembedded-core/message/181989 >> Mute This Topic: https://lists.openembedded.org/mt/99236216/1050810 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II
On Thu, Aug 17, 2023 at 11:30 PM Xiangyu Chen <xiangyu.chen@eng.windriver.com> wrote: > > Hi Bruce, > > > Sorry for being late, update some information I observed in my local setup; > > > On 5/31/23 20:23, Bruce Ashfield wrote: > > CAUTION: This email comes from a non Wind River email account! > > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > > > On Wed, May 31, 2023 at 1:14 AM Xiangyu Chen > > <xiangyu.chen@eng.windriver.com> wrote: > >> From: Xiangyu Chen <xiangyu.chen@windriver.com> > >> > >> The perf would fail to build when BUILDDIR is longer than 190, The "O = ${B}" > >> makes it use absolute path which makes the argument list too long, remove > >> "O = ${B}" will make it use relative path to build will fix the problem. > >> > > Can you elaborate on how the relative path use is happening when an > > explicit output directory isn't passed ? > > When I using a long path/long folder name as below[1] , the perf package > would > > report an error "| make[4]: /bin/sh: Argument list too long" > > > > > > I haven't checked the source, but I'd assume that it will simply build > > directly in the source directory if we don't pass O to the build. > > > > Also, is this fixed in a different way in master ? > > Indeed, simply drop the "O" is a workaround solution, just put the > compile object to source folder. As you know, it isn't ideal to mix the source and objects, it is obviously something that we don't do unless there really are no other options. In the past, we have converted the directories to relative paths versus absolute by specifying B explicitly (and making sure it is relative). Have you tried anything like that ? > > This behavior cannot find after mickledore branch, after comparing > mickledore and kirkstone code, > > I found that it related to make. > > The kirkstone's make using 4.3, the mickledore using 4.4.1. There is a > commit [2] already fix the issue, > > but this fix depends lots of other commits(and not sure those commit has > alternative impaction), so looks > > upgrade package directly is lower risky than backport patches, but i am > not sure bump the make package > > (4.3->4.4.x) is meet the policy, could you give some suggestion? Thanks. > Interesting. And yes, it is unlikely that a new version of Make will appear in Mickledore. It would be interesting to see the list of commits that are required and then we could review and look for impacts. One other question ... it can't just be perf that is being impacted by this issue. How are other packages building in your directory structure without failing on long paths ? Bruce > > > > > Bruce > > > Ref: > > [1] e.g. build_test_logname_xxxxxxxxxxxxxxxxx > > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > > xxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx > > xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx > > xxxxxxxxxxxxxxxxxxxx_dir > > > [2] > https://git.savannah.gnu.org/cgit/make.git/commit/?id=dc2d963989b96161472b2cd38cef5d1f4851ea34 > > > > Thanks, > > Xiangyu > > >> Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com> > >> --- > >> meta/recipes-kernel/perf/perf.bb | 1 - > >> 1 file changed, 1 deletion(-) > >> > >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb > >> index a4ce3169d3..e213c3632e 100644 > >> --- a/meta/recipes-kernel/perf/perf.bb > >> +++ b/meta/recipes-kernel/perf/perf.bb > >> @@ -72,7 +72,6 @@ LDFLAGS="-ldl -lutil" > >> EXTRA_OEMAKE = '\ > >> V=1 \ > >> -C ${S}/tools/perf \ > >> - O=${B} \ > >> CROSS_COMPILE=${TARGET_PREFIX} \ > >> ARCH=${ARCH} \ > >> CC="${CC}" \ > >> -- > >> 2.34.1 > >> > >> > >> > >> > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#186345): https://lists.openembedded.org/g/openembedded-core/message/186345 > Mute This Topic: https://lists.openembedded.org/mt/99236216/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 8/22/23 11:37, Bruce Ashfield wrote: > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Thu, Aug 17, 2023 at 11:30 PM Xiangyu Chen > <xiangyu.chen@eng.windriver.com> wrote: >> Hi Bruce, >> >> >> Sorry for being late, update some information I observed in my local setup; >> >> >> On 5/31/23 20:23, Bruce Ashfield wrote: >>> CAUTION: This email comes from a non Wind River email account! >>> Do not click links or open attachments unless you recognize the sender and know the content is safe. >>> >>> On Wed, May 31, 2023 at 1:14 AM Xiangyu Chen >>> <xiangyu.chen@eng.windriver.com> wrote: >>>> From: Xiangyu Chen <xiangyu.chen@windriver.com> >>>> >>>> The perf would fail to build when BUILDDIR is longer than 190, The "O = ${B}" >>>> makes it use absolute path which makes the argument list too long, remove >>>> "O = ${B}" will make it use relative path to build will fix the problem. >>>> >>> Can you elaborate on how the relative path use is happening when an >>> explicit output directory isn't passed ? >> When I using a long path/long folder name as below[1] , the perf package >> would >> >> report an error "| make[4]: /bin/sh: Argument list too long" >> >> >>> I haven't checked the source, but I'd assume that it will simply build >>> directly in the source directory if we don't pass O to the build. >>> >>> Also, is this fixed in a different way in master ? >> Indeed, simply drop the "O" is a workaround solution, just put the >> compile object to source folder. > As you know, it isn't ideal to mix the source and objects, it is > obviously something that we don't do unless there really are > no other options. > > In the past, we have converted the directories to relative paths > versus absolute by specifying B explicitly (and making sure it > is relative). Have you tried anything like that ? Yes, I have tried to set the B variant to a relative path in bb file but still failed, seems the full of extending TOPDIR is too long to be parsed by Make. If set the B to a shorter path (e.g. absolute path to /tmp/perf-build), the issue won't happen. Looks the clue still point to that long file path. >> This behavior cannot find after mickledore branch, after comparing >> mickledore and kirkstone code, >> >> I found that it related to make. >> >> The kirkstone's make using 4.3, the mickledore using 4.4.1. There is a >> commit [2] already fix the issue, >> >> but this fix depends lots of other commits(and not sure those commit has >> alternative impaction), so looks >> >> upgrade package directly is lower risky than backport patches, but i am >> not sure bump the make package >> >> (4.3->4.4.x) is meet the policy, could you give some suggestion? Thanks. >> > Interesting. And yes, it is unlikely that a new version of Make > will appear in Mickledore. Indeed, this issue is a corner case, as Make is a basic builder tool, upgrade cross two big version might take into more unseen risky on stable kirkstone. > > It would be interesting to see the list of commits that are required > and then we could review and look for impacts. aha, the fix on Make-4.4.0.90, currently, kirkstone using 4.3, lots of changes cross the 5 version (4.4 is a formal release, 4.3.90/91/92 and 4.4.0.90 is alpha release). I tried to backport the patches yesterday, but still in compiling error status, seems need more effort to pick up the commits :p > > One other question ... it can't just be perf that is being impacted > by this issue. How are other packages building in your directory > structure without failing on long paths ? Yes, this is make's limitation, should happens on any packages but currently, i just tried on oe-core layer and only perf found this issue, from the behavior, this should happens on a package which EXTRA_OEMAKE have long enough parameters. Thanks, Xiangyu > Bruce > >>> Bruce >> >> Ref: >> >> [1] e.g. build_test_logname_xxxxxxxxxxxxxxxxx >> >> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> xxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> >> xxxxxxxxxxxxxxxxxxxx_dir >> >> >> [2] >> https://git.savannah.gnu.org/cgit/make.git/commit/?id=dc2d963989b96161472b2cd38cef5d1f4851ea34 >> >> >> >> Thanks, >> >> Xiangyu >> >>>> Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com> >>>> --- >>>> meta/recipes-kernel/perf/perf.bb | 1 - >>>> 1 file changed, 1 deletion(-) >>>> >>>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb >>>> index a4ce3169d3..e213c3632e 100644 >>>> --- a/meta/recipes-kernel/perf/perf.bb >>>> +++ b/meta/recipes-kernel/perf/perf.bb >>>> @@ -72,7 +72,6 @@ LDFLAGS="-ldl -lutil" >>>> EXTRA_OEMAKE = '\ >>>> V=1 \ >>>> -C ${S}/tools/perf \ >>>> - O=${B} \ >>>> CROSS_COMPILE=${TARGET_PREFIX} \ >>>> ARCH=${ARCH} \ >>>> CC="${CC}" \ >>>> -- >>>> 2.34.1 >>>> >>>> >>>> >>>> >>> -- >>> - Thou shalt not follow the NULL pointer, for chaos and madness await >>> thee at its end >>> - "Use the force Harry" - Gandalf, Star Trek II >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#186345): https://lists.openembedded.org/g/openembedded-core/message/186345 >> Mute This Topic: https://lists.openembedded.org/mt/99236216/1050810 >> Group Owner: openembedded-core+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index a4ce3169d3..e213c3632e 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -72,7 +72,6 @@ LDFLAGS="-ldl -lutil" EXTRA_OEMAKE = '\ V=1 \ -C ${S}/tools/perf \ - O=${B} \ CROSS_COMPILE=${TARGET_PREFIX} \ ARCH=${ARCH} \ CC="${CC}" \