| Message ID | 20250424111014.905507-1-alex.kanavin@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/4] devtool: account for sources in UNPACKDIR | expand |
On Thu, 2025-04-24 at 13:10 +0200, Alexander Kanavin via lists.openembedded.org wrote: > From: Alexander Kanavin <alex@linutronix.de> > > There's a couple of assumptions in devtool that sources > are always in a directory under WORKDIR; this is no longer > the case since introduction of UNPACKDIR: some recipes > are starting to use > > S = "${UNPACKDIR}/git" > > or similar, and so the logic to determine source locations > needs to be tweaked accordingly. > > The issue is that oe-core has no concept of 'top level source > path' (S points to where the build is started from, inside that > top level location), and yet devtool needs to know that in order > to move the complete source tree correctly to a workspace (and possibly > other uses). > > And so devtool performs convoluted path calculations from WORKDIR and S; > now this has been extended to include UNPACKDIR and became more convoluted > but hopefully it won't get any worse. > > Signed-off-by: Alexander Kanavin <alex@linutronix.de> > --- > meta/classes/devtool-source.bbclass | 8 ++++++-- > scripts/lib/devtool/ide_sdk.py | 2 +- > scripts/lib/devtool/standard.py | 25 +++++++++++++++++++------ > scripts/lib/devtool/upgrade.py | 2 +- > 4 files changed, 27 insertions(+), 10 deletions(-) UNPACKDIR was meant to help with this. If devtool is changing the location, can't it just change UNPACKDIR and leave everything else alone? S will still be under UNPACKDIR/git even after devtool moves it? Cheers, Richard
On Thu, 24 Apr 2025 at 13:32, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > UNPACKDIR was meant to help with this. If devtool is changing the > location, can't it just change UNPACKDIR and leave everything else > alone? S will still be under UNPACKDIR/git even after devtool moves it? Some recipes use S = /path/to/workdir/git/sub/path and some use S = /path/to/workdir/unpackdir/git/sub/path It's that 'git' part in the middle that needs to be extracted from the final value of S and appended to UNPACKDIR (or WORKDIR), and then the resulting path is moved by devtool to a workspace with shutil. I probably don't fully understand the suggestion? Alex
On Thu, 2025-04-24 at 13:44 +0200, Alexander Kanavin wrote: > On Thu, 24 Apr 2025 at 13:32, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > UNPACKDIR was meant to help with this. If devtool is changing the > > location, can't it just change UNPACKDIR and leave everything else > > alone? S will still be under UNPACKDIR/git even after devtool moves > > it? > > Some recipes use > S = /path/to/workdir/git/sub/path > and some use > S = /path/to/workdir/unpackdir/git/sub/path > > It's that 'git' part in the middle that needs to be extracted from > the final value of S and appended to UNPACKDIR (or WORKDIR), and then > the resulting path is moved by devtool to a workspace with shutil. I > probably don't fully understand the suggestion? In the two examples above, what is UNPACKDIR set to? Cheers, Richard
On Thu, 2025-04-24 at 13:13 +0100, Richard Purdie via lists.openembedded.org wrote: > On Thu, 2025-04-24 at 13:44 +0200, Alexander Kanavin wrote: > > On Thu, 24 Apr 2025 at 13:32, Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > > > > > UNPACKDIR was meant to help with this. If devtool is changing the > > > location, can't it just change UNPACKDIR and leave everything else > > > alone? S will still be under UNPACKDIR/git even after devtool moves > > > it? > > > > Some recipes use > > S = /path/to/workdir/git/sub/path > > and some use > > S = /path/to/workdir/unpackdir/git/sub/path > > > > It's that 'git' part in the middle that needs to be extracted from > > the final value of S and appended to UNPACKDIR (or WORKDIR), and then > > the resulting path is moved by devtool to a workspace with shutil. I > > probably don't fully understand the suggestion? > > In the two examples above, what is UNPACKDIR set to? Sorry, let me be more specific. What are the actual values of S and WORKDIR for the different cases? Cheers, Richard
On Thu, 24 Apr 2025 at 14:14, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > In the two examples above, what is UNPACKDIR set to? > > Sorry, let me be more specific. What are the actual values of S and > WORKDIR for the different cases? Can we step back for a bit? The original error is that 'devtool upgrade' breaks cryptically on recipes that set: S = "${UNPACKDIR}/git" (instead of the usual WORKDIR/git). (everything else being the defaults from oe-core). Devtool, in this case, calculates the 'top level source' to be the same as UNPACKDIR, and breakage follows. An example of such recipe in core is meta/recipes-devtools/rpm-sequoia/rpm-sequoia_1.7.0.bb Alex
On Thu, 2025-04-24 at 14:41 +0200, Alexander Kanavin wrote: > On Thu, 24 Apr 2025 at 14:14, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > In the two examples above, what is UNPACKDIR set to? > > > > Sorry, let me be more specific. What are the actual values of S and > > WORKDIR for the different cases? > > Can we step back for a bit? The original error is that 'devtool > upgrade' breaks cryptically on recipes that set: > > S = "${UNPACKDIR}/git" (instead of the usual WORKDIR/git). > > (everything else being the defaults from oe-core). > > Devtool, in this case, calculates the 'top level source' to be the > same as UNPACKDIR, and breakage follows. > > An example of such recipe in core is > meta/recipes-devtools/rpm-sequoia/rpm-sequoia_1.7.0.bb Stepping back further, we had this mess in devtool in the first place as it could never work out which files in WORKDIR were "sources" which it needed to handle carefully. We now define UNPACKDIR as the place we unpack files to. devtool in theory can therefore just redefine UNPACKDIR and WORKDIR and things should 'just work'. The relation of the subpaths between the two should remain the same. This should therefore make the code in devtool simpler, not more complex. Your patch isn't doing that and I'm wondering if there are code paths in there which are no longer needed if we handle UNPACKDIR correctly/as intended. So in summary, something isn't adding up here to me but I'm not yet sure quite what. Cheers, Richard
Richard Purdie via lists.openembedded.org <richard.purdie= linuxfoundation.org@lists.openembedded.org> schrieb am Do., 24. Apr. 2025, 15:50: > On Thu, 2025-04-24 at 14:41 +0200, Alexander Kanavin wrote: > > On Thu, 24 Apr 2025 at 14:14, Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > > > > In the two examples above, what is UNPACKDIR set to? > > > > > > Sorry, let me be more specific. What are the actual values of S and > > > WORKDIR for the different cases? > > > > Can we step back for a bit? The original error is that 'devtool > > upgrade' breaks cryptically on recipes that set: > > > > S = "${UNPACKDIR}/git" (instead of the usual WORKDIR/git). > > > > (everything else being the defaults from oe-core). > > > > Devtool, in this case, calculates the 'top level source' to be the > > same as UNPACKDIR, and breakage follows. > > > > An example of such recipe in core is > > meta/recipes-devtools/rpm-sequoia/rpm-sequoia_1.7.0.bb > > Stepping back further, we had this mess in devtool in the first place > as it could never work out which files in WORKDIR were "sources" which > it needed to handle carefully. > > We now define UNPACKDIR as the place we unpack files to. devtool in > theory can therefore just redefine UNPACKDIR and WORKDIR and things > should 'just work'. The relation of the subpaths between the two should > remain the same. > > This should therefore make the code in devtool simpler, not more > complex. > > Your patch isn't doing that and I'm wondering if there are code paths > in there which are no longer needed if we handle UNPACKDIR correctly/as > intended. > > So in summary, something isn't adding up here to me but I'm not yet > sure quite what. > At the moment I'm not able to look into the details. But I remember the extra complexity in the code of devtool for handling the corner case when S=WORKDIR with an extra folder containing sym-links to the real source files. This is now removed thanks to UNPACKDIR which no longer allows this. At least for ide_sdk this looked like a huge cleanup to me. Maybe now it is possible to clearly define how devtool uses or maps S, UNPACKDIR and WORKDIR in a generic way rather than trying to fix the code in some way which just works. Or is there already such a concept behind this patch? Adrian > > > Cheers, > > Richard > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#215373): > https://lists.openembedded.org/g/openembedded-core/message/215373 > Mute This Topic: https://lists.openembedded.org/mt/112431116/4454582 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > adrian.freihofer@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Thu, 24 Apr 2025 at 15:50, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > Stepping back further, we had this mess in devtool in the first place > as it could never work out which files in WORKDIR were "sources" which > it needed to handle carefully. > > We now define UNPACKDIR as the place we unpack files to. devtool in > theory can therefore just redefine UNPACKDIR and WORKDIR and things > should 'just work'. The relation of the subpaths between the two should > remain the same. I still struggle to understand this argument. As far as I see, to determine where is the 'source tree', devtool cannot avoid looking at both WORKDIR and S (and now, UNPACKDIR too) and then calculate something reasonable out of both (and with UNPACKDIR in the mix, all three). Let me describe this step by step what happens without the patch. 1. devtool manages a 'workspace': a special layer under build/workspace/. Part of the workspace is sources/: a place where source trees are placed. When a recipe is taken into workspace, its source tree is placed under build/workspace/sources/${PN}/. 2. How does devtool determine where to take the source tree from? It sets WORKDIR to a temporary location under the real oe-core workdir, and then performs an unpack operation. Then it, somehow, needs to find the source tree under that temporary WORKDIR. It cannot simply use S: this can be set to a nested sub-directory deep inside the actual source tree. It cannot use WORKDIR either: this is not the source tree in almost all cases (and is now forbidden anyway). 3. Let's say WORKDIR is set to /some/path/to/workdir, and S is then /some/path/to/workdir/git/some/subpath/. (via S = "${WORKDIR}/git/some/subpath") devtool does this: a) use os.path.relpath on WORKDIR and S to obtain git/some/subpath part b) then it splits that part into individual directories and take the first one: 'git' c) then it's appended to WORKDIR: /some/path/to/workdir/git d) then devtool uses shutil.move to move/rename that into workspace, so /some/path/to/workdir/git becomes build/workspace/sources/${PN} All fine, so far. What happens if the recipe sets S = "${UNPACKDIR}/git/some/subpath? The same calculation as in point three will yield UNPACKDIR (incorrect), instead of UNPACKDIR/git (correct). And this is what the patch is aiming to address: it changes the code to first check if S is under UNPACKDIR, and if so, performs the calculation in point three relative to UNPACKDIR. Otherwise, it's performed relative to WORKDIR, as before. The argument I can't understand is that devtool can somehow avoid looking at S altogether, and achieve the right thing by just redefining UNPACKDIR and WORKDIR and removing the logic described above, but how? How would we end up with the correct source tree in the workspace then? Alex
On Thu, 24 Apr 2025 at 19:36, Adrian Freihofer <adrian.freihofer@gmail.com> wrote: > Maybe now it is possible to clearly define how devtool uses or maps S, UNPACKDIR > and WORKDIR in a generic way rather than trying to fix the code in some way which > just works. Or is there already such a concept behind this patch? I've just described to RP how devtool figures out what to place into workspace/sources/. I'm not sure how we could do something different than what this patch does, without breaking the layout of that directory, and all tools that expect the current layout with it. We could, for example, just copy UNPACKDIR into workspace/sources/${PN} - simple but completely incompatible with what happens now. And it would introduce ambiguity and inconsistency into where the source tree is in there for anyone trying to edit source files. Alex
On Thu, 2025-04-24 at 21:36 +0200, Alexander Kanavin wrote: > On Thu, 24 Apr 2025 at 15:50, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > Stepping back further, we had this mess in devtool in the first > > place > > as it could never work out which files in WORKDIR were "sources" > > which > > it needed to handle carefully. > > > > We now define UNPACKDIR as the place we unpack files to. devtool in > > theory can therefore just redefine UNPACKDIR and WORKDIR and things > > should 'just work'. The relation of the subpaths between the two > > should > > remain the same. > > I still struggle to understand this argument. As far as I see, to > determine where is the 'source tree', devtool cannot avoid looking at > both WORKDIR and S (and now, UNPACKDIR too) and then calculate > something reasonable out of both (and with UNPACKDIR in the mix, all > three). > > Let me describe this step by step what happens without the patch. > > 1. devtool manages a 'workspace': a special layer under > build/workspace/. Part of the workspace is sources/: a place where > source trees are placed. When a recipe is taken into workspace, its > source tree is placed under build/workspace/sources/${PN}/. > > 2. How does devtool determine where to take the source tree from? It > sets WORKDIR to a temporary location under the real oe-core workdir, > and then performs an unpack operation. Why can't it set UNPACKDIR to the the workspace path and then just run it to directly unpack to where it wants it? It can then redefine UNPACKDIR to point at the workspace path and things should "just work"? > Then it, somehow, needs to find > the source tree under that temporary WORKDIR. It cannot simply use S: > this can be set to a nested sub-directory deep inside the actual > source tree. It cannot use WORKDIR either: this is not the source > tree in almost all cases (and is now forbidden anyway). It can however use UNPACKDIR? > > 3. Let's say WORKDIR is set to /some/path/to/workdir, and S is then > /some/path/to/workdir/git/some/subpath/. (via S = > "${WORKDIR}/git/some/subpath") > > devtool does this: > > a) use os.path.relpath on WORKDIR and S to obtain git/some/subpath > part > b) then it splits that part into individual directories and take the > first one: 'git' > c) then it's appended to WORKDIR: /some/path/to/workdir/git > d) then devtool uses shutil.move to move/rename that into workspace, > so /some/path/to/workdir/git becomes build/workspace/sources/${PN} > > All fine, so far. What happens if the recipe sets S = > "${UNPACKDIR}/git/some/subpath? The same calculation as in point > three > will yield UNPACKDIR (incorrect), instead of UNPACKDIR/git (correct). > And this is what the patch is aiming to address: it changes the code > to first check if S is under UNPACKDIR, and if so, performs the > calculation in point three relative to UNPACKDIR. Otherwise, it's > performed relative to WORKDIR, as before. > > The argument I can't understand is that devtool can somehow avoid > looking at S altogether, and achieve the right thing by just > redefining UNPACKDIR and WORKDIR and removing the logic described > above, but how? How would we end up with the correct source tree in > the workspace then? A recipe doesn't have a single source tree but a set of sources, which is the contents of UNPACKDIR. do_unpack has some horrible magic moving which moves the sources into the expected place for the magic value of S (see the SOURCE_BASEDIR mess) and this is perhaps confusing things. In my mind, I'm thinking we ultimately stop that move and put a symlink in place instead for compatibility. There were some issues with autotools and symlinks when I last tried iirc but I'm digressing... From memory, one of the devtool issues was that it couldn't track and customise some of the input sources such as standalone files. The UNPACKDIR changes were intended to give us a path forward so we could actually handle the full set of sources rather than just the "S" contents. So I still believe there is some kind of simplification/improvement we probably can make here... Cheers, Richard
On Thu, 24 Apr 2025 at 22:34, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > Why can't it set UNPACKDIR to the the workspace path and then just run > it to directly unpack to where it wants it? > > It can then redefine UNPACKDIR to point at the workspace path and > things should "just work"? It can. But it will fundamentally change what users are going to get under workspace/sources/${PN} and not necessarily for the better: 1. Right now, if you 'cd workspace/sources/${PN}', you'll see a source tree that matches upstream repository (or tarball) exactly, for the unpacked item that contains ${S} in it. It's obvious what to do: you can modify files, run bitbake with these modifications, make commits and devtool will pick them up and integrate into the recipe. Yes, technically there can be more than one upstream item listed in SRC_URI (I'm not sure what devtool is going to do when it sees them), but the vast majority of recipes have just one item, and this keeps things simple and straightforward for the users (and tools). The cost of it: devtool is a convoluted mess (and to make matters worse, without a maintainer); I don't argue with that. 2. What happens if workspace/sources/${PN} becomes UNPACKDIR? When you go there, you'll see something named 'git/'. Is that where the actual source is? If the recipe is fetching a tarball, there's no git/, but rather ${PN}-${PV}/. So there's an extra, potentially confusing step to get to the source code. It becomes more ambiguous and less consistent for the users. 3. This will also break all the tools that assume the current structure, and another reason this change shouldn't be taken lightly. 4. Maybe there's something I missed that could be done, and won't break the current layout? For example, we can set UNPACKDIR to workspace/unpack/${PN}, and then use devtool 'logic' to provide a hardlink into it from workspace/sources/${PN} ? But it would be the same logic as now with the patch: take original values of WORKDIR, UNPACKDIR and S, and figure out where in the new UNPACKDIR to point to. Alex
On Fri, 2025-04-25 at 10:33 +0200, Alexander Kanavin wrote: > On Thu, 24 Apr 2025 at 22:34, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > Why can't it set UNPACKDIR to the the workspace path and then just > > run > > it to directly unpack to where it wants it? > > > > It can then redefine UNPACKDIR to point at the workspace path and > > things should "just work"? > > It can. But it will fundamentally change what users are going to get > under workspace/sources/${PN} and not necessarily for the better: > > 1. Right now, if you 'cd workspace/sources/${PN}', you'll see a > source > tree that matches upstream repository (or tarball) exactly, for the > unpacked item that contains ${S} in it. It's obvious what to do: you > can modify files, run bitbake with these modifications, make commits > and devtool will pick them up and integrate into the recipe. > > Yes, technically there can be more than one upstream item listed in > SRC_URI (I'm not sure what devtool is going to do when it sees them), > but the vast majority of recipes have just one item, and this keeps > things simple and straightforward for the users (and tools). The cost > of it: devtool is a convoluted mess (and to make matters worse, > without a maintainer); I don't argue with that. There are continual requests to have devtool able to operate on recipes with multiple SRC_URI entries, be able to see/modify files, support things like gcc and the kernel which have interesting build layouts and so on. I thought we'd already added support for some of this which is probably why the logic is already convoluted. I suspect it will only get worse. > 2. What happens if workspace/sources/${PN} becomes UNPACKDIR? When > you go there, you'll see something named 'git/'. Is that where the > actual source is? If the recipe is fetching a tarball, there's no > git/, but rather ${PN}-${PV}/. So there's an extra, potentially > confusing step to get to the source code. It becomes more ambiguous > and less consistent for the users. It does match what happens in a real build in WORKDIR. Where you have a subdir of a git repo, this is already exposed as you don't get S but need to find the subdir. My point being it is already probably inconsistent. > > 3. This will also break all the tools that assume the current > structure, and another reason this change shouldn't be taken lightly. Perhaps, or perhaps not. Do we have a lot of tools that wouldn't cope with this? > 4. Maybe there's something I missed that could be done, and won't > break the current layout? For example, we can set UNPACKDIR to > workspace/unpack/${PN}, and then use devtool 'logic' to provide a > hardlink into it from workspace/sources/${PN} ? But it would be the > same logic as now with the patch: take original values of WORKDIR, > UNPACKDIR and S, and figure out where in the new UNPACKDIR to point > to. That 'logic' is effectively what do_unpack is already doing itself to maintain 'compatibility'. Perhaps we could use the same logic in both places and be consistent! I'm torn here. I could take your patch, bury my head in the sand and pretend everything is fine. We could try and actually improve this and set things in a better direction for the future, at the expense of breaking things in the short/medium term. I don't know what to do for the best here to be honest. Taking the patch is way easier for me I doubt we'll ever improvements if I don't push back on things at times :/. (I'm spelling this out so people understand the dilemma) Cheers, Richard
On Fri, 25 Apr 2025 at 11:31, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > 3. This will also break all the tools that assume the current > > structure, and another reason this change shouldn't be taken lightly. > > Perhaps, or perhaps not. Do we have a lot of tools that wouldn't cope > with this? Likely Visual Studio Code support will have to be fixed? Impact of changes like this is difficult to predict until they're done. Then everyone comes out complaining their private scripts broke. Otherwise, I'm slowly getting convinced that this is the right thing :)T > That 'logic' is effectively what do_unpack is already doing itself to > maintain 'compatibility'. Perhaps we could use the same logic in both > places and be consistent! > > I'm torn here. I could take your patch, bury my head in the sand and > pretend everything is fine. We could try and actually improve this and > set things in a better direction for the future, at the expense of > breaking things in the short/medium term. I don't know what to do for > the best here to be honest. Taking the patch is way easier for me I > doubt we'll ever improvements if I don't push back on things at times > :/. Here's something I want to propose, feel free to shoot it down. Should we start by just dropping support for this legacy compatibility of sources in workdir, and S = "${WORKDIR}/something"? So S, when set that way, becomes a hard recipe_qa error, and sources are always in UNPACKDIR. Yes, there would be a ton of recipes to be fixed, but the fix would be largely a search-and-replace. Admittedly, I didn't follow the arguments around UNPACKDIR closely and it may have been talked about, but why not just do it? Then this fix for devtool wouldn't add more complexity, it would just similarly replace workdir with unpackdir - not making things better but not making them worse either. On the other hand, if we do nothing, then devtool currently doesn't work for recipes that set S to unpackdir, negatively impacting AUH rates for one thing (this is why I got into it in the first place) :-( Alex
diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass index 9762003ba75..d4e307e886b 100644 --- a/meta/classes/devtool-source.bbclass +++ b/meta/classes/devtool-source.bbclass @@ -92,9 +92,13 @@ python devtool_post_unpack() { for fname in local_files: f.write('%s\n' % fname) - if os.path.dirname(srcsubdir) != workdir: + if srcsubdir.startswith(unpackdir) and srcsubdir != unpackdir: + srcparentdir = unpackdir + else: + srcparentdir = workdir + if os.path.dirname(srcsubdir) != srcparentdir: # Handle if S is set to a subdirectory of the source - srcsubdir = os.path.join(workdir, os.path.relpath(srcsubdir, workdir).split(os.sep)[0]) + srcsubdir = os.path.join(srcparentdir, os.path.relpath(srcsubdir, srcparentdir).split(os.sep)[0]) scriptutils.git_convert_standalone_clone(srcsubdir) diff --git a/scripts/lib/devtool/ide_sdk.py b/scripts/lib/devtool/ide_sdk.py index f8cf65f4a84..e1717bc4ab5 100755 --- a/scripts/lib/devtool/ide_sdk.py +++ b/scripts/lib/devtool/ide_sdk.py @@ -334,7 +334,7 @@ class RecipeModified: self.srctree = workspace[workspacepn]['srctree'] # Need to grab this here in case the source is within a subdirectory self.real_srctree = get_real_srctree( - self.srctree, recipe_d.getVar('S'), recipe_d.getVar('WORKDIR')) + self.srctree, recipe_d.getVar('S'), recipe_d.getVar('WORKDIR'), recipe_d.getVar('UNPACKDIR')) self.bbappend = workspace[workspacepn]['bbappend'] self.ide_sdk_dir = os.path.join( diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py index cdfdba43eef..9a8f990d84a 100644 --- a/scripts/lib/devtool/standard.py +++ b/scripts/lib/devtool/standard.py @@ -625,7 +625,12 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works srcsubdir = f.read() except FileNotFoundError as e: raise DevtoolError('Something went wrong with source extraction - the devtool-source class was not active or did not function correctly:\n%s' % str(e)) - srcsubdir_rel = os.path.relpath(srcsubdir, os.path.join(tempdir, 'workdir')) + unpackdir = d.getVar('UNPACKDIR') + if d.getVar('S').startswith(d.getVar('UNPACKDIR')): + srcparentdir = os.path.relpath(d.getVar('UNPACKDIR'), d.getVar('WORKDIR')) + else: + srcparentdir = '' + srcsubdir_rel = os.path.relpath(srcsubdir, os.path.join(tempdir, 'workdir', srcparentdir)) # Check if work-shared is empty, if yes # find source and copy to work-shared @@ -742,14 +747,22 @@ def get_staging_kbranch(srcdir): staging_kbranch = "".join(branch.split('\n')[0]) return staging_kbranch -def get_real_srctree(srctree, s, workdir): +def get_real_srctree(srctree, s, workdir, unpackdir): # Check that recipe isn't using a shared workdir s = os.path.abspath(s) workdir = os.path.abspath(workdir) - if s.startswith(workdir) and s != workdir and os.path.dirname(s) != workdir: + unpackdir = os.path.abspath(unpackdir) + + if s.startswith(workdir) and s != workdir: # Handle if S is set to a subdirectory of the source - srcsubdir = os.path.relpath(s, workdir).split(os.sep, 1)[1] - srctree = os.path.join(srctree, srcsubdir) + if s.startswith(unpackdir) and s != unpackdir: + srcparentdir = unpackdir + else: + srcparentdir = workdir + + if os.path.dirname(s) != srcparentdir: + srcsubdir = os.path.relpath(s, srcparentdir).split(os.sep, 1)[1] + srctree = os.path.join(srctree, srcsubdir) return srctree def modify(args, config, basepath, workspace): @@ -907,7 +920,7 @@ def modify(args, config, basepath, workspace): # Need to grab this here in case the source is within a subdirectory srctreebase = srctree - srctree = get_real_srctree(srctree, rd.getVar('S'), rd.getVar('WORKDIR')) + srctree = get_real_srctree(srctree, rd.getVar('S'), rd.getVar('WORKDIR'), rd.getVar('UNPACKDIR')) bb.utils.mkdirhier(os.path.dirname(appendfile)) with open(appendfile, 'w') as f: diff --git a/scripts/lib/devtool/upgrade.py b/scripts/lib/devtool/upgrade.py index 0dace1fb240..e584e03e17c 100644 --- a/scripts/lib/devtool/upgrade.py +++ b/scripts/lib/devtool/upgrade.py @@ -571,7 +571,7 @@ def upgrade(args, config, basepath, workspace): else: srctree = standard.get_default_srctree(config, pn) - srctree_s = standard.get_real_srctree(srctree, rd.getVar('S'), rd.getVar('WORKDIR')) + srctree_s = standard.get_real_srctree(srctree, rd.getVar('S'), rd.getVar('WORKDIR'), rd.getVar('UNPACKDIR')) # try to automatically discover latest version and revision if not provided on command line if not args.version and not args.srcrev: