| Message ID | 20240914201417.22497-1-martin.jansa@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | Revert "fetch2/gitsm: use configparser to parse .gitmodules" | expand |
Glad we caught this before release... Acked-by: Ross Burton <ross.burton@arm.com> Thanks, Ross > On 14 Sep 2024, at 22:14, Martin Jansa via lists.openembedded.org <martin.jansa=gmail.com@lists.openembedded.org> wrote: > > This reverts commit a4f42e396e2942fde94b8b4944487c1c45f7a295. > > Unfortunately configparser is a bit more strict and fails to parse e.g.: > https://github.com/espressif/esp-idf/blob/e104dd7f27d2e73ab0e9b614dd7b9295099069bf/.gitmodules > > [submodule "components/bt/controller/lib_esp32"] > path = components/bt/controller/lib_esp32 > url = ../../espressif/esp32-bt-lib.git > > The url is using 8 spaces while path 1 tab causing: > Exception: configparser.NoOptionError: No option 'url' in section: 'submodule "components/bt/controller/lib_esp32"' > > It was fixed in: > https://github.com/espressif/esp-idf/commit/62ca8e2fb478a1cdc0e47003025265cd0d840395#diff-fe7afb5c9c916e521401d3fcfb4277d5071798c3baf83baf11d6071742823584 > > but mcuboot is using a bit older esp-idf revision in: > https://github.com/ATmobica/mcuboot/blame/main/.gitmodules > > and mcuboot is then used as submodule in: > https://github.com/project-chip/connectedhomeip/blob/master/.gitmodules > > so it might take a while for the fix to be propagated everywhere. > > Not sure how common these issues are, but configparser parses > "url = ../../espressif/esp32-bt-lib.git" as 2nd line of "path" > value (because it's indented differently) while git submodule > and old gitsm implementation parses it as separate path and > url keys. > > Signed-off-by: Martin Jansa <martin.jansa@gmail.com> > --- > lib/bb/fetch2/gitsm.py | 24 +++++++++++------------- > 1 file changed, 11 insertions(+), 13 deletions(-) > > diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py > index f193ae3c9..f7f3af721 100644 > --- a/lib/bb/fetch2/gitsm.py > +++ b/lib/bb/fetch2/gitsm.py > @@ -47,20 +47,18 @@ class GitSM(Git): > subrevision = {} > > def parse_gitmodules(gitmodules): > - """ > - Parse .gitmodules and return a dictionary of submodule paths to dictionaries with path and url members. > - """ > - import configparser > - cp = configparser.ConfigParser() > - cp.read_string(gitmodules) > - > modules = {} > - for section in [s for s in cp.sections() if s.startswith("submodule ")]: > - module = section.split('"')[1] > - modules[module] = { > - 'path': cp.get(section, 'path'), > - 'url': cp.get(section, 'url') > - } > + module = "" > + for line in gitmodules.splitlines(): > + if line.startswith('[submodule'): > + module = line.split('"')[1] > + modules[module] = {} > + elif module and line.strip().startswith('path'): > + path = line.split('=')[1].strip() > + modules[module]['path'] = path > + elif module and line.strip().startswith('url'): > + url = line.split('=')[1].strip() > + modules[module]['url'] = url > return modules > > # Collect the defined submodules, and their attributes > -- > 2.46.0 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#16561): https://lists.openembedded.org/g/bitbake-devel/message/16561 > Mute This Topic: https://lists.openembedded.org/mt/108454701/6875888 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ross.burton@arm.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py index f193ae3c9..f7f3af721 100644 --- a/lib/bb/fetch2/gitsm.py +++ b/lib/bb/fetch2/gitsm.py @@ -47,20 +47,18 @@ class GitSM(Git): subrevision = {} def parse_gitmodules(gitmodules): - """ - Parse .gitmodules and return a dictionary of submodule paths to dictionaries with path and url members. - """ - import configparser - cp = configparser.ConfigParser() - cp.read_string(gitmodules) - modules = {} - for section in [s for s in cp.sections() if s.startswith("submodule ")]: - module = section.split('"')[1] - modules[module] = { - 'path': cp.get(section, 'path'), - 'url': cp.get(section, 'url') - } + module = "" + for line in gitmodules.splitlines(): + if line.startswith('[submodule'): + module = line.split('"')[1] + modules[module] = {} + elif module and line.strip().startswith('path'): + path = line.split('=')[1].strip() + modules[module]['path'] = path + elif module and line.strip().startswith('url'): + url = line.split('=')[1].strip() + modules[module]['url'] = url return modules # Collect the defined submodules, and their attributes
This reverts commit a4f42e396e2942fde94b8b4944487c1c45f7a295. Unfortunately configparser is a bit more strict and fails to parse e.g.: https://github.com/espressif/esp-idf/blob/e104dd7f27d2e73ab0e9b614dd7b9295099069bf/.gitmodules [submodule "components/bt/controller/lib_esp32"] path = components/bt/controller/lib_esp32 url = ../../espressif/esp32-bt-lib.git The url is using 8 spaces while path 1 tab causing: Exception: configparser.NoOptionError: No option 'url' in section: 'submodule "components/bt/controller/lib_esp32"' It was fixed in: https://github.com/espressif/esp-idf/commit/62ca8e2fb478a1cdc0e47003025265cd0d840395#diff-fe7afb5c9c916e521401d3fcfb4277d5071798c3baf83baf11d6071742823584 but mcuboot is using a bit older esp-idf revision in: https://github.com/ATmobica/mcuboot/blame/main/.gitmodules and mcuboot is then used as submodule in: https://github.com/project-chip/connectedhomeip/blob/master/.gitmodules so it might take a while for the fix to be propagated everywhere. Not sure how common these issues are, but configparser parses "url = ../../espressif/esp32-bt-lib.git" as 2nd line of "path" value (because it's indented differently) while git submodule and old gitsm implementation parses it as separate path and url keys. Signed-off-by: Martin Jansa <martin.jansa@gmail.com> --- lib/bb/fetch2/gitsm.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)