| Message ID | 20220318162320.1072462-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | New, archived |
| Headers | show |
| Series | set_versions/switchers.js: Allow switchers.js version information to be autogenerated | expand |
Hi Richard, On 3/18/22 17:23, Richard Purdie wrote: > A horrible blunt hammer approach to updating the version information in > switchers.js based on the available tag information. > > To merge and work correctly, this will need a change to the autobuilder-helper > docs generation code to pull the swicthers.js and script from master, then > to run the script. That should hopefully remove the need for other patching > even on old docs branches though. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > documentation/.gitignore | 1 + > documentation/set_versions.py | 21 +++++++++++++++++++ > .../{switchers.js => switchers.js.in} | 8 +------ > 3 files changed, 23 insertions(+), 7 deletions(-) > rename documentation/sphinx-static/{switchers.js => switchers.js.in} (97%) > > diff --git a/documentation/.gitignore b/documentation/.gitignore > index e5e2c1708..096b97ec2 100644 > --- a/documentation/.gitignore > +++ b/documentation/.gitignore > @@ -1,6 +1,7 @@ > _build/ > Pipfile.lock > poky.yaml > +sphinx-static/switchers.js > .vscode/ > */svg/*.png > */svg/*.pdf > diff --git a/documentation/set_versions.py b/documentation/set_versions.py > index 6452aeb14..9cd0de46c 100755 > --- a/documentation/set_versions.py > +++ b/documentation/set_versions.py > @@ -13,6 +13,7 @@ import subprocess > import collections > import sys > > +activereleases = ["honister", "hardknott", "dunfell"] > devbranch = "kirkstone" > #devbranch = "langdale" > ltsseries = ["kirkstone", "dunfell"] > @@ -144,3 +145,23 @@ with open("poky.yaml.in", "r") as r, open("poky.yaml", "w") as w: > > print("poky.yaml generated from poky.yaml.in") > > +with open("sphinx-static/switchers.js.in", "r") as r, open("sphinx-static/switchers.js", "w") as w: > + lines = r.readlines() > + for line in lines: > + if "VERSIONS_PLACEHOLDER" in line: > + w.write(" 'dev': 'dev (%s)',\n" % release_series[devbranch]) > + for branch in activereleases: Until now we always had the last version of all sphinx supported doc releases (+ zeus and thud for some reason). This patch is doing a bit more than just automating the whole thing, it actually changes the behavior/result. Python docs still have 3.5 listed for example. I think it would be a mistake to remove the option to select an outdated version in the drop down menu. What happens if I'm stuck on that version and want its docs? > + if branch == devbranch: > + continue > + versions = subprocess.run('git tag --list yocto-%s*' % (release_series[branch]), shell=True, capture_output=True, text=True).stdout.split() I think it'd make sense to do the git tag --list outside of the loop and catch all tags starting with yocto- and then operate on that list from within the loop. Also, git tag --sorted="version:refname" should sort correctly, then you just need to iterate over the list and take the last one before a minor version change (e.g. 3.1.14 and the next is 3.2.0, take 3.1.14 because change from 1 to 2). > + versions = sorted([v.replace("yocto-" + release_series[branch] + ".", "").replace("yocto-" + release_series[branch], "0") for v in versions], key=int) > + version = release_series[branch] > + if versions[-1] != "0": > + version = version + "." + versions[-1] I don't understand what this is supposed to do? Can't you just take X,Y,Z from the yocto-X.Y.Z tag and write it? Which corner case did I miss? Cheers, Quentin
On Mon, 2022-03-21 at 18:08 +0100, Quentin Schulz wrote: > Hi Richard, > > On 3/18/22 17:23, Richard Purdie wrote: > > A horrible blunt hammer approach to updating the version information in > > switchers.js based on the available tag information. > > > > To merge and work correctly, this will need a change to the autobuilder- > > helper > > docs generation code to pull the swicthers.js and script from master, then > > to run the script. That should hopefully remove the need for other patching > > even on old docs branches though. > > > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > > --- > > documentation/.gitignore | 1 + > > documentation/set_versions.py | 21 +++++++++++++++++++ > > .../{switchers.js => switchers.js.in} | 8 +------ > > 3 files changed, 23 insertions(+), 7 deletions(-) > > rename documentation/sphinx-static/{switchers.js => switchers.js.in} (97%) > > > > diff --git a/documentation/.gitignore b/documentation/.gitignore > > index e5e2c1708..096b97ec2 100644 > > --- a/documentation/.gitignore > > +++ b/documentation/.gitignore > > @@ -1,6 +1,7 @@ > > _build/ > > Pipfile.lock > > poky.yaml > > +sphinx-static/switchers.js > > .vscode/ > > */svg/*.png > > */svg/*.pdf > > diff --git a/documentation/set_versions.py b/documentation/set_versions.py > > index 6452aeb14..9cd0de46c 100755 > > --- a/documentation/set_versions.py > > +++ b/documentation/set_versions.py > > @@ -13,6 +13,7 @@ import subprocess > > import collections > > import sys > > > > +activereleases = ["honister", "hardknott", "dunfell"] > > devbranch = "kirkstone" > > #devbranch = "langdale" > > ltsseries = ["kirkstone", "dunfell"] > > @@ -144,3 +145,23 @@ with open("poky.yaml.in", "r") as r, open("poky.yaml", > > "w") as w: > > > > print("poky.yaml generated from poky.yaml.in") > > > > +with open("sphinx-static/switchers.js.in", "r") as r, open("sphinx- > > static/switchers.js", "w") as w: > > + lines = r.readlines() > > + for line in lines: > > + if "VERSIONS_PLACEHOLDER" in line: > > + w.write(" 'dev': 'dev (%s)',\n" % release_series[devbranch]) > > + for branch in activereleases: > > Until now we always had the last version of all sphinx supported doc > releases (+ zeus and thud for some reason). As I understood it, the intent was to have the current active releases (which is why zeus was there). We've just continued to add new ones and not removed any. I've tried to add some "policy" around this as part of the changes in this series. It would be trivial to change the active releases to include zeus and gatesgarth, then it would be equivalent to how it is now but that doesn't match what I thought it was supposed to be doing. It is clear we have different ideas on what we think it should be doing though :/. > This patch is doing a bit more than just automating the whole thing, it > actually changes the behavior/result. Yes, for example in the 1.4.4 release, it would show that release in the docs switcher now. I guess further pieces of that are in later patches or changes in the transition branch. If you want I can change it to add zeus and gatesgarth to the active releases, then it is equivalent (apart from adding new options in old releases) but I will then propose a patch to remove them! > Python docs still have 3.5 listed for example. I think it would be a > mistake to remove the option to select an outdated version in the drop > down menu. What happens if I'm stuck on that version and want its docs? Then you can access it through the URL, or the older releases list. Python doesn't list 3.3 or 2.6. For us, our active releases are dunfell, hardknott (right now), honister and kirkstone. > > > + if branch == devbranch: > > + continue > > + versions = subprocess.run('git tag --list yocto-%s*' % > > (release_series[branch]), shell=True, capture_output=True, > > text=True).stdout.split() > > I think it'd make sense to do the git tag --list outside of the loop and > catch all tags starting with yocto- and then operate on that list from > within the loop. > > Also, git tag --sorted="version:refname" should sort correctly, then you > just need to iterate over the list and take the last one before a minor > version change (e.g. 3.1.14 and the next is 3.2.0, take 3.1.14 because > change from 1 to 2). Perhaps we can look at that as a later incremental change? > > > + versions = sorted([v.replace("yocto-" + release_series[branch] + ".", "").replace("yocto-" + release_series[branch], "0") for v in versions], key=int) > > + version = release_series[branch] > > + if versions[-1] != "0": > + version = version + "." + versions[-1] > > I don't understand what this is supposed to do? Can't you just take > X,Y,Z from the yocto-X.Y.Z tag and write it? Which corner case did I miss? We're trying to get the latest version for each release series. This means taking yocto-3.1 as "0" yocto-3.1.1 as "1" yocto-3.1.15 as "15" and sorting as an int. The issue is that it is 3.1, not 3.1.0. There are probably better ways to do it... Cheers, Richard
Hi Richard, On 3/21/22 19:09, Richard Purdie wrote: > On Mon, 2022-03-21 at 18:08 +0100, Quentin Schulz wrote: >> Hi Richard, >> >> On 3/18/22 17:23, Richard Purdie wrote: >>> A horrible blunt hammer approach to updating the version information in >>> switchers.js based on the available tag information. >>> >>> To merge and work correctly, this will need a change to the autobuilder- >>> helper >>> docs generation code to pull the swicthers.js and script from master, then >>> to run the script. That should hopefully remove the need for other patching >>> even on old docs branches though. >>> >>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> >>> --- >>> documentation/.gitignore | 1 + >>> documentation/set_versions.py | 21 +++++++++++++++++++ >>> .../{switchers.js => switchers.js.in} | 8 +------ >>> 3 files changed, 23 insertions(+), 7 deletions(-) >>> rename documentation/sphinx-static/{switchers.js => switchers.js.in} (97%) >>> >>> diff --git a/documentation/.gitignore b/documentation/.gitignore >>> index e5e2c1708..096b97ec2 100644 >>> --- a/documentation/.gitignore >>> +++ b/documentation/.gitignore >>> @@ -1,6 +1,7 @@ >>> _build/ >>> Pipfile.lock >>> poky.yaml >>> +sphinx-static/switchers.js >>> .vscode/ >>> */svg/*.png >>> */svg/*.pdf >>> diff --git a/documentation/set_versions.py b/documentation/set_versions.py >>> index 6452aeb14..9cd0de46c 100755 >>> --- a/documentation/set_versions.py >>> +++ b/documentation/set_versions.py >>> @@ -13,6 +13,7 @@ import subprocess >>> import collections >>> import sys >>> >>> +activereleases = ["honister", "hardknott", "dunfell"] >>> devbranch = "kirkstone" >>> #devbranch = "langdale" >>> ltsseries = ["kirkstone", "dunfell"] >>> @@ -144,3 +145,23 @@ with open("poky.yaml.in", "r") as r, open("poky.yaml", >>> "w") as w: >>> >>> print("poky.yaml generated from poky.yaml.in") >>> >>> +with open("sphinx-static/switchers.js.in", "r") as r, open("sphinx- >>> static/switchers.js", "w") as w: >>> + lines = r.readlines() >>> + for line in lines: >>> + if "VERSIONS_PLACEHOLDER" in line: >>> + w.write(" 'dev': 'dev (%s)',\n" % release_series[devbranch]) >>> + for branch in activereleases: >> >> Until now we always had the last version of all sphinx supported doc >> releases (+ zeus and thud for some reason). > > As I understood it, the intent was to have the current active releases (which is > why zeus was there). We've just continued to add new ones and not removed any. > I've tried to add some "policy" around this as part of the changes in this > series. > > It would be trivial to change the active releases to include zeus and > gatesgarth, then it would be equivalent to how it is now but that doesn't match > what I thought it was supposed to be doing. It is clear we have different ideas > on what we think it should be doing though :/. > >> This patch is doing a bit more than just automating the whole thing, it >> actually changes the behavior/result. > > Yes, for example in the 1.4.4 release, it would show that release in the docs > switcher now. I guess further pieces of that are in later patches or changes in > the transition branch. > > If you want I can change it to add zeus and gatesgarth to the active releases, > then it is equivalent (apart from adding new options in old releases) but I will > then propose a patch to remove them! > Yes please. So the change is not hidden away and if merged, very explicit as to what the policy should be (since we "disagree" on the current policy or its enforcing (or lack thereof :) )). >> Python docs still have 3.5 listed for example. I think it would be a >> mistake to remove the option to select an outdated version in the drop >> down menu. What happens if I'm stuck on that version and want its docs? > > Then you can access it through the URL, or the older releases list. > Indeed, I had forgotten about the older releases list! > Python doesn't list 3.3 or 2.6. For us, our active releases are dunfell, > hardknott (right now), honister and kirkstone. > >> >>> + if branch == devbranch: >>> + continue >>> + versions = subprocess.run('git tag --list yocto-%s*' % >>> (release_series[branch]), shell=True, capture_output=True, >>> text=True).stdout.split() >> >> I think it'd make sense to do the git tag --list outside of the loop and >> catch all tags starting with yocto- and then operate on that list from >> within the loop. >> >> Also, git tag --sorted="version:refname" should sort correctly, then you >> just need to iterate over the list and take the last one before a minor >> version change (e.g. 3.1.14 and the next is 3.2.0, take 3.1.14 because >> change from 1 to 2). > > Perhaps we can look at that as a later incremental change? > >> >>> + versions = sorted([v.replace("yocto-" + release_series[branch] + ".", "").replace("yocto-" + release_series[branch], "0") for v in versions], key=int) >>> + version = release_series[branch] >>> + if versions[-1] != "0": > + version = version + "." + versions[-1] >> >> I don't understand what this is supposed to do? Can't you just take >> X,Y,Z from the yocto-X.Y.Z tag and write it? Which corner case did I miss? > > We're trying to get the latest version for each release series. This means > taking > > yocto-3.1 as "0" > yocto-3.1.1 as "1" > yocto-3.1.15 as "15" > > and sorting as an int. The issue is that it is 3.1, not 3.1.0. > git tag --list --sort="version:refname" 'yocto-*' order them correctly already. But can be improved in later patches. Could even do a git tag --list --sort="version:refname" 'yocto-3.1*' | tail -1 (provided we never have a X.Y.Z version with Y >= 10 :). Cheers, Quentin
diff --git a/documentation/.gitignore b/documentation/.gitignore index e5e2c1708..096b97ec2 100644 --- a/documentation/.gitignore +++ b/documentation/.gitignore @@ -1,6 +1,7 @@ _build/ Pipfile.lock poky.yaml +sphinx-static/switchers.js .vscode/ */svg/*.png */svg/*.pdf diff --git a/documentation/set_versions.py b/documentation/set_versions.py index 6452aeb14..9cd0de46c 100755 --- a/documentation/set_versions.py +++ b/documentation/set_versions.py @@ -13,6 +13,7 @@ import subprocess import collections import sys +activereleases = ["honister", "hardknott", "dunfell"] devbranch = "kirkstone" #devbranch = "langdale" ltsseries = ["kirkstone", "dunfell"] @@ -144,3 +145,23 @@ with open("poky.yaml.in", "r") as r, open("poky.yaml", "w") as w: print("poky.yaml generated from poky.yaml.in") +with open("sphinx-static/switchers.js.in", "r") as r, open("sphinx-static/switchers.js", "w") as w: + lines = r.readlines() + for line in lines: + if "VERSIONS_PLACEHOLDER" in line: + w.write(" 'dev': 'dev (%s)',\n" % release_series[devbranch]) + for branch in activereleases: + if branch == devbranch: + continue + versions = subprocess.run('git tag --list yocto-%s*' % (release_series[branch]), shell=True, capture_output=True, text=True).stdout.split() + versions = sorted([v.replace("yocto-" + release_series[branch] + ".", "").replace("yocto-" + release_series[branch], "0") for v in versions], key=int) + version = release_series[branch] + if versions[-1] != "0": + version = version + "." + versions[-1] + print(str(versions)) + w.write(" '%s': '%s',\n" % (version, version)) + else: + w.write(line) + +print("switchers.js generated from switchers.js.in") + diff --git a/documentation/sphinx-static/switchers.js b/documentation/sphinx-static/switchers.js.in similarity index 97% rename from documentation/sphinx-static/switchers.js rename to documentation/sphinx-static/switchers.js.in index 3ea8927d7..5d3a4d793 100644 --- a/documentation/sphinx-static/switchers.js +++ b/documentation/sphinx-static/switchers.js.in @@ -10,13 +10,7 @@ by https://git.yoctoproject.org/yocto-autobuilder-helper/tree/scripts/run-docs-b 'use strict'; var all_versions = { - 'dev': 'dev (3.5)', - '3.4.2': '3.4.2', - '3.3.5': '3.3.5', - '3.2.4': '3.2.4', - '3.1.14': '3.1.14', - '3.0.4': '3.0.4', - '2.7.4': '2.7.4', + VERSIONS_PLACEHOLDER }; var all_doctypes = {
A horrible blunt hammer approach to updating the version information in switchers.js based on the available tag information. To merge and work correctly, this will need a change to the autobuilder-helper docs generation code to pull the swicthers.js and script from master, then to run the script. That should hopefully remove the need for other patching even on old docs branches though. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- documentation/.gitignore | 1 + documentation/set_versions.py | 21 +++++++++++++++++++ .../{switchers.js => switchers.js.in} | 8 +------ 3 files changed, 23 insertions(+), 7 deletions(-) rename documentation/sphinx-static/{switchers.js => switchers.js.in} (97%)