| Message ID | 20230607194102.3785739-1-tgamblin@baylibre.com |
|---|---|
| State | New |
| Headers | show |
| Series | [yocto-autobuilder-helper] clobberdir: only move dirs more than 60s old | expand |
On Wed, 2023-06-07 at 15:41 -0400, Trevor Gamblin wrote: > See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14952 > > Help avoid contention by adding a check to ensure targets are at > least 60s old when attempting to move them into a trash directory. > > Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> > --- > janitor/clobberdir | 10 +++++++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/janitor/clobberdir b/janitor/clobberdir > index 16b019e..832173c 100755 > --- a/janitor/clobberdir > +++ b/janitor/clobberdir > @@ -52,8 +52,12 @@ trashdir = utils.getconfig("TRASH_DIR", ourconfig) > for x in [clobberdir]: > if os.path.exists(x) and os.path.exists(trashdir): > if (os.stat(trashdir).st_dev == os.stat(x).st_dev): > - trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 100000, 2)) > - mkdir(trashdest) > - subprocess.check_call(['mv', x, trashdest]) > + x_age = time.time() - os.path.getmtime(x) > + if x_age > 60: > + trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 100000, 2)) > + mkdir(trashdest) > + subprocess.check_call(['mv', x, trashdest]) > + else: > + print("Not moving '%s' - age is only %s seconds. There may be another process using it" % (x, str(int(x_age)))) > else: > subprocess.check_call(['rm', "-rf", x]) I think we need to do this on the other side of the setup. If we ask to run clobberdir, it needs to succeed and I don't think the move causes us issues. What does cause problems is if a bitbake server was still active on a build directory and shutting down. If we move the directory from under it, that is ok, what causes problems is if we actually delete it. The age check therefore needs to be in trash_processor() in janitor/ab- janitor. Cheers, Richard
diff --git a/janitor/clobberdir b/janitor/clobberdir index 16b019e..832173c 100755 --- a/janitor/clobberdir +++ b/janitor/clobberdir @@ -52,8 +52,12 @@ trashdir = utils.getconfig("TRASH_DIR", ourconfig) for x in [clobberdir]: if os.path.exists(x) and os.path.exists(trashdir): if (os.stat(trashdir).st_dev == os.stat(x).st_dev): - trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 100000, 2)) - mkdir(trashdest) - subprocess.check_call(['mv', x, trashdest]) + x_age = time.time() - os.path.getmtime(x) + if x_age > 60: + trashdest = trashdir + "/" + str(int(time.time())) + '-' + str(random.randrange(100, 100000, 2)) + mkdir(trashdest) + subprocess.check_call(['mv', x, trashdest]) + else: + print("Not moving '%s' - age is only %s seconds. There may be another process using it" % (x, str(int(x_age)))) else: subprocess.check_call(['rm', "-rf", x])
See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14952 Help avoid contention by adding a check to ensure targets are at least 60s old when attempting to move them into a trash directory. Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> --- janitor/clobberdir | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)