| Message ID | 20211118135421.4068596-1-richard.purdie@linuxfoundation.org |
|---|---|
| State | Accepted, archived |
| Commit | 63baf3440b16e41ac6601de21ced94a94bdf1509 |
| Headers | show |
| Series | utils: Handle lockfile filenames that are too long for filesystems | expand |
On Thu, Nov 18, 2021 at 6:54 AM Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > The fetcher mirror code can go crazy creating lock filenames which exceed > the > filesystem limits. When this happens, the code will loop/hang. > > Handle the filename too long exception correctly but also truncate lockfile > lengths to under 256 since the worst case situation is lockfile overlap > and lack of parallelism. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > lib/bb/utils.py | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/lib/bb/utils.py b/lib/bb/utils.py > index 70634910f7..23ff64df7a 100644 > --- a/lib/bb/utils.py > +++ b/lib/bb/utils.py > @@ -451,6 +451,11 @@ def lockfile(name, shared=False, retry=True, > block=False): > consider the possibility of sending a signal to the process to break > out - at which point you want block=True rather than retry=True. > """ > + if len(name) > 255: > + components = name.rsplit(".", 1) > + components[0] = components[0][:255 - len(components[1]) - 1] > + name = ".".join(components) > I think this may be easier to read with separate variables and os.path.splitext: root, ext = os.path.splitext(name) name = root[:255 - len(ext) - 1] + ext
diff --git a/lib/bb/utils.py b/lib/bb/utils.py index 70634910f7..23ff64df7a 100644 --- a/lib/bb/utils.py +++ b/lib/bb/utils.py @@ -451,6 +451,11 @@ def lockfile(name, shared=False, retry=True, block=False): consider the possibility of sending a signal to the process to break out - at which point you want block=True rather than retry=True. """ + if len(name) > 255: + components = name.rsplit(".", 1) + components[0] = components[0][:255 - len(components[1]) - 1] + name = ".".join(components) + dirname = os.path.dirname(name) mkdirhier(dirname) @@ -487,7 +492,7 @@ def lockfile(name, shared=False, retry=True, block=False): return lf lf.close() except OSError as e: - if e.errno == errno.EACCES: + if e.errno == errno.EACCES or e.errno == errno.ENAMETOOLONG: logger.error("Unable to acquire lock '%s', %s", e.strerror, name) sys.exit(1)
The fetcher mirror code can go crazy creating lock filenames which exceed the filesystem limits. When this happens, the code will loop/hang. Handle the filename too long exception correctly but also truncate lockfile lengths to under 256 since the worst case situation is lockfile overlap and lack of parallelism. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- lib/bb/utils.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)