diff mbox series

[1/1] buildtools: Add a replacement for file command

Message ID 20250522051651.3711352-2-changqing.li@windriver.com
State New
Headers show
Series *** A workaround for bug of file on fedora42 *** | expand

Commit Message

Changqing Li May 22, 2025, 5:16 a.m. UTC 
From: Changqing Li <changqing.li@windriver.com>

Fedora 42 has file 5.46 installed, but this version has an issue, setup
will failed with error "Setting it up...*** buffer overflow detected
***: terminated", refer [1], has fixed in [2], but not in any release,
also not cherry picked by fedora, refer [3].

This is a workaround for this issue,  use python3-magic as a
replacement, and it increases the SDK install time in an acceptable
range

With file:
real	0m5.899s
user	0m4.880s
sys	0m3.177s

With python3-magic:
real	0m11.306s
user	0m9.842s
sys	0m4.752s

[1] https://bugs.astron.com/view.php?id=638
[2] https://github.com/file/file/commit/FILE5_46-7-gb3384a1f
[3] https://bugzilla.redhat.com/show_bug.cgi?id=2354970

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 meta/files/toolchain-shar-extract.sh         | 12 ++++++-
 meta/files/toolchain-shar-relocate.sh        |  8 ++++-
 meta/recipes-core/meta/buildtools-tarball.bb |  2 ++
 scripts/file.py                              | 34 ++++++++++++++++++++
 4 files changed, 54 insertions(+), 2 deletions(-)
 create mode 100755 scripts/file.py
diff mbox series

Patch

diff --git a/meta/files/toolchain-shar-extract.sh b/meta/files/toolchain-shar-extract.sh
index 06934e5a9a..6149d6c2de 100644
--- a/meta/files/toolchain-shar-extract.sh
+++ b/meta/files/toolchain-shar-extract.sh
@@ -11,6 +11,16 @@  INIT_PYTHON=$(command -v python3 2>/dev/null )
 [ -z "$INIT_PYTHON" ] && INIT_PYTHON=$(command -v python2 2>/dev/null)
 [ -z "$INIT_PYTHON" ] && echo "Error: The SDK needs a python installed" && exit 1
 
+file_version=$(file --version | head -n 1)
+if [ "$file_version" = "file-5.46" ];then
+    if ! python3 -c "import magic" &> /dev/null; then
+        echo "Error: The SDK needs file command, but the version 5.46 on this host has an issue:"
+        echo "Error: https://bugs.astron.com/view.php?id=638, so please upgrade file to 5.46+ or"
+        echo "Error: install python3-magic as a replacement"
+        exit 1
+    fi
+fi
+
 # Remove invalid PATH elements first (maybe from a previously setup toolchain now deleted
 PATH=`$INIT_PYTHON -c 'import os; print(":".join(e for e in os.environ["PATH"].split(":") if os.path.exists(e)))'`
 
@@ -297,7 +307,7 @@  fi
 # delete the relocating script, so that user is forced to re-run the installer
 # if he/she wants another location for the sdk
 if [ $savescripts = 0 ] ; then
-	$SUDO_EXEC rm -f ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
+	$SUDO_EXEC rm -f ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh ${env_setup_script%/*}/file.py
 fi
 
 # Execute post-relocation script
diff --git a/meta/files/toolchain-shar-relocate.sh b/meta/files/toolchain-shar-relocate.sh
index c7170349db..f4efcfc88c 100644
--- a/meta/files/toolchain-shar-relocate.sh
+++ b/meta/files/toolchain-shar-relocate.sh
@@ -57,9 +57,15 @@  fi
 
 # replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc.
 # replace the host perl with SDK perl.
+file_version=$(file --version | head -n 1)
+if [ "$file_version" = "file-5.46" ];then
+    file_cmd="$target_sdk_dir/file.py"
+else
+    file_cmd="file"
+fi
 for replace in "$target_sdk_dir -maxdepth 1" "$native_sysroot"; do
 	$SUDO_EXEC find $replace -type f
-done | xargs -d '\n' -n100 file | \
+done | xargs -d '\n' -n100 $file_cmd | \
     awk -F': ' '{if (match($2, ".*(ASCII|script|source).*text")) {printf "\"%s\"\n", $1}}' | \
     grep -Fv -e "$target_sdk_dir/environment-setup-" \
              -e "$target_sdk_dir/relocate_sdk" \
diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index 6fa6d93a3d..51a54af7e4 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -68,6 +68,8 @@  create_sdk_files:append () {
 	rm -f ${SDK_OUTPUT}/${SDKPATH}/environment-setup-*
 	rm -f ${SDK_OUTPUT}/${SDKPATH}/version-*
 
+	cp ${COREBASE}/scripts/file.py ${SDK_OUTPUT}/${SDKPATH}/
+
 	# Generate new (mini) sdk-environment-setup file
 	script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${SDK_SYS}}
 	touch $script
diff --git a/scripts/file.py b/scripts/file.py
new file mode 100755
index 0000000000..69c8cb3d6c
--- /dev/null
+++ b/scripts/file.py
@@ -0,0 +1,34 @@ 
+#!/usr/bin/env python3
+#
+# Copyright (c) 2025 Wind River Systems, Inc.
+#
+# SPDX-License-Identifier: MIT
+#
+# DESCRIPTION
+# This script may be called by the SDK installer script. It is a replacement
+# for file command, it has a bug in version 5.46. Refer:
+# https://bugs.astron.com/view.php?id=638
+
+from concurrent.futures import ThreadPoolExecutor
+import sys
+import magic
+
+def get_file_type(filename):
+    try:
+        with open(filename, 'rb') as f:
+            data = f.read(2048)
+            file_type = magic.from_buffer(data)
+            print(f"{filename}: {file_type}")
+    except Exception as e:
+        print(f"Error processing {filename}: {e}")
+
+def main():
+    if len(sys.argv) < 2:
+        print("Usage: file.py <file1> <file2> ...")
+        return
+
+    with ThreadPoolExecutor(max_workers=10) as executor:
+        executor.map(get_file_type, sys.argv[1:])
+
+if __name__ == "__main__":
+    main()