| Message ID | 20250310150413.2654942-2-alexandre.marques@ctw.bmwgroup.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add new script to clean hash equivalence db | expand |
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information: --- Testing patch /home/patchtest/share/mboxes/1-1-scripts-Add-clean-hashserver-database-script.patch FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence) PASS: test author valid (test_mbox.TestMbox.test_author_valid) PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence) PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags) PASS: test max line length (test_metadata.TestMetadata.test_max_line_length) PASS: test mbox format (test_mbox.TestMbox.test_mbox_format) PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade) PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format) PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length) PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list) SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint) SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files) SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore) SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format) SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence) SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format) SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format) SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned) SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence) SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence) SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint) SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head) SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files) SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence) --- Please address the issues identified and submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
diff --git a/scripts/clean-hashserver-database b/scripts/clean-hashserver-database new file mode 100755 index 0000000000..6eb006758e --- /dev/null +++ b/scripts/clean-hashserver-database @@ -0,0 +1,73 @@ +#!/bin/bash +set -euo pipefail + +SSTATE_DIR="" +BB_HASHCLIENT="" +BB_HASHSERVER="" + +ALIVE_DB_MARK="alive" +CLEAN_DB="false" + +function help() { + cat <<HELP_TEXT +Usage: $0 --sstate-dir path --hashclient path --hashserver-address address [--mark value] [--clean-db] + +Auxiliary script remove unused or no longer relevant entries from the hashequivalence database, based +on the files available on the sstate directory. + + -h | --help) Show this help message and exit + -s | --sstate-dir) Path to the sstate dir + -c | --hashclient) Path to bitbake-hashclient + -a | --hashserver-adress) bitbake-hashserver address + -m | --mark) Marker string to mark database entries + --clean-db) Remove all unmarked and unused entries from the database +HELP_TEXT +} + +function argument_parser() { + while [ $# -gt 0 ]; do + case "$1" in + -h | --help) help; exit 0 ;; + -s | --sstate-dir) SSTATE_DIR="$2"; shift ;; + -c | --hashclient) BB_HASHCLIENT="$2"; shift ;; + -a | --hashserver-address) BB_HASHSERVER="$2"; shift ;; + -m | --mark) ALIVE_DB_MARK="$2"; shift ;; + --clean-db) CLEAN_DB="true";; + *) + echo "Argument '$1' is not supported" >&2 + help >&2 + exit 1 + ;; + esac + shift + done + + function validate_mandatory_argument() { + local var_value="$1" + local error_message="$2" + + if [ -z "$var_value" ]; then + echo "$error_message" + help >&2 + exit 1 + fi + } + + validate_mandatory_argument "$SSTATE_DIR" "Please provide the path to the sstate dir." + validate_mandatory_argument "$BB_HASHCLIENT" "Please provide the path to bitbake-hashclient." + validate_mandatory_argument "$BB_HASHSERVER" "Please provide the address of bitbake-hashserver." +} + +# -- main code -- +argument_parser $@ + +# Mark all db sstate hashes +find "$SSTATE_DIR" -name "*.tar.zst" | \ +sed 's/.*:\([^_]*\)_.*/unihash \1/' | \ +$BB_HASHCLIENT --address "$BB_HASHSERVER" gc-mark-stream "${ALIVE_DB_MARK}" + +# Remove unmarked and unused entries +if [ "$CLEAN_DB" = "true" ]; then + $BB_HASHCLIENT --address "$BB_HASHSERVER" gc-sweep "${ALIVE_DB_MARK}" + $BB_HASHCLIENT --address "$BB_HASHSERVER" clean-unused 0 +fi
Auxiliary script to clean the hashserver database based on the files available in the sstate directory. It makes used of the new "hashclient gc-mark-stream" command to mark all sstate relevant hashes as "alive" and removes everything else from the database. Usage example: ``` ./scripts/clean-hashserver-database \ --sstate-dir ~/build/sstate-cache \ --hashclient ./bitbake/bin/bitabke-hashclient \ --hashserver-address "ws://localhost:8688/ws" \ --mark "alive" \ --clean-db ``` Issue: NODE0DEV-39586 --- scripts/clean-hashserver-database | 73 +++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100755 scripts/clean-hashserver-database