| Message ID | cover.1642309436.git.tim.orling@konsulko.com |
|---|---|
| State | New |
| Headers | show |
Hi, This seems to be causing build break for nativesdk builds. nativesdk-python3-cryptography: running build_rust error: Error loading target specification: Could not find specification for target "x86_64-pokysdk-linux". Run `rustc --print target-list` for a list of built-in targets The same build for target gives only warning: running build_rust warning: build_rust: Forced Rust target `aarch64-poky-linux` is not compatible with deduced Rust target `aarch64-unknown-linux-gnu` - the built package may not import successfully once installed. -samuli On Sun, 16 Jan 2022 at 07:29, Tim Orling <ticotimo@gmail.com> wrote: > This series upgrades python3-cryptography from the last version to be > purely Python/C extensions (3.3.2) to the latest version which also > includes Rust extensions (36.0.1). > > This has been tested (including full ptest runs) on: > - qemux86-64 > - qemux86-musl > - qemuarm64 > > The test cases require an absolute minimum of 1 GB of free memory > according to the documentation, but in practice it was more performant > to have 2 GB of free memory. For this reason a check-memfree.py test is > added to run-ptest to avoid OOM/Exit code 9 in the ptest-runner. The > test cases were run with: > > runqemu nographic slirp qemuparams="-m 2176 -smp 4" > > On x86-64 with kvm this takes about 5 minutes. > > The results are 2856+/-2 passed, 1654+/-2 skipped. Of the skipped tests > 1600 of them are because of unsupported "counter location middle_fixed" > in https://github.com/pyca/cryptography/pull/2773. A further ~35 test > cases are skipped because of not setting --wychefproof-root, but even > when the Google wychefproof repository is cloned and the option is set, > it adds another ~5000 skipped test cases: > > https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/wip-python3-cryptography-ptest&id=fff4da6e70c97265e17a82f1d043ab16039d510e > > Upstream uses python3-pytest-xdist to parallize test cases (and > python3-pytest-shard to distribute amongst workers), but this uses > stdin/stdout and therefore we have no output to ptest-runner. > > This series introduces two helper classes pyo3.bbclass and > setuptools_rust.bbclass for Python/Rust projects. > > The setuptools_rust.bbclass wraps PyO3, Cargo and the newly added > python3-setuptools-rust-native recipe. Since oe-core only supports > native rustc at this time, only a native recipe is provided. > > To demonstrate that these classes are applicable to multiple recipes, we > also introduce python3-pyruvate, a WSGI implementation in Rust. > > While we are at it we upgrade python3-pytest-subtests from 0.5.0 to > 0.6.0. > > The following changes since commit > df5c1a31fdb8325d1c514f9ed8a7b3805dc581ac: > > python3-diskcache: Upgrade 5.3.0 -> 5.4.0 (2022-01-10 10:35:02 -0800) > > are available in the Git repository at: > > git://git.openembedded.org/meta-openembedded-contrib > timo/python3-cryptography_36.0.1 > > http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=timo/python3-cryptography_36.0.1 > > Tim Orling (7): > python3-setuptools-rust-native: add v1.1.2 recipe > pyo3.bbclass: add class for PyO3 cross-compilation > setuptools3_rust.bbclass: setuptoools Rust plugin > python3-pyruvate: add recipe for v1.1.2 > python3-pytest-subtests: upgrade 0.5.0 -> 0.6.0 > python3-cryptography-vectors: upgrade 35.0.0 => 36.0.1 > python3-cryptography: upgrade 3.3.2 -> 36.0.1 > > meta-python/classes/pyo3.bbclass | 30 +++++ > meta-python/classes/setuptools3_rust.bbclass | 11 ++ > ...=> python3-cryptography-vectors_36.0.1.bb} | 5 +- > .../0001-Cargo.toml-specify-pem-version.patch | 31 +++++ > .../0002-Cargo.toml-edition-2018-2021.patch | 28 ++++ > .../python3-cryptography/check-memfree.py | 10 ++ > .../python/python3-cryptography/h-test.patch | 10 -- > .../python3-cryptography/openssl3.patch | 62 --------- > .../python/python3-cryptography/run-ptest | 4 +- > .../python/python3-cryptography_3.3.2.bb | 68 ---------- > .../python/python3-cryptography_36.0.1.bb | 121 ++++++++++++++++++ > .../python/python3-pyruvate_1.1.2.bb | 119 +++++++++++++++++ > .../python/python3-pytest-subtests_0.5.0.bb | 16 --- > .../python/python3-pytest-subtests_0.6.0.bb | 20 +++ > .../python3-setuptools-rust-native_1.1.2.bb | 26 ++++ > 15 files changed, 403 insertions(+), 158 deletions(-) > create mode 100644 meta-python/classes/pyo3.bbclass > create mode 100644 meta-python/classes/setuptools3_rust.bbclass > rename meta-python/recipes-devtools/python/{ > python3-cryptography-vectors_35.0.0.bb => > python3-cryptography-vectors_36.0.1.bb} (72%) > create mode 100644 > meta-python/recipes-devtools/python/python3-cryptography/0001-Cargo.toml-specify-pem-version.patch > create mode 100644 > meta-python/recipes-devtools/python/python3-cryptography/0002-Cargo.toml-edition-2018-2021.patch > create mode 100755 > meta-python/recipes-devtools/python/python3-cryptography/check-memfree.py > delete mode 100644 > meta-python/recipes-devtools/python/python3-cryptography/h-test.patch > delete mode 100644 > meta-python/recipes-devtools/python/python3-cryptography/openssl3.patch > delete mode 100644 meta-python/recipes-devtools/python/ > python3-cryptography_3.3.2.bb > create mode 100644 meta-python/recipes-devtools/python/ > python3-cryptography_36.0.1.bb > create mode 100644 meta-python/recipes-devtools/python/ > python3-pyruvate_1.1.2.bb > delete mode 100644 meta-python/recipes-devtools/python/ > python3-pytest-subtests_0.5.0.bb > create mode 100644 meta-python/recipes-devtools/python/ > python3-pytest-subtests_0.6.0.bb > create mode 100644 meta-python/recipes-devtools/python/ > python3-setuptools-rust-native_1.1.2.bb > > -- > 2.30.2 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#94875): > https://lists.openembedded.org/g/openembedded-devel/message/94875 > Mute This Topic: https://lists.openembedded.org/mt/88458046/3617605 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ > samuli.piippo@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, Jan 19, 2022 at 4:13 AM Samuli Piippo <samuli.piippo@gmail.com> wrote: > Hi, > > This seems to be causing build break for nativesdk builds. > nativesdk-python3-cryptography: > running build_rust > error: Error loading target specification: Could not find specification > for target "x86_64-pokysdk-linux". Run `rustc --print target-list` for a > list of built-in targets > > We seem to have a problem with Rust in SDK in general, so this is a Rust toolchain problem not a python3-cryptography nor setuptools_rust problem. But that's all I can say for certain. > The same build for target gives only warning: > running build_rust > warning: build_rust: Forced Rust target `aarch64-poky-linux` is not > compatible with deduced Rust target `aarch64-unknown-linux-gnu` - the built > package may not import successfully once installed. > > This warning seems to be harmless, but I don't know how to "fix it". > -samuli > > On Sun, 16 Jan 2022 at 07:29, Tim Orling <ticotimo@gmail.com> wrote: > >> This series upgrades python3-cryptography from the last version to be >> purely Python/C extensions (3.3.2) to the latest version which also >> includes Rust extensions (36.0.1). >> >> This has been tested (including full ptest runs) on: >> - qemux86-64 >> - qemux86-musl >> - qemuarm64 >> >> The test cases require an absolute minimum of 1 GB of free memory >> according to the documentation, but in practice it was more performant >> to have 2 GB of free memory. For this reason a check-memfree.py test is >> added to run-ptest to avoid OOM/Exit code 9 in the ptest-runner. The >> test cases were run with: >> >> runqemu nographic slirp qemuparams="-m 2176 -smp 4" >> >> On x86-64 with kvm this takes about 5 minutes. >> >> The results are 2856+/-2 passed, 1654+/-2 skipped. Of the skipped tests >> 1600 of them are because of unsupported "counter location middle_fixed" >> in https://github.com/pyca/cryptography/pull/2773. A further ~35 test >> cases are skipped because of not setting --wychefproof-root, but even >> when the Google wychefproof repository is cloned and the option is set, >> it adds another ~5000 skipped test cases: >> >> https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/wip-python3-cryptography-ptest&id=fff4da6e70c97265e17a82f1d043ab16039d510e >> >> Upstream uses python3-pytest-xdist to parallize test cases (and >> python3-pytest-shard to distribute amongst workers), but this uses >> stdin/stdout and therefore we have no output to ptest-runner. >> >> This series introduces two helper classes pyo3.bbclass and >> setuptools_rust.bbclass for Python/Rust projects. >> >> The setuptools_rust.bbclass wraps PyO3, Cargo and the newly added >> python3-setuptools-rust-native recipe. Since oe-core only supports >> native rustc at this time, only a native recipe is provided. >> >> To demonstrate that these classes are applicable to multiple recipes, we >> also introduce python3-pyruvate, a WSGI implementation in Rust. >> >> While we are at it we upgrade python3-pytest-subtests from 0.5.0 to >> 0.6.0. >> >> The following changes since commit >> df5c1a31fdb8325d1c514f9ed8a7b3805dc581ac: >> >> python3-diskcache: Upgrade 5.3.0 -> 5.4.0 (2022-01-10 10:35:02 -0800) >> >> are available in the Git repository at: >> >> git://git.openembedded.org/meta-openembedded-contrib >> timo/python3-cryptography_36.0.1 >> >> http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=timo/python3-cryptography_36.0.1 >> >> Tim Orling (7): >> python3-setuptools-rust-native: add v1.1.2 recipe >> pyo3.bbclass: add class for PyO3 cross-compilation >> setuptools3_rust.bbclass: setuptoools Rust plugin >> python3-pyruvate: add recipe for v1.1.2 >> python3-pytest-subtests: upgrade 0.5.0 -> 0.6.0 >> python3-cryptography-vectors: upgrade 35.0.0 => 36.0.1 >> python3-cryptography: upgrade 3.3.2 -> 36.0.1 >> >> meta-python/classes/pyo3.bbclass | 30 +++++ >> meta-python/classes/setuptools3_rust.bbclass | 11 ++ >> ...=> python3-cryptography-vectors_36.0.1.bb} | 5 +- >> .../0001-Cargo.toml-specify-pem-version.patch | 31 +++++ >> .../0002-Cargo.toml-edition-2018-2021.patch | 28 ++++ >> .../python3-cryptography/check-memfree.py | 10 ++ >> .../python/python3-cryptography/h-test.patch | 10 -- >> .../python3-cryptography/openssl3.patch | 62 --------- >> .../python/python3-cryptography/run-ptest | 4 +- >> .../python/python3-cryptography_3.3.2.bb | 68 ---------- >> .../python/python3-cryptography_36.0.1.bb | 121 ++++++++++++++++++ >> .../python/python3-pyruvate_1.1.2.bb | 119 +++++++++++++++++ >> .../python/python3-pytest-subtests_0.5.0.bb | 16 --- >> .../python/python3-pytest-subtests_0.6.0.bb | 20 +++ >> .../python3-setuptools-rust-native_1.1.2.bb | 26 ++++ >> 15 files changed, 403 insertions(+), 158 deletions(-) >> create mode 100644 meta-python/classes/pyo3.bbclass >> create mode 100644 meta-python/classes/setuptools3_rust.bbclass >> rename meta-python/recipes-devtools/python/{ >> python3-cryptography-vectors_35.0.0.bb => >> python3-cryptography-vectors_36.0.1.bb} (72%) >> create mode 100644 >> meta-python/recipes-devtools/python/python3-cryptography/0001-Cargo.toml-specify-pem-version.patch >> create mode 100644 >> meta-python/recipes-devtools/python/python3-cryptography/0002-Cargo.toml-edition-2018-2021.patch >> create mode 100755 >> meta-python/recipes-devtools/python/python3-cryptography/check-memfree.py >> delete mode 100644 >> meta-python/recipes-devtools/python/python3-cryptography/h-test.patch >> delete mode 100644 >> meta-python/recipes-devtools/python/python3-cryptography/openssl3.patch >> delete mode 100644 meta-python/recipes-devtools/python/ >> python3-cryptography_3.3.2.bb >> create mode 100644 meta-python/recipes-devtools/python/ >> python3-cryptography_36.0.1.bb >> create mode 100644 meta-python/recipes-devtools/python/ >> python3-pyruvate_1.1.2.bb >> delete mode 100644 meta-python/recipes-devtools/python/ >> python3-pytest-subtests_0.5.0.bb >> create mode 100644 meta-python/recipes-devtools/python/ >> python3-pytest-subtests_0.6.0.bb >> create mode 100644 meta-python/recipes-devtools/python/ >> python3-setuptools-rust-native_1.1.2.bb >> >> -- >> 2.30.2 >> >> >> -=-=-=-=-=-=-=-=-=-=-=- >> Links: You receive all messages sent to this group. >> View/Reply Online (#94875): >> https://lists.openembedded.org/g/openembedded-devel/message/94875 >> Mute This Topic: https://lists.openembedded.org/mt/88458046/3617605 >> Group Owner: openembedded-devel+owner@lists.openembedded.org >> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ >> samuli.piippo@gmail.com] >> -=-=-=-=-=-=-=-=-=-=-=- >> >>
This series upgrades python3-cryptography from the last version to be purely Python/C extensions (3.3.2) to the latest version which also includes Rust extensions (36.0.1). This has been tested (including full ptest runs) on: - qemux86-64 - qemux86-musl - qemuarm64 The test cases require an absolute minimum of 1 GB of free memory according to the documentation, but in practice it was more performant to have 2 GB of free memory. For this reason a check-memfree.py test is added to run-ptest to avoid OOM/Exit code 9 in the ptest-runner. The test cases were run with: runqemu nographic slirp qemuparams="-m 2176 -smp 4" On x86-64 with kvm this takes about 5 minutes. The results are 2856+/-2 passed, 1654+/-2 skipped. Of the skipped tests 1600 of them are because of unsupported "counter location middle_fixed" in https://github.com/pyca/cryptography/pull/2773. A further ~35 test cases are skipped because of not setting --wychefproof-root, but even when the Google wychefproof repository is cloned and the option is set, it adds another ~5000 skipped test cases: https://git.openembedded.org/meta-openembedded-contrib/commit/?h=timo/wip-python3-cryptography-ptest&id=fff4da6e70c97265e17a82f1d043ab16039d510e Upstream uses python3-pytest-xdist to parallize test cases (and python3-pytest-shard to distribute amongst workers), but this uses stdin/stdout and therefore we have no output to ptest-runner. This series introduces two helper classes pyo3.bbclass and setuptools_rust.bbclass for Python/Rust projects. The setuptools_rust.bbclass wraps PyO3, Cargo and the newly added python3-setuptools-rust-native recipe. Since oe-core only supports native rustc at this time, only a native recipe is provided. To demonstrate that these classes are applicable to multiple recipes, we also introduce python3-pyruvate, a WSGI implementation in Rust. While we are at it we upgrade python3-pytest-subtests from 0.5.0 to 0.6.0. The following changes since commit df5c1a31fdb8325d1c514f9ed8a7b3805dc581ac: python3-diskcache: Upgrade 5.3.0 -> 5.4.0 (2022-01-10 10:35:02 -0800) are available in the Git repository at: git://git.openembedded.org/meta-openembedded-contrib timo/python3-cryptography_36.0.1 http://cgit.openembedded.org/meta-openembedded-contrib/log/?h=timo/python3-cryptography_36.0.1 Tim Orling (7): python3-setuptools-rust-native: add v1.1.2 recipe pyo3.bbclass: add class for PyO3 cross-compilation setuptools3_rust.bbclass: setuptoools Rust plugin python3-pyruvate: add recipe for v1.1.2 python3-pytest-subtests: upgrade 0.5.0 -> 0.6.0 python3-cryptography-vectors: upgrade 35.0.0 => 36.0.1 python3-cryptography: upgrade 3.3.2 -> 36.0.1 meta-python/classes/pyo3.bbclass | 30 +++++ meta-python/classes/setuptools3_rust.bbclass | 11 ++ ...=> python3-cryptography-vectors_36.0.1.bb} | 5 +- .../0001-Cargo.toml-specify-pem-version.patch | 31 +++++ .../0002-Cargo.toml-edition-2018-2021.patch | 28 ++++ .../python3-cryptography/check-memfree.py | 10 ++ .../python/python3-cryptography/h-test.patch | 10 -- .../python3-cryptography/openssl3.patch | 62 --------- .../python/python3-cryptography/run-ptest | 4 +- .../python/python3-cryptography_3.3.2.bb | 68 ---------- .../python/python3-cryptography_36.0.1.bb | 121 ++++++++++++++++++ .../python/python3-pyruvate_1.1.2.bb | 119 +++++++++++++++++ .../python/python3-pytest-subtests_0.5.0.bb | 16 --- .../python/python3-pytest-subtests_0.6.0.bb | 20 +++ .../python3-setuptools-rust-native_1.1.2.bb | 26 ++++ 15 files changed, 403 insertions(+), 158 deletions(-) create mode 100644 meta-python/classes/pyo3.bbclass create mode 100644 meta-python/classes/setuptools3_rust.bbclass rename meta-python/recipes-devtools/python/{python3-cryptography-vectors_35.0.0.bb => python3-cryptography-vectors_36.0.1.bb} (72%) create mode 100644 meta-python/recipes-devtools/python/python3-cryptography/0001-Cargo.toml-specify-pem-version.patch create mode 100644 meta-python/recipes-devtools/python/python3-cryptography/0002-Cargo.toml-edition-2018-2021.patch create mode 100755 meta-python/recipes-devtools/python/python3-cryptography/check-memfree.py delete mode 100644 meta-python/recipes-devtools/python/python3-cryptography/h-test.patch delete mode 100644 meta-python/recipes-devtools/python/python3-cryptography/openssl3.patch delete mode 100644 meta-python/recipes-devtools/python/python3-cryptography_3.3.2.bb create mode 100644 meta-python/recipes-devtools/python/python3-cryptography_36.0.1.bb create mode 100644 meta-python/recipes-devtools/python/python3-pyruvate_1.1.2.bb delete mode 100644 meta-python/recipes-devtools/python/python3-pytest-subtests_0.5.0.bb create mode 100644 meta-python/recipes-devtools/python/python3-pytest-subtests_0.6.0.bb create mode 100644 meta-python/recipes-devtools/python/python3-setuptools-rust-native_1.1.2.bb