diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index cf817e954..de1ff8765 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -513,6 +513,79 @@ system and gives an overview of their function and contents. clean up your :term:`DL_DIR` directory by deleting any Git or other source control work directories. + :term:`BB_PRESSURE_MAX_CPU` + The threshold for maximum CPU pressure before BitBake prevents the + scheduling of new tasks. Once the :term:`BB_PRESSURE_MAX_CPU` threshold + is exceeded, new tasks are not started until the pressure subsides to + below the threshold. If :term:`BB_PRESSURE_MAX_CPU` is not set, CPU + pressure is not monitored. + :: + + BB_PRESSURE_MAX_CPU = "500" + + A suitable value for :term:`BB_PRESSURE_MAX_CPU` can be selected by + inspecting + ``${``\ :term:`BUILDSTATS_BASE`\ ``}/reduced_proc_pressure/cpu.log`` + The format of this file looks similar to:: + + 1657905652 + 0.36 0.08 0.7 142113 + + The first line of the file is the sampling time, 1657905652 in this case. + The second line represents the "some" line from /proc/pressure/cpu. From + this line, the last field (142113 in this case), is relevant. It + represents the difference in "some total" pressure from the previous + sample and is the value that is monitored by BitBake when + :term:`BB_PRESSURE_MAX_CPU` is set. However, while + :ref:`buildstats ` samples approximately once a + second, for a majority of builds BitBake schedules tasks anywhere from + 10-100s of times per second. Given this, a proportionally smaller value + than the threshold determined via ``/reduced_proc_pressure/cpu.log`` + is reasonable. + + By looking at the distribution of values in + ``/reduced_proc_pressure/cpu.log``, one can see that the pressure usually + varies by several orders of magnitude. Currently, there is no alogrithm + for picking a precise threshold but values lower than 100 or higher than + 1000 over/under regulate builds respectively. + + More information about /proc/pressure and each of the fields logged can + be found at `Pressure Stall Information + `__. + + .. note:: + + The one caveat in regulating builds using :term:`BB_PRESSURE_MAX_CPU` + is that Bitbake is designed to always have 1 active task running. This + means that even if the threshold is exceeded, a new task will be + started if there is no other active task, potentially leading to + higher pressures experienced than the value of + :term:`BB_PRESSURE_MAX_CPU`. Furthermore, there is some delay between + starting tasks and observing increased pressure so when large tasks + start below the pressure threshold, there will often be extended + periods of high pressure until the project build tools, such as make + and ninja, are patched to also adhere to the pressure regulation + scheme. + + :term:`BB_PRESSURE_MAX_IO` + The threshold for maximum IO pressure experienced before BitBake + prevents the scheduling of new tasks. The IO pressure is regulated in the + same way as :term:`BB_PRESSURE_MAX_CPU` except that + ``${``\ :term:`BUILDSTATS_BASE`\ ``}/reduced_proc_pressure/io.log`` + can be used to gauge suitable values to use. At this point in time, experience + shows that IO pressure tends to be short-lived and regulating just the CPU can + help to reduce it. + + :term:`BB_PRESSURE_MAX_MEMORY` + The threshold for maximum memory pressure experienced before BitBake + prevents the scheduling of new tasks. The memory pressure is + regulated in the same way as :term:`BB_PRESSURE_MAX_CPU` except that + ``${``\ :term:`BUILDSTATS_BASE`\ ``}/reduced_proc_pressure/memory.log`` + can be used to gauge suitable values to use. Note that any memory + pressure indicates that a system is being pushed beyond it's capacity. At + this point in time, experience shows that memory pressure tends to be + short-lived and regulating just the CPU can help to reduce it. + :term:`BB_NUMBER_THREADS` The maximum number of tasks BitBake should run in parallel at any one time. The OpenEmbedded build system automatically configures this diff --git a/documentation/ref-manual/varlocality.rst b/documentation/ref-manual/varlocality.rst index e2c086ffa..d4d6d06ee 100644 --- a/documentation/ref-manual/varlocality.rst +++ b/documentation/ref-manual/varlocality.rst @@ -88,6 +88,12 @@ configuration through the ``local.conf`` file. - :term:`PACKAGE_CLASSES` +- :term:`BB_PRESSURE_MAX_CPU` + +- :term:`BB_PRESSURE_MAX_IO` + +- :term:`BB_PRESSURE_MAX_MEMORY` + - :term:`BB_NUMBER_THREADS` - :term:`BBINCLUDELOGS`