From patchwork Mon Mar 25 13:27:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63ACCCD1281 for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by mx.groups.io with SMTP id smtpd.web11.52058.1711373265750571006 for ; Mon, 25 Mar 2024 06:27:46 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=HpurgoVo; spf=pass (domain: bootlin.com, ip: 217.70.183.200, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id BD82F20004; Mon, 25 Mar 2024 13:27:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/6lrhs6CVPZ2w09S31sDqoZua3CqXE4DoaoLcQIRyNM=; b=HpurgoVoRoaRAdcdjBW8vDhssZ6pffC2FbczNqag4l35lrqSIACo+9Dw+zmzWZXcJyZBvk ODjpKAZKwMnYfYCYXfrRSybNaoKgKwosR+1TyurZHLZzK28omFGeiEeWyvpA6HmlKe7D1D B2DYk1086Rp2dWSrd5q+i4B0RRyE8tQs9rdpKxhHHISBVx9kNd0zNwAK8ceYrVmUgYq1zJ djJIWjYnDDEkdyBsVfukfgtCIaSOMwNGFgIpCXfO1yDF+/XW7ukcfaQLRMgw0erPp/gdcN APobziyh9LmvqYiE7URF2eH9TyWaWxgiSsziKXJYyecUCOCnVf0Z7b5G3t2w0w== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , Thomas Petazzoni Subject: [kirkstone][PATCH 1/9] manuals: add initial stylechecks with Vale Date: Mon, 25 Mar 2024 14:27:29 +0100 Message-Id: <20240325132737.3018073-2-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/4999 From: Michael Opdenacker Use the "Vale" (https://vale.sh) tool to perform text style checks Run "make stylecheck" to run the checks. This just checks the text, not the Sphinx syntax style choices. Signed-off-by: Michael Opdenacker Suggested-by: Thomas Petazzoni --- documentation/.gitignore | 2 ++ documentation/.vale.ini | 7 +++++++ documentation/Makefile | 13 ++++++++++++- documentation/README | 14 ++++++++++++++ documentation/standards.md | 15 +++++++++++++++ .../config/vocabularies/OpenSource/accept.txt | 4 ++++ .../styles/config/vocabularies/Yocto/accept.txt | 4 ++++ 7 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 documentation/.vale.ini create mode 100644 documentation/styles/config/vocabularies/OpenSource/accept.txt create mode 100644 documentation/styles/config/vocabularies/Yocto/accept.txt diff --git a/documentation/.gitignore b/documentation/.gitignore index 096b97ec28..a8f86f7c98 100644 --- a/documentation/.gitignore +++ b/documentation/.gitignore @@ -5,3 +5,5 @@ sphinx-static/switchers.js .vscode/ */svg/*.png */svg/*.pdf +styles/* +!styles/config diff --git a/documentation/.vale.ini b/documentation/.vale.ini new file mode 100644 index 0000000000..02042bb632 --- /dev/null +++ b/documentation/.vale.ini @@ -0,0 +1,7 @@ +StylesPath = styles +MinAlertLevel = suggestion +Packages = RedHat, proselint, write-good, alex, Readability, Joblint +Vocab = Yocto, OpenSource +[*.rst] +BasedOnStyles = Vale, RedHat, proselint, write-good, alex, Readability, Joblint + diff --git a/documentation/Makefile b/documentation/Makefile index 33bbca0bab..e5f0effeb5 100644 --- a/documentation/Makefile +++ b/documentation/Makefile @@ -5,6 +5,9 @@ # from the environment for the first two. SPHINXOPTS ?= -W --keep-going -j auto SPHINXBUILD ?= sphinx-build +# Release notes are excluded because they contain contributor names and commit messages which can't be modified +VALEOPTS ?= --no-wrap --glob '!migration-guides/release-notes-*.rst' +VALEDOCS ?= . SOURCEDIR = . IMAGEDIRS = */svg BUILDDIR = _build @@ -20,7 +23,7 @@ endif help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -.PHONY: all help Makefile clean publish epub latexpdf +.PHONY: all help Makefile clean stylecheck publish epub latexpdf publish: Makefile html singlehtml rm -rf $(BUILDDIR)/$(DESTDIR)/ @@ -46,6 +49,14 @@ PNGs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.png,$(wildcard $(SOURCED clean: @rm -rf $(BUILDDIR) $(PNGs) $(PDFs) poky.yaml sphinx-static/switchers.js +stylecheck: + vale sync + vale $(VALEOPTS) $(VALEDOCS) + +stylecheck: + vale sync + vale $(VALEOPTS) $(VALEDOCS) + epub: $(PNGs) @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/documentation/README b/documentation/README index 4d31036e69..8035418cac 100644 --- a/documentation/README +++ b/documentation/README @@ -151,6 +151,20 @@ dependencies in a virtual environment: $ pipenv install $ pipenv run make html +Style checking the Yocto Project documentation +============================================== + +The project is starting to use Vale (https://vale.sh/) +to validate the text style. + +To install Vale: + + $ pip install vale + +To run Vale: + + $ make stylecheck + Sphinx theme and CSS customization ================================== diff --git a/documentation/standards.md b/documentation/standards.md index a2274f6d6e..d3b25adfab 100644 --- a/documentation/standards.md +++ b/documentation/standards.md @@ -5,6 +5,21 @@ documentation is created. It is currently a work in progress. +## Automatic style validation + +There is an ongoing effort to automate style validation +through the [Vale](https://vale.sh/). To try it, run: + + $ make stylecheck + +Note that this just applies to text. Therefore, the syntax +conventions described below still apply. + +If you wish to add a new word to an "accept.txt" file +(./styles/config/vocabularies//accept.txt), +make sure the spelling and capitalization matches +what Wikipedia or the project defining this word uses. + ## Text standards This section has not been filled yet diff --git a/documentation/styles/config/vocabularies/OpenSource/accept.txt b/documentation/styles/config/vocabularies/OpenSource/accept.txt new file mode 100644 index 0000000000..98e76ae1f5 --- /dev/null +++ b/documentation/styles/config/vocabularies/OpenSource/accept.txt @@ -0,0 +1,4 @@ +ftrace +toolchain +systemd +LTTng diff --git a/documentation/styles/config/vocabularies/Yocto/accept.txt b/documentation/styles/config/vocabularies/Yocto/accept.txt new file mode 100644 index 0000000000..b725414014 --- /dev/null +++ b/documentation/styles/config/vocabularies/Yocto/accept.txt @@ -0,0 +1,4 @@ +Yocto +BSP +BitBake +OpenEmbedded From patchwork Mon Mar 25 13:27:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42137CD1283 for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by mx.groups.io with SMTP id smtpd.web10.51972.1711373266910278419 for ; Mon, 25 Mar 2024 06:27:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=akx2h9ZL; spf=pass (domain: bootlin.com, ip: 217.70.183.193, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id C2E6E240009; Mon, 25 Mar 2024 13:27:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373264; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=smqtBigxBKDI7Rr1LNsaAZj1BA9q9b4r7a/sIN5BO2g=; b=akx2h9ZLzga1bHRd8cQylWQfqwYtEeH8M2ygpxHClJFff2zDQpXW/4jvjt67FjJQsOvKkT vkSmCPolRTPuFIgUsuAAgZJSbe2UTXYclNYNaEuG6BD9HTm9s6zmrseo17J9jbNO5XmeJV 2t5/2AMglWVTxKv4z7DPwXSRDMpj2qKx/aFhYp+1OdEC4Ejj3YebSAXclAlZbRUOy9wP7C A69mo0s8G6cN3kSCGBpi5m3WyWDtlQ+1ISEg3v+xpf2rbolUPl4JojqhCUVnXmya1Eyegw XM8gdRU8dYC9LxXJEfFIFB/0B4EODuNG4bG1gHXe3nYuChNC4g4zu6v8Qi8exQ== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker Subject: [kirkstone][PATCH 2/9] profile-manual: usage.rst: formatting fixes Date: Mon, 25 Mar 2024 14:27:30 +0100 Message-Id: <20240325132737.3018073-3-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5002 From: Michael Opdenacker Plus a few text styling improvements, some reported by "make stylecheck" Signed-off-by: Michael Opdenacker --- documentation/profile-manual/usage.rst | 473 +++++++++++++------------ 1 file changed, 238 insertions(+), 235 deletions(-) diff --git a/documentation/profile-manual/usage.rst b/documentation/profile-manual/usage.rst index a190e743cd..912e4c9602 100644 --- a/documentation/profile-manual/usage.rst +++ b/documentation/profile-manual/usage.rst @@ -10,10 +10,10 @@ Basic Usage (with examples) for each of the Yocto Tracing Tools This chapter presents basic usage examples for each of the tracing tools. -perf +Perf ==== -The 'perf' tool is the profiling and tracing tool that comes bundled +The perf tool is the profiling and tracing tool that comes bundled with the Linux kernel. Don't let the fact that it's part of the kernel fool you into thinking @@ -61,7 +61,7 @@ Basic Perf Usage ---------------- The perf tool is pretty much self-documenting. To remind yourself of the -available commands, simply type 'perf', which will show you basic usage +available commands, simply type ``perf``, which will show you basic usage along with the available perf subcommands:: root@crownbay:~# perf @@ -97,19 +97,19 @@ along with the available perf subcommands:: Using perf to do Basic Profiling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -As a simple test case, we'll profile the 'wget' of a fairly large file, +As a simple test case, we'll profile the ``wget`` of a fairly large file, which is a minimally interesting case because it has both file and network I/O aspects, and at least in the case of standard Yocto images, it's implemented as part of BusyBox, so the methods we use to analyze it can be used in a very similar way to the whole host of supported BusyBox -applets in Yocto. :: +applets in Yocto:: root@crownbay:~# rm linux-2.6.19.2.tar.bz2; \ wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 The quickest and easiest way to get some basic overall data about what's -going on for a particular workload is to profile it using 'perf stat'. -'perf stat' basically profiles using a few default counters and displays +going on for a particular workload is to profile it using ``perf stat``. +This command basically profiles using a few default counters and displays the summed counts at the end of the run:: root@crownbay:~# perf stat wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 @@ -135,9 +135,9 @@ Many times such a simple-minded test doesn't yield much of interest, but sometimes it does (see Real-world Yocto bug (slow loop-mounted write speed)). -Also, note that 'perf stat' isn't restricted to a fixed set of counters -- basically any event listed in the output of 'perf list' can be tallied -by 'perf stat'. For example, suppose we wanted to see a summary of all +Also, note that ``perf stat`` isn't restricted to a fixed set of counters +--- basically any event listed in the output of ``perf list`` can be tallied +by ``perf stat``. For example, suppose we wanted to see a summary of all the events related to kernel memory allocation/freeing along with cache hits and misses:: @@ -164,22 +164,22 @@ hits and misses:: 44.831023415 seconds time elapsed -So 'perf stat' gives us a nice easy +So ``perf stat`` gives us a nice easy way to get a quick overview of what might be happening for a set of events, but normally we'd need a little more detail in order to understand what's going on in a way that we can act on in a useful way. -To dive down into a next level of detail, we can use 'perf record'/'perf -report' which will collect profiling data and present it to use using an +To dive down into a next level of detail, we can use ``perf record`` / +``perf report`` which will collect profiling data and present it to use using an interactive text-based UI (or simply as text if we specify ``--stdio`` to -'perf report'). +``perf report``). -As our first attempt at profiling this workload, we'll simply run 'perf -record', handing it the workload we want to profile (everything after -'perf record' and any perf options we hand it --- here none, will be +As our first attempt at profiling this workload, we'll simply run ``perf +record``, handing it the workload we want to profile (everything after +``perf record`` and any perf options we hand it --- here none, will be executed in a new shell). perf collects samples until the process exits -and records them in a file named 'perf.data' in the current working -directory. :: +and records them in a file named ``perf.data`` in the current working +directory:: root@crownbay:~# perf record wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 @@ -189,7 +189,7 @@ directory. :: [ perf record: Captured and wrote 0.176 MB perf.data (~7700 samples) ] To see the results in a -'text-based UI' (tui), simply run 'perf report', which will read the +"text-based UI" (tui), simply run ``perf report``, which will read the perf.data file in the current working directory and display the results in an interactive UI:: @@ -198,24 +198,24 @@ in an interactive UI:: .. image:: figures/perf-wget-flat-stripped.png :align: center -The above screenshot displays a 'flat' profile, one entry for each -'bucket' corresponding to the functions that were profiled during the +The above screenshot displays a "flat" profile, one entry for each +"bucket" corresponding to the functions that were profiled during the profiling run, ordered from the most popular to the least (perf has options to sort in various orders and keys as well as display entries only above a certain threshold and so on --- see the perf documentation for details). Note that this includes both userspace functions (entries -containing a [.]) and kernel functions accounted to the process (entries -containing a [k]). (perf has command-line modifiers that can be used to -restrict the profiling to kernel or userspace, among others). +containing a ``[.]``) and kernel functions accounted to the process (entries +containing a ``[k]``). perf has command-line modifiers that can be used to +restrict the profiling to kernel or userspace, among others. -Notice also that the above report shows an entry for 'busybox', which is -the executable that implements 'wget' in Yocto, but that instead of a +Notice also that the above report shows an entry for ``busybox``, which is +the executable that implements ``wget`` in Yocto, but that instead of a useful function name in that entry, it displays a not-so-friendly hex value instead. The steps below will show how to fix that problem. Before we do that, however, let's try running a different profile, one which shows something a little more interesting. The only difference -between the new profile and the previous one is that we'll add the -g +between the new profile and the previous one is that we'll add the ``-g`` option, which will record not just the address of a sampled function, but the entire callchain to the sampled function as well:: @@ -236,11 +236,11 @@ took the most time, but we can also see a summary of how those functions were called and learn something about how the program interacts with the kernel in the process. -Notice that each entry in the above screenshot now contains a '+' on the +Notice that each entry in the above screenshot now contains a ``+`` on the left-hand side. This means that we can expand the entry and drill down -into the callchains that feed into that entry. Pressing 'enter' on any -one of them will expand the callchain (you can also press 'E' to expand -them all at the same time or 'C' to collapse them all). +into the callchains that feed into that entry. Pressing ``Enter`` on any +one of them will expand the callchain (you can also press ``E`` to expand +them all at the same time or ``C`` to collapse them all). In the screenshot above, we've toggled the ``__copy_to_user_ll()`` entry and several subnodes all the way down. This lets us see which callchains @@ -251,13 +251,13 @@ As a bit of background explanation for these callchains, think about what happens at a high level when you run wget to get a file out on the network. Basically what happens is that the data comes into the kernel via the network connection (socket) and is passed to the userspace -program 'wget' (which is actually a part of BusyBox, but that's not +program ``wget`` (which is actually a part of BusyBox, but that's not important for now), which takes the buffers the kernel passes to it and writes it to a disk file to save it. The part of this process that we're looking at in the above call stacks is the part where the kernel passes the data it has read from the socket -down to wget i.e. a copy-to-user. +down to wget i.e. a ``copy-to-user``. Notice also that here there's also a case where the hex value is displayed in the callstack, here in the expanded ``sys_clock_gettime()`` @@ -267,8 +267,8 @@ busybox. .. image:: figures/perf-wget-g-copy-from-user-expanded-stripped.png :align: center -The above screenshot shows the other half of the journey for the data - -from the wget program's userspace buffers to disk. To get the buffers to +The above screenshot shows the other half of the journey for the data --- +from the ``wget`` program's userspace buffers to disk. To get the buffers to disk, the wget program issues a ``write(2)``, which does a ``copy-from-user`` to the kernel, which then takes care via some circuitous path (probably also present somewhere in the profile data), to get it safely to disk. @@ -278,7 +278,7 @@ of how to extract useful information out of it, let's get back to the task at hand and see if we can get some basic idea about where the time is spent in the program we're profiling, wget. Remember that wget is actually implemented as an applet in BusyBox, so while the process name -is 'wget', the executable we're actually interested in is BusyBox. So +is ``wget``, the executable we're actually interested in is BusyBox. So let's expand the first entry containing BusyBox: .. image:: figures/perf-wget-busybox-expanded-stripped.png @@ -307,16 +307,15 @@ To generate the debug info for the packages in the image, we can add EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs" Additionally, in order to generate the type of debuginfo that perf -understands, we also need to set -:term:`PACKAGE_DEBUG_SPLIT_STYLE` +understands, we also need to set :term:`PACKAGE_DEBUG_SPLIT_STYLE` in the ``local.conf`` file:: PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory' -Once we've done that, we can install the -debuginfo for BusyBox. The debug packages once built can be found in -``build/tmp/deploy/rpm/*`` on the host system. Find the busybox-dbg-...rpm -file and copy it to the target. For example:: +Once we've done that, we can install the debuginfo for BusyBox. The +debug packages once built can be found in ``build/tmp/deploy/rpm/*`` +on the host system. Find the ``busybox-dbg-...rpm`` file and copy it +to the target. For example:: [trz@empanada core2]$ scp /home/trz/yocto/crownbay-tracing-dbg/build/tmp/deploy/rpm/core2_32/busybox-dbg-1.20.2-r2.core2_32.rpm root@192.168.1.31: busybox-dbg-1.20.2-r2.core2_32.rpm 100% 1826KB 1.8MB/s 00:01 @@ -331,7 +330,7 @@ their functions symbolically: .. image:: figures/perf-wget-busybox-debuginfo.png :align: center -If we expand one of the entries and press 'enter' on a leaf node, we're +If we expand one of the entries and press ``Enter`` on a leaf node, we're presented with a menu of actions we can take to get more information related to that entry: @@ -340,7 +339,7 @@ related to that entry: One of these actions allows us to show a view that displays a busybox-centric view of the profiled functions (in this case we've also -expanded all the nodes using the 'E' key): +expanded all the nodes using the ``E`` key): .. image:: figures/perf-wget-busybox-dso-zoom.png :align: center @@ -348,8 +347,8 @@ expanded all the nodes using the 'E' key): Finally, we can see that now that the BusyBox debuginfo is installed, the previously unresolved symbol in the ``sys_clock_gettime()`` entry mentioned previously is now resolved, and shows that the -sys_clock_gettime system call that was the source of 6.75% of the -copy-to-user overhead was initiated by the ``handle_input()`` BusyBox +``sys_clock_gettime`` system call that was the source of 6.75% of the +``copy-to-user`` overhead was initiated by the ``handle_input()`` BusyBox function: .. image:: figures/perf-wget-g-copy-to-user-expanded-debuginfo.png @@ -357,14 +356,14 @@ function: At the lowest level of detail, we can dive down to the assembly level and see which instructions caused the most overhead in a function. -Pressing 'enter' on the 'udhcpc_main' function, we're again presented +Pressing ``Enter`` on the ``udhcpc_main`` function, we're again presented with a menu: .. image:: figures/perf-wget-busybox-annotate-menu.png :align: center -Selecting 'Annotate udhcpc_main', we get a detailed listing of -percentages by instruction for the udhcpc_main function. From the +Selecting ``Annotate udhcpc_main``, we get a detailed listing of +percentages by instruction for the ``udhcpc_main`` function. From the display, we can see that over 50% of the time spent in this function is taken up by a couple tests and the move of a constant (1) to a register: @@ -372,7 +371,7 @@ taken up by a couple tests and the move of a constant (1) to a register: :align: center As a segue into tracing, let's try another profile using a different -counter, something other than the default 'cycles'. +counter, something other than the default ``cycles``. The tracing and profiling infrastructure in Linux has become unified in a way that allows us to use the same tool with a completely different @@ -382,7 +381,7 @@ traditional tools can also make use of the expanded possibilities now available to them, and in some cases have, as mentioned previously). We can get a list of the available events that can be used to profile a -workload via 'perf list':: +workload via ``perf list``:: root@crownbay:~# perf list @@ -518,13 +517,13 @@ workload via 'perf list':: .. admonition:: Tying it Together These are exactly the same set of events defined by the trace event - subsystem and exposed by ftrace/tracecmd/kernelshark as files in - /sys/kernel/debug/tracing/events, by SystemTap as + subsystem and exposed by ftrace / tracecmd / kernelshark as files in + ``/sys/kernel/debug/tracing/events``, by SystemTap as kernel.trace("tracepoint_name") and (partially) accessed by LTTng. Only a subset of these would be of interest to us when looking at this workload, so let's choose the most likely subsystems (identified by the -string before the colon in the Tracepoint events) and do a 'perf stat' +string before the colon in the Tracepoint events) and do a ``perf stat`` run using only those wildcarded subsystems:: root@crownbay:~# perf stat -e skb:* -e net:* -e napi:* -e sched:* -e workqueue:* -e irq:* -e syscalls:* wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 @@ -596,8 +595,8 @@ and tell perf to do a profile using it as the sampling event:: The screenshot above shows the results of running a profile using sched:sched_switch tracepoint, which shows the relative costs of various -paths to sched_wakeup (note that sched_wakeup is the name of the -tracepoint --- it's actually defined just inside ttwu_do_wakeup(), which +paths to ``sched_wakeup`` (note that ``sched_wakeup`` is the name of the +tracepoint --- it's actually defined just inside ``ttwu_do_wakeup()``, which accounts for the function name actually displayed in the profile: .. code-block:: c @@ -620,10 +619,10 @@ receive paths that presumably end up waking up wget (busybox) when network data is ready. Note that because tracepoints are normally used for tracing, the default -sampling period for tracepoints is 1 i.e. for tracepoints perf will -sample on every event occurrence (this can be changed using the -c +sampling period for tracepoints is ``1`` i.e. for tracepoints perf will +sample on every event occurrence (this can be changed using the ``-c`` option). This is in contrast to hardware counters such as for example -the default 'cycles' hardware counter used for normal profiling, where +the default ``cycles`` hardware counter used for normal profiling, where sampling periods are much higher (in the thousands) because profiling should have as low an overhead as possible and sampling on every cycle would be prohibitively expensive. @@ -634,10 +633,10 @@ Using perf to do Basic Tracing Profiling is a great tool for solving many problems or for getting a high-level view of what's going on with a workload or across the system. It is however by definition an approximation, as suggested by the most -prominent word associated with it, 'sampling'. On the one hand, it +prominent word associated with it, ``sampling``. On the one hand, it allows a representative picture of what's going on in the system to be cheaply taken, but on the other hand, that cheapness limits its utility -when that data suggests a need to 'dive down' more deeply to discover +when that data suggests a need to "dive down" more deeply to discover what's really going on. In such cases, the only way to see what's really going on is to be able to look at (or summarize more intelligently) the individual steps that go into the higher-level behavior exposed by the @@ -650,7 +649,7 @@ applicable to our workload:: -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 -We can look at the raw trace output using 'perf script' with no +We can look at the raw trace output using ``perf script`` with no arguments:: root@crownbay:~# perf script @@ -681,7 +680,7 @@ arguments:: This gives us a detailed timestamped sequence of events that occurred within the workload with respect to those events. -In many ways, profiling can be viewed as a subset of tracing - +In many ways, profiling can be viewed as a subset of tracing --- theoretically, if you have a set of trace events that's sufficient to capture all the important aspects of a workload, you can derive any of the results or views that a profiling run can. @@ -706,7 +705,7 @@ problem at hand. For example, if we wanted to make use of a 'counter' that maps to the value of the time difference between when a process was scheduled to run on a processor and the time it actually ran, we wouldn't expect such a counter to exist on its own, but we could derive -one called say 'wakeup_latency' and use it to extract a useful view of +one called say ``wakeup_latency`` and use it to extract a useful view of that metric from trace data. Likewise, we really can't figure out from standard profiling tools how much data every process on the system reads and writes, along with how many of those reads and writes fail @@ -715,7 +714,7 @@ right tools easily extract and present that information, but we'd need something other than pre-canned profiling tools to do that. Luckily, there is a general-purpose way to handle such needs, called -'programming languages'. Making programming languages easily available +"programming languages". Making programming languages easily available to apply to such problems given the specific format of data is called a 'programming language binding' for that data and language. Perf supports two programming language bindings, one for Python and one for Perl. @@ -733,15 +732,15 @@ two programming language bindings, one for Python and one for Perl. created an elaborate compiler-based machinery to translate its language into kernel modules written in C. -Now that we have the trace data in perf.data, we can use 'perf script --g' to generate a skeleton script with handlers for the read/write -entry/exit events we recorded:: +Now that we have the trace data in ``perf.data``, we can use ``perf script +-g`` to generate a skeleton script with handlers for the read / write +entry / exit events we recorded:: root@crownbay:~# perf script -g python generated Python script: perf-script.py -The skeleton script simply creates a python function for each event type in the -perf.data file. The body of each function simply prints the event name along +The skeleton script simply creates a Python function for each event type in the +``perf.data`` file. The body of each function simply prints the event name along with its parameters. For example: .. code-block:: python @@ -755,7 +754,7 @@ with its parameters. For example: print "skbaddr=%u, len=%u, name=%s\n" % (skbaddr, len, name), We can run that script directly to print all of the events contained in the -perf.data file:: +``perf.data`` file:: root@crownbay:~# perf script -s perf-script.py @@ -784,8 +783,8 @@ perf.data file:: syscalls__sys_exit_read 1 11624.859944032 1262 wget nr=3, ret=1024 That in itself isn't very useful; after all, we can accomplish pretty much the -same thing by simply running 'perf script' without arguments in the same -directory as the perf.data file. +same thing by simply running ``perf script`` without arguments in the same +directory as the ``perf.data`` file. We can however replace the print statements in the generated function bodies with whatever we want, and thereby make it infinitely more @@ -806,8 +805,8 @@ event. For example: Each event handler function in the generated code is modified to do this. For convenience, we define a common function -called inc_counts() that each handler calls; inc_counts() simply tallies -a count for each event using the 'counts' hash, which is a specialized +called ``inc_counts()`` that each handler calls; ``inc_counts()`` simply tallies +a count for each event using the ``counts`` hash, which is a specialized hash function that does Perl-like autovivification, a capability that's extremely useful for kinds of multi-level aggregation commonly used in processing traces (see perf's documentation on the Python language @@ -825,7 +824,7 @@ binding for details): Finally, at the end of the trace processing run, we want to print the result of all the per-event tallies. For that, we use the special -'trace_end()' function: +``trace_end()`` function: .. code-block:: python @@ -854,7 +853,7 @@ The end result is a summary of all the events recorded in the trace:: syscalls__sys_exit_write 8990 Note that this is -pretty much exactly the same information we get from 'perf stat', which +pretty much exactly the same information we get from ``perf stat``, which goes a little way to support the idea mentioned previously that given the right kind of trace data, higher-level profiling-type summaries can be derived from it. @@ -867,29 +866,29 @@ System-Wide Tracing and Profiling The examples so far have focused on tracing a particular program or workload --- in other words, every profiling run has specified the program -to profile in the command-line e.g. 'perf record wget ...'. +to profile in the command-line e.g. ``perf record wget ...``. It's also possible, and more interesting in many cases, to run a system-wide profile or trace while running the workload in a separate shell. -To do system-wide profiling or tracing, you typically use the -a flag to -'perf record'. +To do system-wide profiling or tracing, you typically use the ``-a`` flag to +``perf record``. To demonstrate this, open up one window and start the profile using the --a flag (press Ctrl-C to stop tracing):: +``-a`` flag (press ``Ctrl-C`` to stop tracing):: root@crownbay:~# perf record -g -a ^C[ perf record: Woken up 6 times to write data ] [ perf record: Captured and wrote 1.400 MB perf.data (~61172 samples) ] -In another window, run the wget test:: +In another window, run the ``wget`` test:: root@crownbay:~# wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 Connecting to downloads.yoctoproject.org (140.211.169.59:80) linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA -Here we see entries not only for our wget load, but for +Here we see entries not only for our ``wget`` load, but for other processes running on the system as well: .. image:: figures/perf-systemwide.png @@ -897,12 +896,12 @@ other processes running on the system as well: In the snapshot above, we can see callchains that originate in libc, and a callchain from Xorg that demonstrates that we're using a proprietary X -driver in userspace (notice the presence of 'PVR' and some other +driver in userspace (notice the presence of ``PVR`` and some other unresolvable symbols in the expanded Xorg callchain). Note also that we have both kernel and userspace entries in the above snapshot. We can also tell perf to focus on userspace but providing a -modifier, in this case 'u', to the 'cycles' hardware counter when we +modifier, in this case ``u``, to the ``cycles`` hardware counter when we record a profile:: root@crownbay:~# perf record -g -a -e cycles:u @@ -912,24 +911,24 @@ record a profile:: .. image:: figures/perf-report-cycles-u.png :align: center -Notice in the screenshot above, we see only userspace entries ([.]) +Notice in the screenshot above, we see only userspace entries (``[.]``) -Finally, we can press 'enter' on a leaf node and select the 'Zoom into -DSO' menu item to show only entries associated with a specific DSO. In -the screenshot below, we've zoomed into the 'libc' DSO which shows all -the entries associated with the libc-xxx.so DSO. +Finally, we can press ``Enter`` on a leaf node and select the ``Zoom into +DSO`` menu item to show only entries associated with a specific DSO. In +the screenshot below, we've zoomed into the ``libc`` DSO which shows all +the entries associated with the ``libc-xxx.so`` DSO. .. image:: figures/perf-systemwide-libc.png :align: center -We can also use the system-wide -a switch to do system-wide tracing. +We can also use the system-wide ``-a`` switch to do system-wide tracing. Here we'll trace a couple of scheduler events:: root@crownbay:~# perf record -a -e sched:sched_switch -e sched:sched_wakeup ^C[ perf record: Woken up 38 times to write data ] [ perf record: Captured and wrote 9.780 MB perf.data (~427299 samples) ] -We can look at the raw output using 'perf script' with no arguments:: +We can look at the raw output using ``perf script`` with no arguments:: root@crownbay:~# perf script @@ -948,10 +947,10 @@ Filtering ^^^^^^^^^ Notice that there are a lot of events that don't really have anything to -do with what we're interested in, namely events that schedule 'perf' +do with what we're interested in, namely events that schedule ``perf`` itself in and out or that wake perf up. We can get rid of those by using -the '--filter' option --- for each event we specify using -e, we can add a ---filter after that to filter out trace events that contain fields with +the ``--filter`` option --- for each event we specify using ``-e``, we can add a +``--filter`` after that to filter out trace events that contain fields with specific values:: root@crownbay:~# perf record -a -e sched:sched_switch --filter 'next_comm != perf && prev_comm != perf' -e sched:sched_wakeup --filter 'comm != perf' @@ -977,16 +976,16 @@ specific values:: kworker/0:3 1209 [000] 7932.326214: sched_switch: prev_comm=kworker/0:3 prev_pid=1209 prev_prio=120 prev_state=S ==> next_comm=swapper/0 next_pid=0 next_prio=120 In this case, we've filtered out all events that have -'perf' in their 'comm' or 'comm_prev' or 'comm_next' fields. Notice that +``perf`` in their ``comm``, ``comm_prev`` or ``comm_next`` fields. Notice that there are still events recorded for perf, but notice that those events -don't have values of 'perf' for the filtered fields. To completely +don't have values of ``perf`` for the filtered fields. To completely filter out anything from perf will require a bit more work, but for the purpose of demonstrating how to use filters, it's close enough. .. admonition:: Tying it Together These are exactly the same set of event filters defined by the trace - event subsystem. See the ftrace/tracecmd/kernelshark section for more + event subsystem. See the ftrace / tracecmd / kernelshark section for more discussion about these event filters. .. admonition:: Tying it Together @@ -1014,9 +1013,9 @@ Using Dynamic Tracepoints ~~~~~~~~~~~~~~~~~~~~~~~~~ perf isn't restricted to the fixed set of static tracepoints listed by -'perf list'. Users can also add their own 'dynamic' tracepoints anywhere +``perf list``. Users can also add their own "dynamic" tracepoints anywhere in the kernel. For instance, suppose we want to define our own -tracepoint on do_fork(). We can do that using the 'perf probe' perf +tracepoint on ``do_fork()``. We can do that using the ``perf probe`` perf subcommand:: root@crownbay:~# perf probe do_fork @@ -1028,8 +1027,8 @@ subcommand:: perf record -e probe:do_fork -aR sleep 1 Adding a new tracepoint via -'perf probe' results in an event with all the expected files and format -in /sys/kernel/debug/tracing/events, just the same as for static +``perf probe`` results in an event with all the expected files and format +in ``/sys/kernel/debug/tracing/events``, just the same as for static tracepoints (as discussed in more detail in the trace events subsystem section:: @@ -1062,7 +1061,7 @@ existence:: probe:do_fork (on do_fork) probe:schedule (on schedule) -Let's record system-wide ('sleep 30' is a +Let's record system-wide (``sleep 30`` is a trick for recording system-wide but basically do nothing and then wake up after 30 seconds):: @@ -1070,7 +1069,7 @@ up after 30 seconds):: [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.087 MB perf.data (~3812 samples) ] -Using 'perf script' we can see each do_fork event that fired:: +Using ``perf script`` we can see each do_fork event that fired:: root@crownbay:~# perf script @@ -1111,7 +1110,7 @@ Using 'perf script' we can see each do_fork event that fired:: matchbox-deskto 1311 [001] 34237.114106: do_fork: (c1028460) gaku 1312 [000] 34237.202388: do_fork: (c1028460) -And using 'perf report' on the same file, we can see the +And using ``perf report`` on the same file, we can see the callgraphs from starting a few programs during those 30 seconds: .. image:: figures/perf-probe-do_fork-profile.png @@ -1155,7 +1154,7 @@ section can be found here: - The top-level `perf(1) manpage `__. Normally, you should be able to invoke the man pages via perf itself -e.g. 'perf help' or 'perf help record'. +e.g. ``perf help`` or ``perf help record``. To have the perf manpages installed on your target, modify your configuration as follows:: @@ -1164,7 +1163,7 @@ configuration as follows:: DISTRO_FEATURES:append = " api-documentation" The man pages in text form, along with some other files, such as a set -of examples, can also be found in the 'perf' directory of the kernel tree:: +of examples, can also be found in the ``perf`` directory of the kernel tree:: tools/perf/Documentation @@ -1175,7 +1174,7 @@ Tutorial `__ ftrace ====== -'ftrace' literally refers to the 'ftrace function tracer' but in reality +"ftrace" literally refers to the "ftrace function tracer" but in reality this encompasses a number of related tracers along with the infrastructure that they all make use of. @@ -1189,16 +1188,16 @@ ftrace, trace-cmd, and kernelshark run on the target system, and are ready to go out-of-the-box --- no additional setup is necessary. For the rest of this section we assume you've ssh'ed to the host and will be running ftrace on the target. kernelshark is a GUI application and if -you use the '-X' option to ssh you can have the kernelshark GUI run on +you use the ``-X`` option to ssh you can have the kernelshark GUI run on the target but display remotely on the host if you want. Basic ftrace usage ------------------ -'ftrace' essentially refers to everything included in the /tracing +"ftrace" essentially refers to everything included in the ``/tracing`` directory of the mounted debugfs filesystem (Yocto follows the standard -convention and mounts it at /sys/kernel/debug). Here's a listing of all -the files found in /sys/kernel/debug/tracing on a Yocto system:: +convention and mounts it at ``/sys/kernel/debug``). Here's a listing of all +the files found in ``/sys/kernel/debug/tracing`` on a Yocto system:: root@sugarbay:/sys/kernel/debug/tracing# ls README kprobe_events trace @@ -1214,7 +1213,7 @@ the files found in /sys/kernel/debug/tracing on a Yocto system:: free_buffer set_graph_function The files listed above are used for various purposes -- some relate directly to the tracers themselves, others are used to set +--- some relate directly to the tracers themselves, others are used to set tracing options, and yet others actually contain the tracing output when a tracer is in effect. Some of the functions can be guessed from their names, others need explanation; in any case, we'll cover some of the @@ -1223,30 +1222,30 @@ the ftrace documentation. We'll start by looking at some of the available built-in tracers. -cat'ing the 'available_tracers' file lists the set of available tracers:: +cat'ing the ``available_tracers`` file lists the set of available tracers:: root@sugarbay:/sys/kernel/debug/tracing# cat available_tracers blk function_graph function nop -The 'current_tracer' file contains the tracer currently in effect:: +The ``current_tracer`` file contains the tracer currently in effect:: root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer nop The above listing of current_tracer shows that the -'nop' tracer is in effect, which is just another way of saying that +``nop`` tracer is in effect, which is just another way of saying that there's actually no tracer currently in effect. -echo'ing one of the available_tracers into current_tracer makes the +echo'ing one of the available_tracers into ``current_tracer`` makes the specified tracer the current tracer:: root@sugarbay:/sys/kernel/debug/tracing# echo function > current_tracer root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer function -The above sets the current tracer to be the 'function tracer'. This tracer +The above sets the current tracer to be the ``function`` tracer. This tracer traces every function call in the kernel and makes it available as the -contents of the 'trace' file. Reading the 'trace' file lists the +contents of the ``trace`` file. Reading the ``trace`` file lists the currently buffered function calls that have been traced by the function tracer:: @@ -1303,11 +1302,11 @@ great way to learn about how the kernel code works in a dynamic sense. .. admonition:: Tying it Together The ftrace function tracer is also available from within perf, as the - ftrace:function tracepoint. + ``ftrace:function`` tracepoint. It is a little more difficult to follow the call chains than it needs to be --- luckily there's a variant of the function tracer that displays the -callchains explicitly, called the 'function_graph' tracer:: +callchains explicitly, called the ``function_graph`` tracer:: root@sugarbay:/sys/kernel/debug/tracing# echo function_graph > current_tracer root@sugarbay:/sys/kernel/debug/tracing# cat trace | less @@ -1422,7 +1421,7 @@ callchains explicitly, called the 'function_graph' tracer:: 3) + 13.784 us | } 3) | sys_ioctl() { -As you can see, the function_graph display is much easier +As you can see, the ``function_graph`` display is much easier to follow. Also note that in addition to the function calls and associated braces, other events such as scheduler events are displayed in context. In fact, you can freely include any tracepoint available in @@ -1440,9 +1439,9 @@ The 'trace events' Subsystem ---------------------------- One especially important directory contained within the -/sys/kernel/debug/tracing directory is the 'events' subdirectory, which +``/sys/kernel/debug/tracing`` directory is the ``events`` subdirectory, which contains representations of every tracepoint in the system. Listing out -the contents of the 'events' subdirectory, we see mainly another set of +the contents of the ``events`` subdirectory, we see mainly another set of subdirectories:: root@sugarbay:/sys/kernel/debug/tracing# cd events @@ -1490,9 +1489,9 @@ subdirectories:: drwxr-xr-x 26 root root 0 Nov 14 23:19 writeback Each one of these subdirectories -corresponds to a 'subsystem' and contains yet again more subdirectories, +corresponds to a "subsystem" and contains yet again more subdirectories, each one of those finally corresponding to a tracepoint. For example, -here are the contents of the 'kmem' subsystem:: +here are the contents of the ``kmem`` subsystem:: root@sugarbay:/sys/kernel/debug/tracing/events# cd kmem root@sugarbay:/sys/kernel/debug/tracing/events/kmem# ls -al @@ -1514,7 +1513,7 @@ here are the contents of the 'kmem' subsystem:: drwxr-xr-x 2 root root 0 Nov 14 23:19 mm_page_pcpu_drain Let's see what's inside the subdirectory for a -specific tracepoint, in this case the one for kmalloc:: +specific tracepoint, in this case the one for ``kmalloc``:: root@sugarbay:/sys/kernel/debug/tracing/events/kmem# cd kmalloc root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# ls -al @@ -1525,12 +1524,12 @@ specific tracepoint, in this case the one for kmalloc:: -r--r--r-- 1 root root 0 Nov 14 23:19 format -r--r--r-- 1 root root 0 Nov 14 23:19 id -The 'format' file for the +The ``format`` file for the tracepoint describes the event in memory, which is used by the various tracing tools that now make use of these tracepoint to parse the event -and make sense of it, along with a 'print fmt' field that allows tools +and make sense of it, along with a ``print fmt`` field that allows tools like ftrace to display the event as text. Here's what the format of the -kmalloc event looks like:: +``kmalloc`` event looks like:: root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# cat format name: kmalloc @@ -1565,11 +1564,11 @@ kmalloc event looks like:: long)(( gfp_t)0x08u), "GFP_MOVABLE"}, {(unsigned long)(( gfp_t)0), "GFP_NOTRACK"}, {(unsigned long)(( gfp_t)0x400000u), "GFP_NO_KSWAPD"}, {(unsigned long)(( gfp_t)0x800000u), "GFP_OTHER_NODE"} ) : "GFP_NOWAIT" -The 'enable' file +The ``enable`` file in the tracepoint directory is what allows the user (or tools such as -trace-cmd) to actually turn the tracepoint on and off. When enabled, the -corresponding tracepoint will start appearing in the ftrace 'trace' file -described previously. For example, this turns on the kmalloc tracepoint:: +``trace-cmd``) to actually turn the tracepoint on and off. When enabled, the +corresponding tracepoint will start appearing in the ftrace ``trace`` file +described previously. For example, this turns on the ``kmalloc`` tracepoint:: root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 1 > enable @@ -1581,7 +1580,7 @@ events in the output buffer:: root@sugarbay:/sys/kernel/debug/tracing# echo nop > current_tracer root@sugarbay:/sys/kernel/debug/tracing# echo 1 > tracing_on -Now, if we look at the 'trace' file, we see nothing +Now, if we look at the ``trace`` file, we see nothing but the kmalloc events we just turned on:: root@sugarbay:/sys/kernel/debug/tracing# cat trace | less @@ -1628,17 +1627,17 @@ but the kmalloc events we just turned on:: -0 [000] ..s3 18156.400660: kmalloc: call_site=ffffffff81619b36 ptr=ffff88006d554800 bytes_req=512 bytes_alloc=512 gfp_flags=GFP_ATOMIC matchbox-termin-1361 [001] ...1 18156.552800: kmalloc: call_site=ffffffff81614050 ptr=ffff88006db34800 bytes_req=576 bytes_alloc=1024 gfp_flags=GFP_KERNEL|GFP_REPEAT -To again disable the kmalloc event, we need to send 0 to the enable file:: +To again disable the ``kmalloc`` event, we need to send ``0`` to the ``enable`` file:: root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# echo 0 > enable You can enable any number of events or complete subsystems (by -using the 'enable' file in the subsystem directory) and get an +using the ``enable`` file in the subsystem directory) and get an arbitrarily fine-grained idea of what's going on in the system by enabling as many of the appropriate tracepoints as applicable. A number of the tools described in this HOWTO do just that, including -trace-cmd and kernelshark in the next section. +``trace-cmd`` and kernelshark in the next section. .. admonition:: Tying it Together @@ -1646,27 +1645,27 @@ trace-cmd and kernelshark in the next section. ftrace, but by many of the other tools covered in this document and they form a central point of integration for the various tracers available in Linux. They form a central part of the instrumentation - for the following tools: perf, lttng, ftrace, blktrace and SystemTap + for the following tools: perf, LTTng, ftrace, blktrace and SystemTap .. admonition:: Tying it Together Eventually all the special-purpose tracers currently available in - /sys/kernel/debug/tracing will be removed and replaced with - equivalent tracers based on the 'trace events' subsystem. + ``/sys/kernel/debug/tracing`` will be removed and replaced with + equivalent tracers based on the "trace events" subsystem. -trace-cmd/kernelshark ---------------------- +trace-cmd / kernelshark +----------------------- -trace-cmd is essentially an extensive command-line 'wrapper' interface +trace-cmd is essentially an extensive command-line "wrapper" interface that hides the details of all the individual files in -/sys/kernel/debug/tracing, allowing users to specify specific particular -events within the /sys/kernel/debug/tracing/events/ subdirectory and to +``/sys/kernel/debug/tracing``, allowing users to specify specific particular +events within the ``/sys/kernel/debug/tracing/events/`` subdirectory and to collect traces and avoid having to deal with those details directly. As yet another layer on top of that, kernelshark provides a GUI that allows users to start and stop traces and specify sets of events using an intuitive interface, and view the output as both trace events and as -a per-CPU graphical display. It directly uses 'trace-cmd' as the +a per-CPU graphical display. It directly uses trace-cmd as the plumbing that accomplishes all that underneath the covers (and actually displays the trace-cmd command it uses, as we'll see). @@ -1674,7 +1673,7 @@ To start a trace using kernelshark, first start kernelshark:: root@sugarbay:~# kernelshark -Then bring up the 'Capture' dialog by +Then bring up the ``Capture`` dialog by choosing from the kernelshark menu:: Capture | Record @@ -1691,10 +1690,10 @@ gets them for kernelshark. In the above screenshot, we've decided to explore the graphics subsystem a bit and so have chosen to trace all the tracepoints contained within -the 'i915' and 'drm' subsystems. +the ``i915`` and ``drm`` subsystems. -After doing that, we can start and stop the trace using the 'Run' and -'Stop' button on the lower right corner of the dialog (the same button +After doing that, we can start and stop the trace using the ``Run`` and +``Stop`` button on the lower right corner of the dialog (the same button will turn into the 'Stop' button after the trace has started): .. image:: figures/kernelshark-output-display.png @@ -1704,15 +1703,15 @@ Notice that the right-hand pane shows the exact trace-cmd command-line that's used to run the trace, along with the results of the trace-cmd run. -Once the 'Stop' button is pressed, the graphical view magically fills up +Once the ``Stop`` button is pressed, the graphical view magically fills up with a colorful per-cpu display of the trace data, along with the detailed event listing below that: .. image:: figures/kernelshark-i915-display.png :align: center -Here's another example, this time a display resulting from tracing 'all -events': +Here's another example, this time a display resulting from tracing ``all +events``: .. image:: figures/kernelshark-all.png :align: center @@ -1760,12 +1759,12 @@ systemtap SystemTap is a system-wide script-based tracing and profiling tool. SystemTap scripts are C-like programs that are executed in the kernel to -gather/print/aggregate data extracted from the context they end up being +gather / print / aggregate data extracted from the context they end up being invoked under. For example, this probe from the `SystemTap tutorial `__ simply prints a -line every time any process on the system open()s a file. For each line, +line every time any process on the system runs ``open()`` on a file. For each line, it prints the executable name of the program that opened the file, along with its PID, and the name of the file it opened (or tried to open), which it extracts from the open syscall's argstr. @@ -1790,19 +1789,19 @@ file containing the above text is trace_open.stp:: # stap trace_open.stp What systemtap does under the covers to run this probe is 1) parse and -convert the probe to an equivalent 'C' form, 2) compile the 'C' form +convert the probe to an equivalent "C" form, 2) compile the "C" form into a kernel module, 3) insert the module into the kernel, which arms it, and 4) collect the data generated by the probe and display it to the user. -In order to accomplish steps 1 and 2, the 'stap' program needs access to +In order to accomplish steps 1 and 2, the ``stap`` program needs access to the kernel build system that produced the kernel that the probed system -is running. In the case of a typical embedded system (the 'target'), the +is running. In the case of a typical embedded system (the "target"), the kernel build system unfortunately isn't typically part of the image -running on the target. It is normally available on the 'host' system +running on the target. It is normally available on the "host" system that produced the target image however; in such cases, steps 1 and 2 are executed on the host system, and steps 3 and 4 are executed on the -target system, using only the systemtap 'runtime'. +target system, using only the systemtap "runtime". The systemtap support in Yocto assumes that only steps 3 and 4 are run on the target; it is possible to do everything on the target, but this @@ -1819,12 +1818,12 @@ systemtap Setup --------------- Those are a lot of steps and a lot of details, but fortunately Yocto -includes a script called 'crosstap' that will take care of those +includes a script called ``crosstap`` that will take care of those details, allowing you to simply execute a systemtap script on the remote target, with arguments if necessary. In order to do this from a remote host, however, you need to have access -to the build for the image you booted. The 'crosstap' script provides +to the build for the image you booted. The ``crosstap`` script provides details on how to do this if you run the script on the host without having done a build:: @@ -1833,29 +1832,35 @@ having done a build:: Error: No target kernel build found. Did you forget to create a local build of your image? - 'crosstap' requires a local sdk build of the target system - (or a build that includes 'tools-profile') in order to build - kernel modules that can probe the target system. - - Practically speaking, that means you need to do the following: - - If you're running a pre-built image, download the release - and/or BSP tarballs used to build the image. - - If you're working from git sources, just clone the metadata - and BSP layers needed to build the image you'll be booting. - - Make sure you're properly set up to build a new image (see - the BSP README and/or the widely available basic documentation - that discusses how to build images). - - Build an -sdk version of the image e.g.: - $ bitbake core-image-sato-sdk - OR - - Build a non-sdk image but include the profiling tools: - [ edit local.conf and add 'tools-profile' to the end of - the EXTRA_IMAGE_FEATURES variable ] - $ bitbake core-image-sato +'crosstap' requires a local sdk build of the target system +(or a build that includes 'tools-profile') in order to build +kernel modules that can probe the target system. + +Practically speaking, that means you need to do the following: + +- If you're running a pre-built image, download the release + and/or BSP tarballs used to build the image. + +- If you're working from git sources, just clone the metadata + and BSP layers needed to build the image you'll be booting. + +- Make sure you're properly set up to build a new image (see + the BSP README and/or the widely available basic documentation + that discusses how to build images). + +- Build an -sdk version of the image e.g.:: + + $ bitbake core-image-sato-sdk + +- Or build a non-sdk image but include the profiling tools + (edit ``local.conf`` and add ``tools-profile`` to the end of + :term:``EXTRA_IMAGE_FEATURES`` variable):: + + $ bitbake core-image-sato Once you've build the image on the host system, you're ready to - boot it (or the equivalent pre-built image) and use 'crosstap' - to probe it (you need to source the environment as usual first): + boot it (or the equivalent pre-built image) and use ``crosstap`` + to probe it (you need to source the environment as usual first):: $ source oe-init-build-env $ cd ~/my/systemtap/scripts @@ -1863,23 +1868,22 @@ having done a build:: .. note:: - SystemTap, which uses 'crosstap', assumes you can establish an ssh + SystemTap, which uses ``crosstap``, assumes you can establish an ssh connection to the remote target. Please refer to the crosstap wiki - page for details on verifying ssh connections at - . Also, the ability to ssh into the target system is not enabled by - default in \*-minimal images. + page for details on verifying ssh connections. Also, the ability to ssh + into the target system is not enabled by default in ``*-minimal`` images. So essentially what you need to -do is build an SDK image or image with 'tools-profile' as detailed in +do is build an SDK image or image with ``tools-profile`` as detailed in the ":ref:`profile-manual/intro:General Setup`" section of this manual, and boot the resulting target image. .. note:: - If you have a build directory containing multiple machines, you need - to have the MACHINE you're connecting to selected in local.conf, and - the kernel in that machine's build directory must match the kernel on - the booted system exactly, or you'll get the above 'crosstap' message + If you have a :term:`Build Directory` containing multiple machines, you need + to have the :term:`MACHINE` you're connecting to selected in ``local.conf``, and + the kernel in that machine's :term:`Build Directory` must match the kernel on + the booted system exactly, or you'll get the above ``crosstap`` message when you try to invoke a script. Running a Script on a Target @@ -1903,8 +1907,8 @@ the target:: You can also run generated QEMU images with a command like 'runqemu qemux86-64' -Once you've done that, you can cd to whatever -directory contains your scripts and use 'crosstap' to run the script:: +Once you've done that, you can ``cd`` to whatever +directory contains your scripts and use ``crosstap`` to run the script:: $ cd /path/to/my/systemap/script $ crosstap root@192.168.7.2 trace_open.stp @@ -1918,9 +1922,8 @@ Try ssh'ing to the target and see what happens:: $ ssh root@192.168.7.2 -A lot of the time, connection -problems are due specifying a wrong IP address or having a 'host key -verification error'. +Connection problems are often due specifying a wrong IP address or having a ``host key +verification error``. If everything worked as planned, you should see something like this (enter the password when prompted, or press enter if it's set up to use @@ -1958,16 +1961,16 @@ outlined in the ":ref:`profile-manual/intro:General Setup`" section. Sysprof is a GUI-based application that runs on the target system. For the rest of this document we assume you've ssh'ed to the host and will -be running Sysprof on the target (you can use the '-X' option to ssh and +be running Sysprof on the target (you can use the ``-X`` option to ssh and have the Sysprof GUI run on the target but display remotely on the host if you want). Basic Sysprof Usage ------------------- -To start profiling the system, you simply press the 'Start' button. To +To start profiling the system, you simply press the ``Start`` button. To stop profiling and to start viewing the profile data in one easy step, -press the 'Profile' button. +press the ``Profile`` button. Once you've pressed the profile button, the three panes will fill up with profiling data: @@ -1982,7 +1985,7 @@ perf's default callee-oriented callchain display. In the screenshot above, we're focusing on ``__copy_to_user_ll()`` and looking up the callchain we can see that one of the callers of -``__copy_to_user_ll`` is sys_read() and the complete callpath between them. +``__copy_to_user_ll`` is ``sys_read()`` and the complete callpath between them. Notice that this is essentially a portion of the same information we saw in the perf display shown in the perf section of this page. @@ -1990,7 +1993,7 @@ in the perf display shown in the perf section of this page. :align: center Similarly, the above is a snapshot of the Sysprof display of a -copy-from-user callchain. +``copy-from-user`` callchain. Finally, looking at the third Sysprof pane in the lower left, we can see a list of all the callers of a particular function selected in the top @@ -2005,10 +2008,10 @@ to the selected function, and so on. .. admonition:: Tying it Together - If you like sysprof's 'caller-oriented' display, you may be able to - approximate it in other tools as well. For example, 'perf report' has - the -g (--call-graph) option that you can experiment with; one of the - options is 'caller' for an inverted caller-based callgraph display. + If you like sysprof's ``caller-oriented`` display, you may be able to + approximate it in other tools as well. For example, ``perf report`` has + the ``-g`` (``--call-graph``) option that you can experiment with; one of the + options is ``caller`` for an inverted caller-based callgraph display. Sysprof Documentation --------------------- @@ -2032,7 +2035,7 @@ Collecting and Viewing Traces ----------------------------- Once you've applied the above commits and built and booted your image -(you need to build the core-image-sato-sdk image or use one of the other +(you need to build the ``core-image-sato-sdk`` image or use one of the other methods described in the ":ref:`profile-manual/intro:General Setup`" section), you're ready to start tracing. @@ -2117,14 +2120,14 @@ You can now view the trace in text form on the target:: You can now safely destroy the trace session (note that this doesn't delete the trace --- it's still there in -~/lttng-traces):: +``~/lttng-traces``):: root@crownbay:~# lttng destroy Session auto-20121015-232120 destroyed at /home/root Note that the trace is saved in a directory of the same name as returned by -'lttng create', under the ~/lttng-traces directory (note that you can change this by -supplying your own name to 'lttng create'):: +``lttng create``, under the ``~/lttng-traces`` directory (note that you can change this by +supplying your own name to ``lttng create``):: root@crownbay:~# ls -al ~/lttng-traces drwxrwx--- 3 root root 1024 Oct 15 23:21 . @@ -2135,12 +2138,12 @@ Collecting and viewing a userspace trace on the target (inside a shell) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For LTTng userspace tracing, you need to have a properly instrumented -userspace program. For this example, we'll use the 'hello' test program -generated by the lttng-ust build. +userspace program. For this example, we'll use the ``hello`` test program +generated by the ``lttng-ust`` build. -The 'hello' test program isn't installed on the root filesystem by the lttng-ust -build, so we need to copy it over manually. First cd into the build -directory that contains the hello executable:: +The ``hello`` test program isn't installed on the root filesystem by the ``lttng-ust`` +build, so we need to copy it over manually. First ``cd`` into the build +directory that contains the ``hello`` executable:: $ cd build/tmp/work/core2_32-poky-linux/lttng-ust/2.0.5-r0/git/tests/hello/.libs @@ -2148,7 +2151,7 @@ Copy that over to the target machine:: $ scp hello root@192.168.1.20: -You now have the instrumented lttng 'hello world' test program on the +You now have the instrumented LTTng "hello world" test program on the target, ready to test. First, from the host, ssh to the target:: @@ -2176,7 +2179,7 @@ Start the trace:: root@crownbay:~# lttng start Tracing started for session auto-20190303-021943 -Run the instrumented hello world program:: +Run the instrumented "hello world" program:: root@crownbay:~# ./hello Hello, World! @@ -2200,7 +2203,7 @@ You can now view the trace in text form on the target:: . You can now safely destroy the trace session (note that this doesn't delete the -trace --- it's still there in ~/lttng-traces):: +trace --- it's still there in ``~/lttng-traces``):: root@crownbay:~# lttng destroy Session auto-20190303-021943 destroyed at /home/root @@ -2244,12 +2247,12 @@ will be running blkrace on the target. Basic blktrace Usage -------------------- -To record a trace, simply run the 'blktrace' command, giving it the name +To record a trace, simply run the ``blktrace`` command, giving it the name of the block device you want to trace activity on:: root@crownbay:~# blktrace /dev/sdc -In another shell, execute a workload you want to trace. :: +In another shell, execute a workload you want to trace:: root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2; sync Connecting to downloads.yoctoproject.org (140.211.169.59:80) @@ -2258,7 +2261,7 @@ In another shell, execute a workload you want to trace. :: Press Ctrl-C in the blktrace shell to stop the trace. It will display how many events were logged, along with the per-cpu file sizes (blktrace records traces in per-cpu kernel buffers and simply -dumps them to userspace for blkparse to merge and sort later). :: +dumps them to userspace for blkparse to merge and sort later):: ^C=== sdc === CPU 0: 7082 events, 332 KiB data @@ -2274,7 +2277,7 @@ with the device name as the first part of the filename:: -rw-r--r-- 1 root root 339938 Oct 27 22:40 sdc.blktrace.0 -rw-r--r-- 1 root root 75753 Oct 27 22:40 sdc.blktrace.1 -To view the trace events, simply invoke 'blkparse' in the directory +To view the trace events, simply invoke ``blkparse`` in the directory containing the trace files, giving it the device name that forms the first part of the filenames:: @@ -2373,7 +2376,7 @@ Live Mode ~~~~~~~~~ blktrace and blkparse are designed from the ground up to be able to -operate together in a 'pipe mode' where the stdout of blktrace can be +operate together in a "pipe mode" where the stdout of blktrace can be fed directly into the stdin of blkparse:: root@crownbay:~# blktrace /dev/sdc -o - | blkparse -i - @@ -2419,13 +2422,13 @@ On the host system, you should see this:: server: connection from 192.168.1.43 -In another shell, execute a workload you want to trace. :: +In another shell, execute a workload you want to trace:: root@crownbay:/media/sdc# rm linux-2.6.19.2.tar.bz2; wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2; sync Connecting to downloads.yoctoproject.org (140.211.169.59:80) linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA -When it's done, do a Ctrl-C on the target system to stop the +When it's done, do a ``Ctrl-C`` on the target system to stop the trace:: ^C=== sdc === @@ -2450,7 +2453,7 @@ save the target output inside a hostname-timestamp directory:: drwxr-sr-x 4 root root 1024 Oct 26 18:24 .. drwxr-xr-x 2 root root 1024 Oct 28 02:40 192.168.1.43-2012-10-28-02:40:56 -cd into that directory to see the output files:: +``cd`` into that directory to see the output files:: $ ls -l -rw-r--r-- 1 root root 369193 Oct 28 02:44 sdc.blktrace.0 @@ -2517,14 +2520,14 @@ It's also possible to trace block I/O using only can be useful for casual tracing if you don't want to bother dealing with the userspace tools. -To enable tracing for a given device, use /sys/block/xxx/trace/enable, -where xxx is the device name. This for example enables tracing for -/dev/sdc:: +To enable tracing for a given device, use ``/sys/block/xxx/trace/enable``, +where ``xxx`` is the device name. This for example enables tracing for +``/dev/sdc``:: root@crownbay:/sys/kernel/debug/tracing# echo 1 > /sys/block/sdc/trace/enable Once you've selected the device(s) you want -to trace, selecting the 'blk' tracer will turn the blk tracer on:: +to trace, selecting the ``blk`` tracer will turn the blk tracer on:: root@crownbay:/sys/kernel/debug/tracing# cat available_tracers blk function_graph function nop @@ -2535,7 +2538,7 @@ Execute the workload you're interested in:: root@crownbay:/sys/kernel/debug/tracing# cat /media/sdc/testfile.txt -And look at the output (note here that we're using 'trace_pipe' instead of +And look at the output (note here that we're using ``trace_pipe`` instead of trace to capture this trace --- this allows us to wait around on the pipe for data to appear):: @@ -2572,8 +2575,8 @@ section can be found here: - https://linux.die.net/man/8/btrace -The above manpages, along with manpages for the other blktrace utilities -(btt, blkiomon, etc) can be found in the /doc directory of the blktrace +The above manpages, along with manuals for the other blktrace utilities +(btt, blkiomon, etc) can be found in the ``/doc`` directory of the blktrace tools git repo:: $ git clone git://git.kernel.dk/blktrace.git From patchwork Mon Mar 25 13:27:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 020A9CD11DD for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by mx.groups.io with SMTP id smtpd.web10.51971.1711373266687661269 for ; Mon, 25 Mar 2024 06:27:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=IwhRMUP1; spf=pass (domain: bootlin.com, ip: 217.70.183.200, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 3B1F420011; Mon, 25 Mar 2024 13:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8ey1jskUojM+nsOnuxh0WZlX46YwqoxAcrJLSEGVfE=; b=IwhRMUP1bNKFwZ8Deh62egvkavlJfRfM8FQC/ExSyWXZ3P7pNQR6yBKt5uTSDiFji3vhFX pZhInwliOVYKiJIefYYHvZwquhu9rAyzNYnW7ftids20dqYjd3mbuX+1UmFht2vinxipLX bERUGJE10xY8lAHo2IE4cY2S3XibBEYd+9sVkuq4qDTOBKTEqg49HRn+aOeS6SYUgBZq5K CzElDbARcssp/3CJjF5NV4E0W7yer6qcqfqTOGElZM+/qoBeKNt3XPm6ZePy07xuVzL2qg jwlQmfYsoO5gIhH0n+eRWBerT7bYPWWNQpNfDXAtM5oAEVyKBJ+/yiK7n/CXHQ== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker Subject: [kirkstone][PATCH 3/9] manuals: use "manual page(s)" Date: Mon, 25 Mar 2024 14:27:31 +0100 Message-Id: <20240325132737.3018073-4-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5000 From: Michael Opdenacker Instead of "manpage(s)" or "man page(s)". To address one of the errors reported by "make stylecheck" Signed-off-by: Michael Opdenacker --- documentation/profile-manual/usage.rst | 28 +++++++++++++------------- documentation/ref-manual/classes.rst | 2 +- documentation/ref-manual/variables.rst | 2 +- 3 files changed, 16 insertions(+), 16 deletions(-) diff --git a/documentation/profile-manual/usage.rst b/documentation/profile-manual/usage.rst index 912e4c9602..41a2d9ba3f 100644 --- a/documentation/profile-manual/usage.rst +++ b/documentation/profile-manual/usage.rst @@ -38,7 +38,7 @@ other tools when it seems useful to do so. The coverage below details some of the most common ways you'll likely want to apply the tool; full documentation can be found either within -the tool itself or in the man pages at +the tool itself or in the manual pages at `perf(1) `__. Perf Setup @@ -1132,37 +1132,37 @@ callgraphs from starting a few programs during those 30 seconds: Perf Documentation ------------------ -Online versions of the man pages for the commands discussed in this +Online versions of the manual pages for the commands discussed in this section can be found here: -- The `'perf stat' manpage `__. +- The `'perf stat' manual page `__. - The `'perf record' - manpage `__. + manual page `__. - The `'perf report' - manpage `__. + manual page `__. -- The `'perf probe' manpage `__. +- The `'perf probe' manual page `__. - The `'perf script' - manpage `__. + manual page `__. - Documentation on using the `'perf script' python binding `__. -- The top-level `perf(1) manpage `__. +- The top-level `perf(1) manual page `__. -Normally, you should be able to invoke the man pages via perf itself +Normally, you should be able to invoke the manual pages via perf itself e.g. ``perf help`` or ``perf help record``. -To have the perf manpages installed on your target, modify your +To have the perf manual pages installed on your target, modify your configuration as follows:: IMAGE_INSTALL:append = " perf perf-doc" DISTRO_FEATURES:append = " api-documentation" -The man pages in text form, along with some other files, such as a set +The manual pages in text form, along with some other files, such as a set of examples, can also be found in the ``perf`` directory of the kernel tree:: tools/perf/Documentation @@ -2369,7 +2369,7 @@ first part of the filenames:: The report shows each event that was found in the blktrace data, along with a summary of the overall block I/O traffic during the run. You can look at the -`blkparse `__ manpage to learn the +`blkparse `__ manual page to learn the meaning of each field displayed in the trace listing. Live Mode @@ -2566,7 +2566,7 @@ And this turns off tracing for the specified device:: blktrace Documentation ---------------------- -Online versions of the man pages for the commands discussed in this +Online versions of the manual pages for the commands discussed in this section can be found here: - https://linux.die.net/man/8/blktrace @@ -2575,7 +2575,7 @@ section can be found here: - https://linux.die.net/man/8/btrace -The above manpages, along with manuals for the other blktrace utilities +The above manual pages, along with manuals for the other blktrace utilities (btt, blkiomon, etc) can be found in the ``/doc`` directory of the blktrace tools git repo:: diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index 05473edcca..e7bf5e9a6e 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -346,7 +346,7 @@ and BusyBox. It could have been called "kconfig" too. ``compress_doc`` ================ -Enables compression for man pages and info pages. This class is intended +Enables compression for manual and info pages. This class is intended to be inherited globally. The default compression mechanism is gz (gzip) but you can select an alternative mechanism by setting the :term:`DOC_COMPRESS` variable. diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index ccd1963339..aa1d5f393d 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -2107,7 +2107,7 @@ system and gives an overview of their function and contents. :term:`DOC_COMPRESS` When inheriting the :ref:`compress_doc ` class, this variable sets the compression policy used when the - OpenEmbedded build system compresses man pages and info pages. By + OpenEmbedded build system compresses manual and info pages. By default, the compression method used is gz (gzip). Other policies available are xz and bz2. From patchwork Mon Mar 25 13:27:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51A17CD1284 for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by mx.groups.io with SMTP id smtpd.web10.51973.1711373267056792115 for ; Mon, 25 Mar 2024 06:27:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=NqY8O+c/; spf=pass (domain: bootlin.com, ip: 217.70.183.193, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 9908A24000F; Mon, 25 Mar 2024 13:27:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PgfXx36XQu/4AfpvWNfYrBxFrF2/nJKieod6XR+irMU=; b=NqY8O+c/AqPWQUDZYe5ifRPgO6amURfg/QfDmuQPQ/cSvmPRrwFRinraQpWiu/filfHrJ2 bh3QaeJbiEedFSZwrQjVk05NTN+8vKiyDxtsRKLOoPpG0cnq7ccmHZTbxsakdeU3DDb6lb VFn1NlSKnwKtgRG7ZjoTo6bA4my2qmDccyFtkNnkxoU5EPfU1csRurM5OqF32b1fNFbtMZ +J2DbRxCYAkQk3dhBYWrfEb37Ep75WSs/CcVb8GFcudCNJicerAUj4pTBUgTypK1pTjAwi PfY5gD+6qfTd1NMRMQUCKBXIxIp7/2aYfX1Rc+/51puM/aUQjo8FBkHOIxL42g== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker Subject: [kirkstone][PATCH 4/9] profile-manual: usage.rst: fix reference to bug report Date: Mon, 25 Mar 2024 14:27:32 +0100 Message-Id: <20240325132737.3018073-5-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5001 From: Michael Opdenacker Allowing to remove nested parentheses in the text! Signed-off-by: Michael Opdenacker --- documentation/profile-manual/usage.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/documentation/profile-manual/usage.rst b/documentation/profile-manual/usage.rst index 41a2d9ba3f..28dcb969a0 100644 --- a/documentation/profile-manual/usage.rst +++ b/documentation/profile-manual/usage.rst @@ -131,9 +131,9 @@ the summed counts at the end of the run:: 59.836627620 seconds time elapsed -Many times such a simple-minded test doesn't yield much of -interest, but sometimes it does (see Real-world Yocto bug (slow -loop-mounted write speed)). +Such a simple-minded test doesn't always yield much of interest, but sometimes +it does (see the :yocto_bugs:`Slow write speed on live images with denzil +` bug report). Also, note that ``perf stat`` isn't restricted to a fixed set of counters --- basically any event listed in the output of ``perf list`` can be tallied From patchwork Mon Mar 25 13:27:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2927DCD11DF for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by mx.groups.io with SMTP id smtpd.web11.52060.1711373268599563550 for ; Mon, 25 Mar 2024 06:27:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=Sco+8gtt; spf=pass (domain: bootlin.com, ip: 217.70.183.201, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 5D0C41BF215; Mon, 25 Mar 2024 13:27:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BdOa5SVdPXcXnz9BeAG8hIbmqkznWOIaw7r3aLLc4FQ=; b=Sco+8gttJH4HmJ/itWbBUx6caGI0BDmAIY63uk7vXFnBf1YQl7xUxTKwmAHBKoiDiP1Wou BbE2IbZc0jyN63Xvvtx2jdXAaNtvdJknB/IrBCd7Ot+VDD63jnszuU+U1umQiuQLxY+G90 O5MsQeHVC47lT2B/pPkbTj2jIO87N6EGAf5ujgmyQUM7dfUQ0RVDwVTOehKU7fL7+/6tvB ZC5yA0y8md4iSOn4UjrmMEXlbyjucCZwxEwt0c1GkiGx59JoAa1vQvwtx3fPCeRDBNjYkA h7XReuA8w0q2lVPpTZtX/YVqJ24wzPBzyVQAk371+dGeMV4yCvzPGHw63IfUaA== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , Lee Chee Yang Subject: [kirkstone][PATCH 5/9] migration-guides: add release notes for 4.0.17 Date: Mon, 25 Mar 2024 14:27:33 +0100 Message-Id: <20240325132737.3018073-6-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5003 From: Michael Opdenacker From: Lee Chee Yang Signed-off-by: Lee Chee Yang Reviewed-by: Michael Opdenacker --- .../migration-guides/release-4.0.rst | 1 + .../migration-guides/release-notes-4.0.17.rst | 238 ++++++++++++++++++ 2 files changed, 239 insertions(+) create mode 100644 documentation/migration-guides/release-notes-4.0.17.rst diff --git a/documentation/migration-guides/release-4.0.rst b/documentation/migration-guides/release-4.0.rst index dfe5e186e5..685799e268 100644 --- a/documentation/migration-guides/release-4.0.rst +++ b/documentation/migration-guides/release-4.0.rst @@ -23,3 +23,4 @@ Release 4.0 (kirkstone) release-notes-4.0.14 release-notes-4.0.15 release-notes-4.0.16 + release-notes-4.0.17 diff --git a/documentation/migration-guides/release-notes-4.0.17.rst b/documentation/migration-guides/release-notes-4.0.17.rst new file mode 100644 index 0000000000..1dfd10ce20 --- /dev/null +++ b/documentation/migration-guides/release-notes-4.0.17.rst @@ -0,0 +1,238 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +Release notes for Yocto-4.0.17 (Kirkstone) +------------------------------------------ + +Security Fixes in Yocto-4.0.17 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- bind: Fix :cve:`2023-4408`, :cve:`2023-50387`, :cve:`2023-50868`, :cve:`2023-5517` and :cve:`2023-5679` +- binutils: Fix :cve:`2023-39129` and :cve:`2023-39130` +- curl: Fix :cve:`2023-46219` +- curl: Ignore :cve:`2023-42915` +- gcc: Ignore :cve:`2023-4039` +- gdb: Fix :cve:`2023-39129` and :cve:`2023-39130` +- glibc: Ignore :cve:`2023-0687` +- go: Fix :cve:`2023-29406`, :cve:`2023-45285`, :cve:`2023-45287`, :cve:`2023-45289`, :cve:`2023-45290`, :cve:`2024-24784` and :cve:`2024-24785` +- less: Fix :cve:`2022-48624` +- libgit2: Fix :cve:`2024-24575` and :cve:`2024-24577` +- libuv: fix :cve:`2024-24806` +- libxml2: Fix for :cve:`2024-25062` +- linux-yocto/5.15: Fix :cve:`2022-36402`, :cve:`2022-40982`, :cve:`2022-47940`, :cve:`2023-1193`, :cve:`2023-1194`, :cve:`2023-20569`, :cve:`2023-20588`, :cve:`2023-25775`, :cve:`2023-31085`, :cve:`2023-32247`, :cve:`2023-32250`, :cve:`2023-32252`, :cve:`2023-32254`, :cve:`2023-32257`, :cve:`2023-32258`, :cve:`2023-34324`, :cve:`2023-35827`, :cve:`2023-3772`, :cve:`2023-38427`, :cve:`2023-38430`, :cve:`2023-38431`, :cve_mitre:`2023-3867`, :cve:`2023-39189`, :cve:`2023-39192`, :cve:`2023-39193`, :cve:`2023-39194`, :cve:`2023-39198`, :cve:`2023-40283`, :cve:`2023-4128`, :cve:`2023-4206`, :cve:`2023-4207`, :cve:`2023-4208`, :cve:`2023-4244`, :cve:`2023-4273`, :cve:`2023-42752`, :cve:`2023-42753`, :cve:`2023-42754`, :cve:`2023-42755`, :cve:`2023-4563`, :cve:`2023-4569`, :cve:`2023-45871`, :cve:`2023-4623`, :cve:`2023-46343`, :cve:`2023-46813`, :cve:`2023-46838`, :cve:`2023-46862`, :cve:`2023-4881`, :cve:`2023-4921`, :cve:`2023-51042`, :cve:`2023-5158`, :cve:`2023-51779`, :cve_mitre:`2023-52340`, :cve:`2023-52429`, :cve:`2023-52435`, :cve:`2023-52436`, :cve:`2023-52438`, :cve:`2023-52439`, :cve:`2023-52441`, :cve:`2023-52442`, :cve:`2023-52443`, :cve:`2023-52444`, :cve:`2023-52445`, :cve:`2023-52448`, :cve:`2023-52449`, :cve:`2023-52451`, :cve:`2023-52454`, :cve:`2023-52456`, :cve:`2023-52457`, :cve:`2023-52458`, :cve:`2023-52463`, :cve:`2023-52464`, :cve:`2023-5717`, :cve:`2023-6040`, :cve:`2023-6121`, :cve:`2023-6176`, :cve:`2023-6546`, :cve:`2023-6606`, :cve:`2023-6622`, :cve:`2023-6817`, :cve:`2023-6915`, :cve:`2023-6931`, :cve:`2023-6932`, :cve:`2024-0340`, :cve:`2024-0584`, :cve:`2024-0607`, :cve:`2024-0641`, :cve:`2024-0646`, :cve:`2024-1085`, :cve:`2024-1086`, :cve:`2024-1151`, :cve:`2024-22705`, :cve:`2024-23849`, :cve:`2024-23850`, :cve:`2024-23851`, :cve:`2024-24860`, :cve:`2024-26586`, :cve:`2024-26589`, :cve:`2024-26591`, :cve:`2024-26592`, :cve:`2024-26593`, :cve:`2024-26594`, :cve:`2024-26597` and :cve:`2024-26598` +- linux-yocto/5.15: Ignore :cve:`2020-27418`, :cve:`2020-36766`, :cve:`2021-33630`, :cve:`2021-33631`, :cve:`2022-48619`, :cve:`2023-2430`, :cve:`2023-40791`, :cve:`2023-42756`, :cve:`2023-44466`, :cve:`2023-45862`, :cve:`2023-45863`, :cve:`2023-45898`, :cve:`2023-4610`, :cve:`2023-4732`, :cve:`2023-5090`, :cve:`2023-51043`, :cve:`2023-5178`, :cve:`2023-51780`, :cve:`2023-51781`, :cve:`2023-51782`, :cve:`2023-5197`, :cve:`2023-52433`, :cve:`2023-52440`, :cve:`2023-52446`, :cve:`2023-52450`, :cve:`2023-52453`, :cve:`2023-52455`, :cve:`2023-52459`, :cve:`2023-52460`, :cve:`2023-52461`, :cve:`2023-52462`, :cve:`2023-5345`, :cve:`2023-5633`, :cve:`2023-5972`, :cve:`2023-6111`, :cve:`2023-6200`, :cve:`2023-6531`, :cve:`2023-6679`, :cve:`2023-7192`, :cve:`2024-0193`, :cve:`2024-0443`, :cve:`2024-0562`, :cve:`2024-0582`, :cve:`2024-0639`, :cve:`2024-0775`, :cve:`2024-26581`, :cve:`2024-26582`, :cve:`2024-26590`, :cve:`2024-26596` and :cve:`2024-26599` +- linux-yocto/5.10: Fix :cve:`2023-39198`, :cve:`2023-46838`, :cve:`2023-51779`, :cve:`2023-51780`, :cve:`2023-51781`, :cve:`2023-51782`, :cve_mitre:`2023-52340`, :cve:`2023-6040`, :cve:`2023-6121`, :cve:`2023-6606`, :cve:`2023-6817`, :cve:`2023-6915`, :cve:`2023-6931`, :cve:`2023-6932`, :cve:`2024-0584` and :cve:`2024-0646` +- linux-yocto/5.10: Ignore :cve:`2021-33630`, :cve:`2021-33631`, :cve:`2022-1508`, :cve:`2022-36402`, :cve:`2022-48619`, :cve:`2023-2430`, :cve:`2023-4610`, :cve:`2023-46343`, :cve:`2023-51042`, :cve:`2023-51043`, :cve:`2023-5972`, :cve:`2023-6039`, :cve:`2023-6200`, :cve:`2023-6531`, :cve:`2023-6546`, :cve:`2023-6622`, :cve:`2023-6679`, :cve:`2023-7192`, :cve:`2024-0193`, :cve:`2024-0443`, :cve:`2024-0562`, :cve:`2024-0582`, :cve:`2024-0639`, :cve:`2024-0641`, :cve:`2024-0775`, :cve:`2024-1085` and :cve:`2024-22705` +- openssl: Fix :cve:`2024-0727` +- python3-pycryptodome: Fix :cve:`2023-52323` +- qemu: Fix :cve:`2023-42467`, :cve:`2023-6693` and :cve:`2024-24474` +- vim: Fix :cve:`2024-22667` +- xwayland: Fix :cve:`2023-6377` and :cve:`2023-6478` + + +Fixes in Yocto-4.0.17 +~~~~~~~~~~~~~~~~~~~~~ + +- bind: Upgrade to 9.18.24 +- bitbake: bitbake/codeparser.py: address ast module deprecations in py 3.12 +- bitbake: bitbake/lib/bs4/tests/test_tree.py: python 3.12 regex +- bitbake: codeparser: replace deprecated ast.Str and 's' +- bitbake: fetch2: Ensure that git LFS objects are available +- bitbake: tests/fetch: Add real git lfs tests and decorator +- bitbake: tests/fetch: git-lfs restore _find_git_lfs +- bitbake: toaster/toastergui: Bug-fix verify given layer path only if import/add local layer +- build-appliance-image: Update to kirkstone head revision +- cmake: Unset CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES +- contributor-guide: fix lore URL +- curl: don't enable debug builds +- cve_check: cleanup logging +- dbus: Add missing :term:`CVE_PRODUCT` +- dev-manual: sbom: Rephrase spdx creation +- dev-manual: runtime-testing: gen-tapdevs need iptables installed +- dev-manual: packages: clarify shared :term:`PR` service constraint +- dev-manual: packages: need enough free space +- dev-manual: start: remove idle line +- feature-microblaze-versions.inc: python 3.12 regex +- ghostscript: correct :term:`LICENSE` with AGPLv3 +- image-live.bbclass: LIVE_ROOTFS_TYPE support compression +- kernel.bbclass: Set pkg-config variables for building modules +- kernel.bbclass: introduce KERNEL_LOCALVERSION +- kernel: fix localversion in v6.3+ +- kernel: make LOCALVERSION consistent between recipes +- ldconfig-native: Fix to point correctly on the DT_NEEDED entries in an ELF file +- librsvg: Fix do_package_qa error for librsvg +- linux-firmware: upgrade to 20231211 +- linux-yocto/5.10: update to v5.10.210 +- linux-yocto/5.15: update to v5.15.150 +- manuals: add minimum RAM requirements +- manuals: suppress excess use of "following" word +- manuals: update disk space requirements +- manuals: update references to buildtools +- manuals: updates for building on Windows (WSL 2) +- meta/lib/oeqa: python 3.12 regex +- meta/recipes: python 3.12 regex +- migration-guide: add release notes for 4.0.16 +- oeqa/selftest/oelib/buildhistory: git default branch +- oeqa/selftest/recipetool: downgrade meson version to not use pyproject.toml +- oeqa/selftest/recipetool: expect meson.bb +- oeqa/selftest/recipetool: fix for python 3.12 +- oeqa/selftest/runtime_test: only run the virgl tests on qemux86-64 +- oeqa: replace deprecated assertEquals +- openssl: Upgrade to 3.0.13 +- poky.conf: bump version for 4.0.17 +- populate_sdk_ext: use ConfigParser instead of SafeConfigParser +- python3-jinja2: upgrade to 3.1.3 +- recipetool/create_buildsys_python: use importlib instead of imp +- ref-manual: system-requirements: recommend buildtools for not supported distros +- ref-manual: system-requirements: add info on buildtools-make-tarball +- ref-manual: release-process: grammar fix +- ref-manual: system-requirements: fix AlmaLinux variable name +- ref-manual: system-requirements: modify anchor +- ref-manual: system-requirements: remove outdated note +- ref-manual: system-requirements: simplify supported distro requirements +- ref-manual: system-requirements: update packages to build docs +- scripts/runqemu: add qmp socket support +- scripts/runqemu: direct mesa to use its own drivers, rather than ones provided by host distro +- scripts/runqemu: fix regex escape sequences +- scripts: python 3.12 regex +- selftest: skip virgl gtk/sdl test on ubuntu 18.04 +- systemd: Only add myhostname to nsswitch.conf if in :term:`PACKAGECONFIG` +- tzdata : Upgrade to 2024a +- u-boot: Move UBOOT_INITIAL_ENV back to u-boot.inc +- useradd-example: do not use unsupported clear text password +- vim: upgrade to v9.0.2190 +- yocto-bsp: update to v5.15.150 + + +Known Issues in Yocto-4.0.17 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- N/A + + +Contributors to Yocto-4.0.17 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Adrian Freihofer +- Alassane Yattara +- Alexander Kanavin +- Alexander Sverdlin +- Archana Polampalli +- Baruch Siach +- Bruce Ashfield +- Chen Qi +- Chris Laplante +- Deepthi Hemraj +- Dhairya Nagodra +- Fabien Mahot +- Fabio Estevam +- Hitendra Prajapati +- Hugo SIMELIERE +- Jermain Horsman +- Kai Kang +- Lee Chee Yang +- Ludovic Jozeau +- Michael Opdenacker +- Ming Liu +- Munehisa Kamata +- Narpat Mali +- Nikhil R +- Paul Eggleton +- Paulo Neves +- Peter Marko +- Philip Lorenz +- Poonam Jadhav +- Priyal Doshi +- Ross Burton +- Simone Weiß +- Soumya Sambu +- Steve Sakoman +- Tim Orling +- Trevor Gamblin +- Vijay Anusuri +- Vivek Kumbhar +- Wang Mingyu +- Zahir Hussain + + +Repositories / Downloads for Yocto-4.0.17 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +poky + +- Repository Location: :yocto_git:`/poky` +- Branch: :yocto_git:`kirkstone ` +- Tag: :yocto_git:`yocto-4.0.17 ` +- Git Revision: :yocto_git:`6d1a878bbf24c66f7186b270f823fcdf82e35383 ` +- Release Artefact: poky-6d1a878bbf24c66f7186b270f823fcdf82e35383 +- sha: 3bc3010340b674f7b0dd0a7997f0167b2240b794fbd4aa28c0c4217bddd15e30 +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/poky-6d1a878bbf24c66f7186b270f823fcdf82e35383.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/poky-6d1a878bbf24c66f7186b270f823fcdf82e35383.tar.bz2 + +openembedded-core + +- Repository Location: :oe_git:`/openembedded-core` +- Branch: :oe_git:`kirkstone ` +- Tag: :oe_git:`yocto-4.0.17 ` +- Git Revision: :oe_git:`2501534c9581c6c3439f525d630be11554a57d24 ` +- Release Artefact: oecore-2501534c9581c6c3439f525d630be11554a57d24 +- sha: 52cc6cce9e920bdce078584b89136e81cc01e0c55616fab5fca6c3e04264c88e +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/oecore-2501534c9581c6c3439f525d630be11554a57d24.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/oecore-2501534c9581c6c3439f525d630be11554a57d24.tar.bz2 + +meta-mingw + +- Repository Location: :yocto_git:`/meta-mingw` +- Branch: :yocto_git:`kirkstone ` +- Tag: :yocto_git:`yocto-4.0.17 ` +- Git Revision: :yocto_git:`f6b38ce3c90e1600d41c2ebb41e152936a0357d7 ` +- Release Artefact: meta-mingw-f6b38ce3c90e1600d41c2ebb41e152936a0357d7 +- sha: 7d57167c19077f4ab95623d55a24c2267a3a3fb5ed83688659b4c03586373b25 +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/meta-mingw-f6b38ce3c90e1600d41c2ebb41e152936a0357d7.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/meta-mingw-f6b38ce3c90e1600d41c2ebb41e152936a0357d7.tar.bz2 + +meta-gplv2 + +- Repository Location: :yocto_git:`/meta-gplv2` +- Branch: :yocto_git:`kirkstone ` +- Tag: :yocto_git:`yocto-4.0.17 ` +- Git Revision: :yocto_git:`d2f8b5cdb285b72a4ed93450f6703ca27aa42e8a ` +- Release Artefact: meta-gplv2-d2f8b5cdb285b72a4ed93450f6703ca27aa42e8a +- sha: c386f59f8a672747dc3d0be1d4234b6039273d0e57933eb87caa20f56b9cca6d +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/meta-gplv2-d2f8b5cdb285b72a4ed93450f6703ca27aa42e8a.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/meta-gplv2-d2f8b5cdb285b72a4ed93450f6703ca27aa42e8a.tar.bz2 + +meta-clang + +- Repository Location: :yocto_git:`/meta-clang` +- Branch: :yocto_git:`kirkstone ` +- Tag: :yocto_git:`yocto-4.0.17 ` +- Git Revision: :yocto_git:`eebe4ff2e539f3ffb01c5060cc4ca8b226ea8b52 ` +- Release Artefact: meta-clang-eebe4ff2e539f3ffb01c5060cc4ca8b226ea8b52 +- sha: 3299e96e069a22c0971e903fbc191f2427efffc83d910ac51bf0237caad01d17 +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/meta-clang-eebe4ff2e539f3ffb01c5060cc4ca8b226ea8b52.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/meta-clang-eebe4ff2e539f3ffb01c5060cc4ca8b226ea8b52.tar.bz2 + +bitbake + +- Repository Location: :oe_git:`/bitbake` +- Branch: :oe_git:`2.0 ` +- Tag: :oe_git:`yocto-4.0.17 ` +- Git Revision: :oe_git:`40fd5f4eef7460ca67f32cfce8e229e67e1ff607 ` +- Release Artefact: bitbake-40fd5f4eef7460ca67f32cfce8e229e67e1ff607 +- sha: 5d20a0e4c5d0fce44bd84778168714a261a30a4b83f67c88df3b8a7e7115e444 +- Download Locations: + http://downloads.yoctoproject.org/releases/yocto/yocto-4.0.17/bitbake-40fd5f4eef7460ca67f32cfce8e229e67e1ff607.tar.bz2 + http://mirrors.kernel.org/yocto/yocto/yocto-4.0.17/bitbake-40fd5f4eef7460ca67f32cfce8e229e67e1ff607.tar.bz2 + +yocto-docs + +- Repository Location: :yocto_git:`/yocto-docs` +- Branch: :yocto_git:`kirkstone ` +- Tag: :yocto_git:`yocto-4.0.17 ` +- Git Revision: :yocto_git:`08ce7db2aa3a38deb8f5aa59bafc78542986babb ` + From patchwork Mon Mar 25 13:27:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1862FCD1280 for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by mx.groups.io with SMTP id smtpd.web10.51974.1711373269072443268 for ; Mon, 25 Mar 2024 06:27:49 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=MshSyefW; spf=pass (domain: bootlin.com, ip: 217.70.183.194, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 2A2B940006; Mon, 25 Mar 2024 13:27:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=N5f1FbkEu3bxSE/sMeMvmCkuo6UZAq+EZZ9EsKQJeN4=; b=MshSyefWzZgp9l/yOYntshRPdq64wFYSm2U+d+WHiB66hHzBfB3Z95vOo730EMfSa3KRlP z0VeBXVgvbob6IoOy2Jm6YFxgsby+wBzyY0Ni8OEj3HPMqMIUr9+D3vCRGhCXKR1QJLLKS UfiP9G4KkjiFXWamC+z1VDnEZrv2yoXN5cacORxwV35WBMhbt6I1G0AMZGYOceJGB4aLRK gjYwJgTyNGlNkXUU0Iz8nzZQmxu1GReW5fWj7rjCTykKypWYVxKvX1xE8kgpENcwqPtGwH 97q9ZQGt+gS6gYGY25tN3ZBX2zNCX5JmSMTASFUvZtbUVFEr9re/37oxkpp/7g== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , Martin Jansa Subject: [kirkstone][PATCH 6/9] contributor-guide: be more specific about meta-* trees Date: Mon, 25 Mar 2024 14:27:34 +0100 Message-Id: <20240325132737.3018073-7-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5004 From: Michael Opdenacker From: Martin Jansa * this is often confused to apply for e.g. meta-oe as well where it doesn't apply as meta-oe has own ML mentioned in README. Signed-off-by: Martin Jansa Reviewed-by: Michael Opdenacker --- documentation/contributor-guide/submit-changes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/contributor-guide/submit-changes.rst b/documentation/contributor-guide/submit-changes.rst index c0f2aa7cb4..15566701f6 100644 --- a/documentation/contributor-guide/submit-changes.rst +++ b/documentation/contributor-guide/submit-changes.rst @@ -391,7 +391,7 @@ varies by component: :oe_lists:`bitbake-devel ` mailing list. -- *"meta-\*" trees:* These trees contain Metadata. Use the +- *meta-poky* and *meta-yocto-bsp* trees: These trees contain Metadata. Use the :yocto_lists:`poky ` mailing list. - *Documentation*: For changes to the Yocto Project documentation, use the From patchwork Mon Mar 25 13:27:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18E62CD1282 for ; Mon, 25 Mar 2024 13:27:52 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by mx.groups.io with SMTP id smtpd.web10.51975.1711373269857985877 for ; Mon, 25 Mar 2024 06:27:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=X8TU2kfq; spf=pass (domain: bootlin.com, ip: 217.70.183.195, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id D044960007; Mon, 25 Mar 2024 13:27:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373268; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3vhJGYpW4LLCI/hgEPqJJOA6HiofMOBDmcFFnWHCNoI=; b=X8TU2kfqw3zuJ5cyWc8r/oYkgxaDXGA9dIIco5pwU8BbEVj9ucupzAybBXFlMUAyuDKfGw 9t++INXvGxrGUfB1UZZIXwJ8Dz1eV4rRJ6Mao8n21Z3HS0TjTpIS9PoRWKDM+Z61vS+Op0 EseGcnLJQkUg6XgioE2tlO0C8swVr+XBvvtpp7vesezeid5i4r93OYRYo4QsmMQsLqAzYb Vf2mEyHMY4EBbvu7h1PSHla3+qRNJ0QKTOPwF+vlsc1MAneNys4VMfzUC9tFtrdPNbtRRR YgodpG4Du7XAY1D9PGGMN/ufAW0kNMua+TP3ylBTo43VIDS6U1+4QVNc1R5Alw== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker Subject: [kirkstone][PATCH 7/9] profile-manual: usage.rst: further style improvements Date: Mon, 25 Mar 2024 14:27:35 +0100 Message-Id: <20240325132737.3018073-8-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:52 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5005 From: Michael Opdenacker According to errors reported by "make stylecheck" Signed-off-by: Michael Opdenacker --- documentation/profile-manual/usage.rst | 335 +++++++++--------- .../config/vocabularies/OpenSource/accept.txt | 20 +- .../config/vocabularies/Yocto/accept.txt | 5 +- 3 files changed, 187 insertions(+), 173 deletions(-) diff --git a/documentation/profile-manual/usage.rst b/documentation/profile-manual/usage.rst index 28dcb969a0..542bd918b9 100644 --- a/documentation/profile-manual/usage.rst +++ b/documentation/profile-manual/usage.rst @@ -10,7 +10,7 @@ Basic Usage (with examples) for each of the Yocto Tracing Tools This chapter presents basic usage examples for each of the tracing tools. -Perf +perf ==== The perf tool is the profiling and tracing tool that comes bundled @@ -26,12 +26,12 @@ of what's going on. In many ways, perf aims to be a superset of all the tracing and profiling tools available in Linux today, including all the other tools -covered in this HOWTO. The past couple of years have seen perf subsume a +covered in this How-to. The past couple of years have seen perf subsume a lot of the functionality of those other tools and, at the same time, those other tools have removed large portions of their previous functionality and replaced it with calls to the equivalent functionality now implemented by the perf subsystem. Extrapolation suggests that at -some point those other tools will simply become completely redundant and +some point those other tools will become completely redundant and go away; until then, we'll cover those other tools in these pages and in many cases show how the same things can be accomplished in perf and the other tools when it seems useful to do so. @@ -41,7 +41,7 @@ want to apply the tool; full documentation can be found either within the tool itself or in the manual pages at `perf(1) `__. -Perf Setup +perf Setup ---------- For this section, we'll assume you've already performed the basic setup @@ -54,14 +54,14 @@ image built with the following in your ``local.conf`` file:: perf runs on the target system for the most part. You can archive profile data and copy it to the host for analysis, but for the rest of -this document we assume you've ssh'ed to the host and will be running -the perf commands on the target. +this document we assume you're connected to the host through SSH and will be +running the perf commands on the target. -Basic Perf Usage +Basic perf Usage ---------------- The perf tool is pretty much self-documenting. To remind yourself of the -available commands, simply type ``perf``, which will show you basic usage +available commands, just type ``perf``, which will show you basic usage along with the available perf subcommands:: root@crownbay:~# perf @@ -101,7 +101,7 @@ As a simple test case, we'll profile the ``wget`` of a fairly large file, which is a minimally interesting case because it has both file and network I/O aspects, and at least in the case of standard Yocto images, it's implemented as part of BusyBox, so the methods we use to analyze it -can be used in a very similar way to the whole host of supported BusyBox +can be used in a similar way to the whole host of supported BusyBox applets in Yocto:: root@crownbay:~# rm linux-2.6.19.2.tar.bz2; \ @@ -164,17 +164,17 @@ hits and misses:: 44.831023415 seconds time elapsed -So ``perf stat`` gives us a nice easy +As you can see, ``perf stat`` gives us a nice easy way to get a quick overview of what might be happening for a set of events, but normally we'd need a little more detail in order to understand what's going on in a way that we can act on in a useful way. To dive down into a next level of detail, we can use ``perf record`` / ``perf report`` which will collect profiling data and present it to use using an -interactive text-based UI (or simply as text if we specify ``--stdio`` to +interactive text-based UI (or just as text if we specify ``--stdio`` to ``perf report``). -As our first attempt at profiling this workload, we'll simply run ``perf +As our first attempt at profiling this workload, we'll just run ``perf record``, handing it the workload we want to profile (everything after ``perf record`` and any perf options we hand it --- here none, will be executed in a new shell). perf collects samples until the process exits @@ -189,7 +189,7 @@ directory:: [ perf record: Captured and wrote 0.176 MB perf.data (~7700 samples) ] To see the results in a -"text-based UI" (tui), simply run ``perf report``, which will read the +"text-based UI" (tui), just run ``perf report``, which will read the perf.data file in the current working directory and display the results in an interactive UI:: @@ -203,10 +203,10 @@ The above screenshot displays a "flat" profile, one entry for each profiling run, ordered from the most popular to the least (perf has options to sort in various orders and keys as well as display entries only above a certain threshold and so on --- see the perf documentation -for details). Note that this includes both userspace functions (entries +for details). Note that this includes both user space functions (entries containing a ``[.]``) and kernel functions accounted to the process (entries containing a ``[k]``). perf has command-line modifiers that can be used to -restrict the profiling to kernel or userspace, among others. +restrict the profiling to kernel or user space, among others. Notice also that the above report shows an entry for ``busybox``, which is the executable that implements ``wget`` in Yocto, but that instead of a @@ -217,7 +217,7 @@ Before we do that, however, let's try running a different profile, one which shows something a little more interesting. The only difference between the new profile and the previous one is that we'll add the ``-g`` option, which will record not just the address of a sampled function, -but the entire callchain to the sampled function as well:: +but the entire call chain to the sampled function as well:: root@crownbay:~# perf record -g wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 Connecting to downloads.yoctoproject.org (140.211.169.59:80) @@ -231,26 +231,26 @@ but the entire callchain to the sampled function as well:: .. image:: figures/perf-wget-g-copy-to-user-expanded-stripped.png :align: center -Using the callgraph view, we can actually see not only which functions +Using the call graph view, we can actually see not only which functions took the most time, but we can also see a summary of how those functions were called and learn something about how the program interacts with the kernel in the process. Notice that each entry in the above screenshot now contains a ``+`` on the -left-hand side. This means that we can expand the entry and drill down -into the callchains that feed into that entry. Pressing ``Enter`` on any -one of them will expand the callchain (you can also press ``E`` to expand +left side. This means that we can expand the entry and drill down +into the call chains that feed into that entry. Pressing ``Enter`` on any +one of them will expand the call chain (you can also press ``E`` to expand them all at the same time or ``C`` to collapse them all). In the screenshot above, we've toggled the ``__copy_to_user_ll()`` entry -and several subnodes all the way down. This lets us see which callchains +and several subnodes all the way down. This lets us see which call chains contributed to the profiled ``__copy_to_user_ll()`` function which contributed 1.77% to the total profile. -As a bit of background explanation for these callchains, think about -what happens at a high level when you run wget to get a file out on the +As a bit of background explanation for these call chains, think about +what happens at a high level when you run ``wget`` to get a file out on the network. Basically what happens is that the data comes into the kernel -via the network connection (socket) and is passed to the userspace +via the network connection (socket) and is passed to the user space program ``wget`` (which is actually a part of BusyBox, but that's not important for now), which takes the buffers the kernel passes to it and writes it to a disk file to save it. @@ -260,15 +260,15 @@ is the part where the kernel passes the data it has read from the socket down to wget i.e. a ``copy-to-user``. Notice also that here there's also a case where the hex value is -displayed in the callstack, here in the expanded ``sys_clock_gettime()`` -function. Later we'll see it resolve to a userspace function call in -busybox. +displayed in the call stack, here in the expanded ``sys_clock_gettime()`` +function. Later we'll see it resolve to a user space function call in +BusyBox. .. image:: figures/perf-wget-g-copy-from-user-expanded-stripped.png :align: center The above screenshot shows the other half of the journey for the data --- -from the ``wget`` program's userspace buffers to disk. To get the buffers to +from the ``wget`` program's user space buffers to disk. To get the buffers to disk, the wget program issues a ``write(2)``, which does a ``copy-from-user`` to the kernel, which then takes care via some circuitous path (probably also present somewhere in the profile data), to get it safely to disk. @@ -278,8 +278,8 @@ of how to extract useful information out of it, let's get back to the task at hand and see if we can get some basic idea about where the time is spent in the program we're profiling, wget. Remember that wget is actually implemented as an applet in BusyBox, so while the process name -is ``wget``, the executable we're actually interested in is BusyBox. So -let's expand the first entry containing BusyBox: +is ``wget``, the executable we're actually interested in is ``busybox``. +Therefore, let's expand the first entry containing BusyBox: .. image:: figures/perf-wget-busybox-expanded-stripped.png :align: center @@ -289,7 +289,7 @@ hex value instead of a symbol as with most of the kernel entries. Expanding the BusyBox entry doesn't make it any better. The problem is that perf can't find the symbol information for the -busybox binary, which is actually stripped out by the Yocto build +``busybox`` binary, which is actually stripped out by the Yocto build system. One way around that is to put the following in your ``local.conf`` file @@ -299,20 +299,20 @@ when you build the image:: However, we already have an image with the binaries stripped, so what can we do to get perf to resolve the symbols? Basically we need to -install the debuginfo for the BusyBox package. +install the debugging information for the BusyBox package. To generate the debug info for the packages in the image, we can add ``dbg-pkgs`` to :term:`EXTRA_IMAGE_FEATURES` in ``local.conf``. For example:: EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs" -Additionally, in order to generate the type of debuginfo that perf +Additionally, in order to generate the type of debugging information that perf understands, we also need to set :term:`PACKAGE_DEBUG_SPLIT_STYLE` in the ``local.conf`` file:: PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory' -Once we've done that, we can install the debuginfo for BusyBox. The +Once we've done that, we can install the debugging information for BusyBox. The debug packages once built can be found in ``build/tmp/deploy/rpm/*`` on the host system. Find the ``busybox-dbg-...rpm`` file and copy it to the target. For example:: @@ -320,11 +320,11 @@ to the target. For example:: [trz@empanada core2]$ scp /home/trz/yocto/crownbay-tracing-dbg/build/tmp/deploy/rpm/core2_32/busybox-dbg-1.20.2-r2.core2_32.rpm root@192.168.1.31: busybox-dbg-1.20.2-r2.core2_32.rpm 100% 1826KB 1.8MB/s 00:01 -Now install the debug rpm on the target:: +Now install the debug RPM on the target:: root@crownbay:~# rpm -i busybox-dbg-1.20.2-r2.core2_32.rpm -Now that the debuginfo is installed, we see that the BusyBox entries now display +Now that the debugging information is installed, we see that the BusyBox entries now display their functions symbolically: .. image:: figures/perf-wget-busybox-debuginfo.png @@ -344,7 +344,7 @@ expanded all the nodes using the ``E`` key): .. image:: figures/perf-wget-busybox-dso-zoom.png :align: center -Finally, we can see that now that the BusyBox debuginfo is installed, +Finally, we can see that now that the BusyBox debugging information is installed, the previously unresolved symbol in the ``sys_clock_gettime()`` entry mentioned previously is now resolved, and shows that the ``sys_clock_gettime`` system call that was the source of 6.75% of the @@ -376,8 +376,8 @@ counter, something other than the default ``cycles``. The tracing and profiling infrastructure in Linux has become unified in a way that allows us to use the same tool with a completely different set of counters, not just the standard hardware counters that -traditional tools have had to restrict themselves to (of course the -traditional tools can also make use of the expanded possibilities now +traditional tools have had to restrict themselves to (the +traditional tools can now actually make use of the expanded possibilities now available to them, and in some cases have, as mentioned previously). We can get a list of the available events that can be used to profile a @@ -517,14 +517,14 @@ workload via ``perf list``:: .. admonition:: Tying it Together These are exactly the same set of events defined by the trace event - subsystem and exposed by ftrace / tracecmd / kernelshark as files in + subsystem and exposed by ftrace / trace-cmd / KernelShark as files in ``/sys/kernel/debug/tracing/events``, by SystemTap as kernel.trace("tracepoint_name") and (partially) accessed by LTTng. Only a subset of these would be of interest to us when looking at this workload, so let's choose the most likely subsystems (identified by the -string before the colon in the Tracepoint events) and do a ``perf stat`` -run using only those wildcarded subsystems:: +string before the colon in the ``Tracepoint`` events) and do a ``perf stat`` +run using only those subsystem wildcards:: root@crownbay:~# perf stat -e skb:* -e net:* -e napi:* -e sched:* -e workqueue:* -e irq:* -e syscalls:* wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2 Performance counter stats for 'wget &YOCTO_DL_URL;/mirror/sources/linux-2.6.19.2.tar.bz2': @@ -614,8 +614,8 @@ accounts for the function name actually displayed in the profile: } A couple of the more interesting -callchains are expanded and displayed above, basically some network -receive paths that presumably end up waking up wget (busybox) when +call chains are expanded and displayed above, basically some network +receive paths that presumably end up waking up wget (BusyBox) when network data is ready. Note that because tracepoints are normally used for tracing, the default @@ -635,7 +635,7 @@ high-level view of what's going on with a workload or across the system. It is however by definition an approximation, as suggested by the most prominent word associated with it, ``sampling``. On the one hand, it allows a representative picture of what's going on in the system to be -cheaply taken, but on the other hand, that cheapness limits its utility +cheaply taken, but alternatively, that cheapness limits its utility when that data suggests a need to "dive down" more deeply to discover what's really going on. In such cases, the only way to see what's really going on is to be able to look at (or summarize more intelligently) the @@ -700,7 +700,7 @@ an infinite variety of ways. Another way to look at it is that there are only so many ways that the 'primitive' counters can be used on their own to generate interesting output; to get anything more complicated than simple counts requires -some amount of additional logic, which is typically very specific to the +some amount of additional logic, which is typically specific to the problem at hand. For example, if we wanted to make use of a 'counter' that maps to the value of the time difference between when a process was scheduled to run on a processor and the time it actually ran, we @@ -711,12 +711,12 @@ standard profiling tools how much data every process on the system reads and writes, along with how many of those reads and writes fail completely. If we have sufficient trace data, however, we could with the right tools easily extract and present that information, but we'd need -something other than pre-canned profiling tools to do that. +something other than ready-made profiling tools to do that. Luckily, there is a general-purpose way to handle such needs, called "programming languages". Making programming languages easily available to apply to such problems given the specific format of data is called a -'programming language binding' for that data and language. Perf supports +'programming language binding' for that data and language. perf supports two programming language bindings, one for Python and one for Perl. .. admonition:: Tying it Together @@ -726,7 +726,7 @@ two programming language bindings, one for Python and one for Perl. DProbes dpcc compiler, an ANSI C compiler which targeted a low-level assembly language running on an in-kernel interpreter on the target system. This is exactly analogous to what Sun's DTrace did, except - that DTrace invented its own language for the purpose. Systemtap, + that DTrace invented its own language for the purpose. SystemTap, heavily inspired by DTrace, also created its own one-off language, but rather than running the product on an in-kernel interpreter, created an elaborate compiler-based machinery to translate its @@ -739,8 +739,8 @@ entry / exit events we recorded:: root@crownbay:~# perf script -g python generated Python script: perf-script.py -The skeleton script simply creates a Python function for each event type in the -``perf.data`` file. The body of each function simply prints the event name along +The skeleton script just creates a Python function for each event type in the +``perf.data`` file. The body of each function just prints the event name along with its parameters. For example: .. code-block:: python @@ -783,7 +783,7 @@ We can run that script directly to print all of the events contained in the syscalls__sys_exit_read 1 11624.859944032 1262 wget nr=3, ret=1024 That in itself isn't very useful; after all, we can accomplish pretty much the -same thing by simply running ``perf script`` without arguments in the same +same thing by just running ``perf script`` without arguments in the same directory as the ``perf.data`` file. We can however replace the print statements in the generated function @@ -805,7 +805,7 @@ event. For example: Each event handler function in the generated code is modified to do this. For convenience, we define a common function -called ``inc_counts()`` that each handler calls; ``inc_counts()`` simply tallies +called ``inc_counts()`` that each handler calls; ``inc_counts()`` just tallies a count for each event using the ``counts`` hash, which is a specialized hash function that does Perl-like autovivification, a capability that's extremely useful for kinds of multi-level aggregation commonly used in @@ -865,7 +865,7 @@ System-Wide Tracing and Profiling ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The examples so far have focused on tracing a particular program or -workload --- in other words, every profiling run has specified the program +workload --- that is, every profiling run has specified the program to profile in the command-line e.g. ``perf record wget ...``. It's also possible, and more interesting in many cases, to run a @@ -894,13 +894,13 @@ other processes running on the system as well: .. image:: figures/perf-systemwide.png :align: center -In the snapshot above, we can see callchains that originate in libc, and -a callchain from Xorg that demonstrates that we're using a proprietary X -driver in userspace (notice the presence of ``PVR`` and some other -unresolvable symbols in the expanded Xorg callchain). +In the snapshot above, we can see call chains that originate in ``libc``, and +a call chain from ``Xorg`` that demonstrates that we're using a proprietary X +driver in user space (notice the presence of ``PVR`` and some other +unresolvable symbols in the expanded ``Xorg`` call chain). -Note also that we have both kernel and userspace entries in the above -snapshot. We can also tell perf to focus on userspace but providing a +Note also that we have both kernel and user space entries in the above +snapshot. We can also tell perf to focus on user space but providing a modifier, in this case ``u``, to the ``cycles`` hardware counter when we record a profile:: @@ -911,7 +911,7 @@ record a profile:: .. image:: figures/perf-report-cycles-u.png :align: center -Notice in the screenshot above, we see only userspace entries (``[.]``) +Notice in the screenshot above, we see only user space entries (``[.]``) Finally, we can press ``Enter`` on a leaf node and select the ``Zoom into DSO`` menu item to show only entries associated with a specific DSO. In @@ -946,7 +946,7 @@ We can look at the raw output using ``perf script`` with no arguments:: Filtering ^^^^^^^^^ -Notice that there are a lot of events that don't really have anything to +Notice that there are many events that don't really have anything to do with what we're interested in, namely events that schedule ``perf`` itself in and out or that wake perf up. We can get rid of those by using the ``--filter`` option --- for each event we specify using ``-e``, we can add a @@ -985,7 +985,7 @@ purpose of demonstrating how to use filters, it's close enough. .. admonition:: Tying it Together These are exactly the same set of event filters defined by the trace - event subsystem. See the ftrace / tracecmd / kernelshark section for more + event subsystem. See the ftrace / trace-cmd / KernelShark section for more discussion about these event filters. .. admonition:: Tying it Together @@ -995,14 +995,14 @@ purpose of demonstrating how to use filters, it's close enough. indispensable part of the perf design as it relates to tracing. kernel-based event filters provide a mechanism to precisely throttle the event stream that appears in user space, where it makes sense to - provide bindings to real programming languages for postprocessing the + provide bindings to real programming languages for post-processing the event stream. This architecture allows for the intelligent and flexible partitioning of processing between the kernel and user space. Contrast this with other tools such as SystemTap, which does all of its processing in the kernel and as such requires a special project-defined language in order to accommodate that design, or - LTTng, where everything is sent to userspace and as such requires a - super-efficient kernel-to-userspace transport mechanism in order to + LTTng, where everything is sent to user space and as such requires a + super-efficient kernel-to-user space transport mechanism in order to function properly. While perf certainly can benefit from for instance advances in the design of the transport, it doesn't fundamentally depend on them. Basically, if you find that your perf tracing @@ -1014,7 +1014,7 @@ Using Dynamic Tracepoints perf isn't restricted to the fixed set of static tracepoints listed by ``perf list``. Users can also add their own "dynamic" tracepoints anywhere -in the kernel. For instance, suppose we want to define our own +in the kernel. For example, suppose we want to define our own tracepoint on ``do_fork()``. We can do that using the ``perf probe`` perf subcommand:: @@ -1069,7 +1069,7 @@ up after 30 seconds):: [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.087 MB perf.data (~3812 samples) ] -Using ``perf script`` we can see each do_fork event that fired:: +Using ``perf script`` we can see each ``do_fork`` event that fired:: root@crownbay:~# perf script @@ -1111,7 +1111,7 @@ Using ``perf script`` we can see each do_fork event that fired:: gaku 1312 [000] 34237.202388: do_fork: (c1028460) And using ``perf report`` on the same file, we can see the -callgraphs from starting a few programs during those 30 seconds: +call graphs from starting a few programs during those 30 seconds: .. image:: figures/perf-probe-do_fork-profile.png :align: center @@ -1125,11 +1125,11 @@ callgraphs from starting a few programs during those 30 seconds: .. admonition:: Tying it Together - Dynamic tracepoints are implemented under the covers by kprobes and - uprobes. kprobes and uprobes are also used by and in fact are the + Dynamic tracepoints are implemented under the covers by Kprobes and + Uprobes. Kprobes and Uprobes are also used by and in fact are the main focus of SystemTap. -Perf Documentation +perf Documentation ------------------ Online versions of the manual pages for the commands discussed in this @@ -1153,7 +1153,7 @@ section can be found here: - The top-level `perf(1) manual page `__. -Normally, you should be able to invoke the manual pages via perf itself +Normally, you should be able to open the manual pages via perf itself e.g. ``perf help`` or ``perf help record``. To have the perf manual pages installed on your target, modify your @@ -1168,14 +1168,14 @@ of examples, can also be found in the ``perf`` directory of the kernel tree:: tools/perf/Documentation There's also a nice perf tutorial on the perf -wiki that goes into more detail than we do here in certain areas: `Perf +wiki that goes into more detail than we do here in certain areas: `perf Tutorial `__ ftrace ====== "ftrace" literally refers to the "ftrace function tracer" but in reality -this encompasses a number of related tracers along with the +this encompasses several related tracers along with the infrastructure that they all make use of. ftrace Setup @@ -1184,11 +1184,11 @@ ftrace Setup For this section, we'll assume you've already performed the basic setup outlined in the ":ref:`profile-manual/intro:General Setup`" section. -ftrace, trace-cmd, and kernelshark run on the target system, and are +ftrace, trace-cmd, and KernelShark run on the target system, and are ready to go out-of-the-box --- no additional setup is necessary. For the -rest of this section we assume you've ssh'ed to the host and will be -running ftrace on the target. kernelshark is a GUI application and if -you use the ``-X`` option to ssh you can have the kernelshark GUI run on +rest of this section we assume you're connected to the host through SSH and +will be running ftrace on the target. KernelShark is a GUI application and if +you use the ``-X`` option to ``ssh`` you can have the KernelShark GUI run on the target but display remotely on the host if you want. Basic ftrace usage @@ -1196,8 +1196,8 @@ Basic ftrace usage "ftrace" essentially refers to everything included in the ``/tracing`` directory of the mounted debugfs filesystem (Yocto follows the standard -convention and mounts it at ``/sys/kernel/debug``). Here's a listing of all -the files found in ``/sys/kernel/debug/tracing`` on a Yocto system:: +convention and mounts it at ``/sys/kernel/debug``). All the files found in +``/sys/kernel/debug/tracing`` on a Yocto system are:: root@sugarbay:/sys/kernel/debug/tracing# ls README kprobe_events trace @@ -1222,7 +1222,7 @@ the ftrace documentation. We'll start by looking at some of the available built-in tracers. -cat'ing the ``available_tracers`` file lists the set of available tracers:: +The ``available_tracers`` file lists the set of available tracers:: root@sugarbay:/sys/kernel/debug/tracing# cat available_tracers blk function_graph function nop @@ -1232,11 +1232,11 @@ The ``current_tracer`` file contains the tracer currently in effect:: root@sugarbay:/sys/kernel/debug/tracing# cat current_tracer nop -The above listing of current_tracer shows that the +The above listing of ``current_tracer`` shows that the ``nop`` tracer is in effect, which is just another way of saying that there's actually no tracer currently in effect. -echo'ing one of the available_tracers into ``current_tracer`` makes the +Writing one of the available tracers into ``current_tracer`` makes the specified tracer the current tracer:: root@sugarbay:/sys/kernel/debug/tracing# echo function > current_tracer @@ -1292,7 +1292,7 @@ tracer:: . Each line in the trace above shows what was happening in the kernel on a given -cpu, to the level of detail of function calls. Each entry shows the function +CPU, to the level of detail of function calls. Each entry shows the function called, followed by its caller (after the arrow). The function tracer gives you an extremely detailed idea of what the @@ -1306,7 +1306,7 @@ great way to learn about how the kernel code works in a dynamic sense. It is a little more difficult to follow the call chains than it needs to be --- luckily there's a variant of the function tracer that displays the -callchains explicitly, called the ``function_graph`` tracer:: +call chains explicitly, called the ``function_graph`` tracer:: root@sugarbay:/sys/kernel/debug/tracing# echo function_graph > current_tracer root@sugarbay:/sys/kernel/debug/tracing# cat trace | less @@ -1425,7 +1425,7 @@ As you can see, the ``function_graph`` display is much easier to follow. Also note that in addition to the function calls and associated braces, other events such as scheduler events are displayed in context. In fact, you can freely include any tracepoint available in -the trace events subsystem described in the next section by simply +the trace events subsystem described in the next section by just enabling those events, and they'll appear in context in the function graph display. Quite a powerful tool for understanding kernel dynamics. @@ -1528,7 +1528,7 @@ The ``format`` file for the tracepoint describes the event in memory, which is used by the various tracing tools that now make use of these tracepoint to parse the event and make sense of it, along with a ``print fmt`` field that allows tools -like ftrace to display the event as text. Here's what the format of the +like ftrace to display the event as text. The format of the ``kmalloc`` event looks like:: root@sugarbay:/sys/kernel/debug/tracing/events/kmem/kmalloc# cat format @@ -1581,7 +1581,7 @@ events in the output buffer:: root@sugarbay:/sys/kernel/debug/tracing# echo 1 > tracing_on Now, if we look at the ``trace`` file, we see nothing -but the kmalloc events we just turned on:: +but the ``kmalloc`` events we just turned on:: root@sugarbay:/sys/kernel/debug/tracing# cat trace | less # tracer: nop @@ -1636,8 +1636,8 @@ using the ``enable`` file in the subsystem directory) and get an arbitrarily fine-grained idea of what's going on in the system by enabling as many of the appropriate tracepoints as applicable. -A number of the tools described in this HOWTO do just that, including -``trace-cmd`` and kernelshark in the next section. +Several tools described in this How-to do just that, including +``trace-cmd`` and KernelShark in the next section. .. admonition:: Tying it Together @@ -1653,7 +1653,7 @@ A number of the tools described in this HOWTO do just that, including ``/sys/kernel/debug/tracing`` will be removed and replaced with equivalent tracers based on the "trace events" subsystem. -trace-cmd / kernelshark +trace-cmd / KernelShark ----------------------- trace-cmd is essentially an extensive command-line "wrapper" interface @@ -1662,31 +1662,30 @@ that hides the details of all the individual files in events within the ``/sys/kernel/debug/tracing/events/`` subdirectory and to collect traces and avoid having to deal with those details directly. -As yet another layer on top of that, kernelshark provides a GUI that +As yet another layer on top of that, KernelShark provides a GUI that allows users to start and stop traces and specify sets of events using an intuitive interface, and view the output as both trace events and as a per-CPU graphical display. It directly uses trace-cmd as the plumbing that accomplishes all that underneath the covers (and actually displays the trace-cmd command it uses, as we'll see). -To start a trace using kernelshark, first start kernelshark:: +To start a trace using KernelShark, first start this tool:: root@sugarbay:~# kernelshark -Then bring up the ``Capture`` dialog by -choosing from the kernelshark menu:: +Then open up the ``Capture`` dialog by choosing from the KernelShark menu:: Capture | Record That will display the following dialog, which allows you to choose one or more -events (or even one or more complete subsystems) to trace: +events (or even entire subsystems) to trace: .. image:: figures/kernelshark-choose-events.png :align: center Note that these are exactly the same sets of events described in the previous trace events subsystem section, and in fact is where trace-cmd -gets them for kernelshark. +gets them for KernelShark. In the above screenshot, we've decided to explore the graphics subsystem a bit and so have chosen to trace all the tracepoints contained within @@ -1699,12 +1698,12 @@ will turn into the 'Stop' button after the trace has started): .. image:: figures/kernelshark-output-display.png :align: center -Notice that the right-hand pane shows the exact trace-cmd command-line +Notice that the right pane shows the exact trace-cmd command-line that's used to run the trace, along with the results of the trace-cmd run. Once the ``Stop`` button is pressed, the graphical view magically fills up -with a colorful per-cpu display of the trace data, along with the +with a colorful per-CPU display of the trace data, along with the detailed event listing below that: .. image:: figures/kernelshark-i915-display.png @@ -1717,7 +1716,7 @@ events``: :align: center The tool is pretty self-explanatory, but for more detailed information -on navigating through the data, see the `kernelshark +on navigating through the data, see the `KernelShark website `__. ftrace Documentation @@ -1733,41 +1732,41 @@ Documentation directory:: Documentation/trace/events.txt -There is a nice series of articles on using ftrace and trace-cmd at LWN: +A nice series of articles on using ftrace and trace-cmd are available at LWN: -- `Debugging the kernel using Ftrace - part +- `Debugging the kernel using ftrace - part 1 `__ -- `Debugging the kernel using Ftrace - part +- `Debugging the kernel using ftrace - part 2 `__ -- `Secrets of the Ftrace function +- `Secrets of the ftrace function tracer `__ - `trace-cmd: A front-end for - Ftrace `__ + ftrace `__ See also `KernelShark's documentation `__ for further usage details. -An amusing yet useful README (a tracing mini-HOWTO) can be found in +An amusing yet useful README (a tracing mini-How-to) can be found in ``/sys/kernel/debug/tracing/README``. -systemtap +SystemTap ========= SystemTap is a system-wide script-based tracing and profiling tool. SystemTap scripts are C-like programs that are executed in the kernel to gather / print / aggregate data extracted from the context they end up being -invoked under. +called under. For example, this probe from the `SystemTap -tutorial `__ simply prints a +tutorial `__ just prints a line every time any process on the system runs ``open()`` on a file. For each line, it prints the executable name of the program that opened the file, along -with its PID, and the name of the file it opened (or tried to open), -which it extracts from the open syscall's argstr. +with its PID, and the name of the file it opened (or tried to open), which it +extracts from the argument string (``argstr``) of the ``open`` system call. .. code-block:: none @@ -1782,13 +1781,13 @@ which it extracts from the open syscall's argstr. } Normally, to execute this -probe, you'd simply install systemtap on the system you want to probe, +probe, you'd just install SystemTap on the system you want to probe, and directly run the probe on that system e.g. assuming the name of the -file containing the above text is trace_open.stp:: +file containing the above text is ``trace_open.stp``:: # stap trace_open.stp -What systemtap does under the covers to run this probe is 1) parse and +What SystemTap does under the covers to run this probe is 1) parse and convert the probe to an equivalent "C" form, 2) compile the "C" form into a kernel module, 3) insert the module into the kernel, which arms it, and 4) collect the data generated by the probe and display it to the @@ -1801,25 +1800,25 @@ kernel build system unfortunately isn't typically part of the image running on the target. It is normally available on the "host" system that produced the target image however; in such cases, steps 1 and 2 are executed on the host system, and steps 3 and 4 are executed on the -target system, using only the systemtap "runtime". +target system, using only the SystemTap "runtime". -The systemtap support in Yocto assumes that only steps 3 and 4 are run +The SystemTap support in Yocto assumes that only steps 3 and 4 are run on the target; it is possible to do everything on the target, but this section assumes only the typical embedded use-case. -So basically what you need to do in order to run a systemtap script on +Therefore, what you need to do in order to run a SystemTap script on the target is to 1) on the host system, compile the probe into a kernel module that makes sense to the target, 2) copy the module onto the target system and 3) insert the module into the target kernel, which arms it, and 4) collect the data generated by the probe and display it to the user. -systemtap Setup +SystemTap Setup --------------- -Those are a lot of steps and a lot of details, but fortunately Yocto +Those are many steps and details, but fortunately Yocto includes a script called ``crosstap`` that will take care of those -details, allowing you to simply execute a systemtap script on the remote +details, allowing you to just execute a SystemTap script on the remote target, with arguments if necessary. In order to do this from a remote host, however, you need to have access @@ -1832,7 +1831,7 @@ having done a build:: Error: No target kernel build found. Did you forget to create a local build of your image? -'crosstap' requires a local sdk build of the target system +'crosstap' requires a local SDK build of the target system (or a build that includes 'tools-profile') in order to build kernel modules that can probe the target system. @@ -1848,11 +1847,11 @@ Practically speaking, that means you need to do the following: the BSP README and/or the widely available basic documentation that discusses how to build images). -- Build an -sdk version of the image e.g.:: +- Build an ``-sdk`` version of the image e.g.:: $ bitbake core-image-sato-sdk -- Or build a non-sdk image but include the profiling tools +- Or build a non-SDK image but include the profiling tools (edit ``local.conf`` and add ``tools-profile`` to the end of :term:``EXTRA_IMAGE_FEATURES`` variable):: @@ -1868,15 +1867,14 @@ Practically speaking, that means you need to do the following: .. note:: - SystemTap, which uses ``crosstap``, assumes you can establish an ssh + SystemTap, which uses ``crosstap``, assumes you can establish an SSH connection to the remote target. Please refer to the crosstap wiki - page for details on verifying ssh connections. Also, the ability to ssh + page for details on verifying SSH connections. Also, the ability to SSH into the target system is not enabled by default in ``*-minimal`` images. -So essentially what you need to -do is build an SDK image or image with ``tools-profile`` as detailed in -the ":ref:`profile-manual/intro:General Setup`" section of this -manual, and boot the resulting target image. +Therefore, what you need to do is build an SDK image or image with +``tools-profile`` as detailed in the ":ref:`profile-manual/intro:General Setup`" +section of this manual, and boot the resulting target image. .. note:: @@ -1884,12 +1882,12 @@ manual, and boot the resulting target image. to have the :term:`MACHINE` you're connecting to selected in ``local.conf``, and the kernel in that machine's :term:`Build Directory` must match the kernel on the booted system exactly, or you'll get the above ``crosstap`` message - when you try to invoke a script. + when you try to call a script. Running a Script on a Target ---------------------------- -Once you've done that, you should be able to run a systemtap script on +Once you've done that, you should be able to run a SystemTap script on the target:: $ cd /path/to/yocto @@ -1918,7 +1916,7 @@ If you get an error connecting to the target e.g.:: $ crosstap root@192.168.7.2 trace_open.stp error establishing ssh connection on remote 'root@192.168.7.2' -Try ssh'ing to the target and see what happens:: +Try connecting to the target through SSH and see what happens:: $ ssh root@192.168.7.2 @@ -1936,7 +1934,7 @@ no password): matchbox-termin(1036) open ("/tmp/vte3FS2LW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) matchbox-termin(1036) open ("/tmp/vteJMC7LW", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) -systemtap Documentation +SystemTap Documentation ----------------------- The SystemTap language reference can be found here: `SystemTap Language @@ -1949,7 +1947,7 @@ page `__ Sysprof ======= -Sysprof is a very easy to use system-wide profiler that consists of a +Sysprof is an easy to use system-wide profiler that consists of a single window with three panes and a few buttons which allow you to start, stop, and view the profile from one place. @@ -1959,16 +1957,16 @@ Sysprof Setup For this section, we'll assume you've already performed the basic setup outlined in the ":ref:`profile-manual/intro:General Setup`" section. -Sysprof is a GUI-based application that runs on the target system. For -the rest of this document we assume you've ssh'ed to the host and will -be running Sysprof on the target (you can use the ``-X`` option to ssh and +Sysprof is a GUI-based application that runs on the target system. For the rest +of this document we assume you're connected to the host through SSH and will be +running Sysprof on the target (you can use the ``-X`` option to ``ssh`` and have the Sysprof GUI run on the target but display remotely on the host if you want). Basic Sysprof Usage ------------------- -To start profiling the system, you simply press the ``Start`` button. To +To start profiling the system, you just press the ``Start`` button. To stop profiling and to start viewing the profile data in one easy step, press the ``Profile`` button. @@ -1981,11 +1979,11 @@ with profiling data: The left pane shows a list of functions and processes. Selecting one of those expands that function in the right pane, showing all its callees. Note that this caller-oriented display is essentially the inverse of -perf's default callee-oriented callchain display. +perf's default callee-oriented call chain display. In the screenshot above, we're focusing on ``__copy_to_user_ll()`` and -looking up the callchain we can see that one of the callers of -``__copy_to_user_ll`` is ``sys_read()`` and the complete callpath between them. +looking up the call chain we can see that one of the callers of +``__copy_to_user_ll`` is ``sys_read()`` and the complete call path between them. Notice that this is essentially a portion of the same information we saw in the perf display shown in the perf section of this page. @@ -1993,7 +1991,7 @@ in the perf display shown in the perf section of this page. :align: center Similarly, the above is a snapshot of the Sysprof display of a -``copy-from-user`` callchain. +``copy-from-user`` call chain. Finally, looking at the third Sysprof pane in the lower left, we can see a list of all the callers of a particular function selected in the top @@ -2008,18 +2006,17 @@ to the selected function, and so on. .. admonition:: Tying it Together - If you like sysprof's ``caller-oriented`` display, you may be able to + If you like Sysprof's ``caller-oriented`` display, you may be able to approximate it in other tools as well. For example, ``perf report`` has the ``-g`` (``--call-graph``) option that you can experiment with; one of the - options is ``caller`` for an inverted caller-based callgraph display. + options is ``caller`` for an inverted caller-based call graph display. Sysprof Documentation --------------------- There doesn't seem to be any documentation for Sysprof, but maybe that's -because it's pretty self-explanatory. The Sysprof website, however, is -here: `Sysprof, System-wide Performance Profiler for -Linux `__ +because it's pretty self-explanatory. The Sysprof website, however, is here: +`Sysprof, System-wide Performance Profiler for Linux `__ LTTng (Linux Trace Toolkit, next generation) ============================================ @@ -2029,7 +2026,7 @@ LTTng Setup For this section, we'll assume you've already performed the basic setup outlined in the ":ref:`profile-manual/intro:General Setup`" section. -LTTng is run on the target system by ssh'ing to it. +LTTng is run on the target system by connecting to it through SSH. Collecting and Viewing Traces ----------------------------- @@ -2042,7 +2039,7 @@ tracing. Collecting and viewing a trace on the target (inside a shell) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -First, from the host, ssh to the target:: +First, from the host, connect to the target through SSH:: $ ssh -l root 192.168.1.47 The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established. @@ -2134,11 +2131,11 @@ supplying your own name to ``lttng create``):: drwxr-xr-x 5 root root 1024 Oct 15 23:57 .. drwxrwx--- 3 root root 1024 Oct 15 23:21 auto-20121015-232120 -Collecting and viewing a userspace trace on the target (inside a shell) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Collecting and viewing a user space trace on the target (inside a shell) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -For LTTng userspace tracing, you need to have a properly instrumented -userspace program. For this example, we'll use the ``hello`` test program +For LTTng user space tracing, you need to have a properly instrumented +user space program. For this example, we'll use the ``hello`` test program generated by the ``lttng-ust`` build. The ``hello`` test program isn't installed on the root filesystem by the ``lttng-ust`` @@ -2154,7 +2151,7 @@ Copy that over to the target machine:: You now have the instrumented LTTng "hello world" test program on the target, ready to test. -First, from the host, ssh to the target:: +First, from the host, connect to the target through SSH:: $ ssh -l root 192.168.1.47 The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established. @@ -2169,7 +2166,7 @@ Once on the target, use these steps to create a trace:: Session auto-20190303-021943 created. Traces will be written in /home/root/lttng-traces/auto-20190303-021943 -Enable the events you want to trace (in this case all userspace events):: +Enable the events you want to trace (in this case all user space events):: root@crownbay:~# lttng enable-event --userspace --all All UST events are enabled in channel channel0 @@ -2241,13 +2238,13 @@ the entire blktrace and blkparse pipeline on the target, or you can run blktrace in 'listen' mode on the target and have blktrace and blkparse collect and analyze the data on the host (see the ":ref:`profile-manual/usage:Using blktrace Remotely`" section -below). For the rest of this section we assume you've ssh'ed to the host and -will be running blkrace on the target. +below). For the rest of this section we assume you've to the host through SSH +and will be running blktrace on the target. Basic blktrace Usage -------------------- -To record a trace, simply run the ``blktrace`` command, giving it the name +To record a trace, just run the ``blktrace`` command, giving it the name of the block device you want to trace activity on:: root@crownbay:~# blktrace /dev/sdc @@ -2258,10 +2255,10 @@ In another shell, execute a workload you want to trace:: Connecting to downloads.yoctoproject.org (140.211.169.59:80) linux-2.6.19.2.tar.b 100% \|*******************************\| 41727k 0:00:00 ETA -Press Ctrl-C in the blktrace shell to stop the trace. It +Press ``Ctrl-C`` in the blktrace shell to stop the trace. It will display how many events were logged, along with the per-cpu file -sizes (blktrace records traces in per-cpu kernel buffers and simply -dumps them to userspace for blkparse to merge and sort later):: +sizes (blktrace records traces in per-cpu kernel buffers and just +dumps them to user space for blkparse to merge and sort later):: ^C=== sdc === CPU 0: 7082 events, 332 KiB data @@ -2277,7 +2274,7 @@ with the device name as the first part of the filename:: -rw-r--r-- 1 root root 339938 Oct 27 22:40 sdc.blktrace.0 -rw-r--r-- 1 root root 75753 Oct 27 22:40 sdc.blktrace.1 -To view the trace events, simply invoke ``blkparse`` in the directory +To view the trace events, just call ``blkparse`` in the directory containing the trace files, giving it the device name that forms the first part of the filenames:: @@ -2376,8 +2373,8 @@ Live Mode ~~~~~~~~~ blktrace and blkparse are designed from the ground up to be able to -operate together in a "pipe mode" where the stdout of blktrace can be -fed directly into the stdin of blkparse:: +operate together in a "pipe mode" where the standard output of blktrace can be +fed directly into the standard input of blkparse:: root@crownbay:~# blktrace /dev/sdc -o - | blkparse -i - @@ -2446,7 +2443,7 @@ just ended:: Total: 11800 events (dropped 0), 554 KiB data The blktrace instance on the host will -save the target output inside a hostname-timestamp directory:: +save the target output inside a ``-`` directory:: $ ls -al drwxr-xr-x 10 root root 1024 Oct 28 02:40 . @@ -2518,7 +2515,7 @@ Tracing Block I/O via 'ftrace' It's also possible to trace block I/O using only :ref:`profile-manual/usage:The 'trace events' Subsystem`, which can be useful for casual tracing if you don't want to bother dealing with the -userspace tools. +user space tools. To enable tracing for a given device, use ``/sys/block/xxx/trace/enable``, where ``xxx`` is the device name. This for example enables tracing for @@ -2576,7 +2573,7 @@ section can be found here: - https://linux.die.net/man/8/btrace The above manual pages, along with manuals for the other blktrace utilities -(btt, blkiomon, etc) can be found in the ``/doc`` directory of the blktrace -tools git repo:: +(``btt``, ``blkiomon``, etc) can be found in the ``/doc`` directory of the blktrace +tools git repository:: $ git clone git://git.kernel.dk/blktrace.git diff --git a/documentation/styles/config/vocabularies/OpenSource/accept.txt b/documentation/styles/config/vocabularies/OpenSource/accept.txt index 98e76ae1f5..e378fbf79b 100644 --- a/documentation/styles/config/vocabularies/OpenSource/accept.txt +++ b/documentation/styles/config/vocabularies/OpenSource/accept.txt @@ -1,4 +1,20 @@ +autovivification +blkparse +blktrace +callee +debugfs ftrace -toolchain -systemd +KernelShark +Kprobe LTTng +perf +profiler +subcommand +subnode +superset +Sysprof +systemd +toolchain +tracepoint +Uprobe +wget diff --git a/documentation/styles/config/vocabularies/Yocto/accept.txt b/documentation/styles/config/vocabularies/Yocto/accept.txt index b725414014..ca622ba412 100644 --- a/documentation/styles/config/vocabularies/Yocto/accept.txt +++ b/documentation/styles/config/vocabularies/Yocto/accept.txt @@ -1,4 +1,5 @@ -Yocto -BSP BitBake +BSP +crosstap OpenEmbedded +Yocto From patchwork Mon Mar 25 13:27:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC5F0C54E58 for ; Mon, 25 Mar 2024 13:27:51 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx.groups.io with SMTP id smtpd.web11.52061.1711373270606563331 for ; Mon, 25 Mar 2024 06:27:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=GQZQczHq; spf=pass (domain: bootlin.com, ip: 217.70.183.197, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id BAC171C0004; Mon, 25 Mar 2024 13:27:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373268; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JrxlxbiEYssaVIVHVRX66cFaIhR5yEoqTvAJAVnpqJE=; b=GQZQczHqR5qeUrAPEjP+BeI2p1pCdlEglRHVThc3XjnJwzvXqn+QJmd1VZz4ag5rSFH/DQ v/ZJvS/GfekBQO7MW8ZBUz2wMuhbhXnu8p3QlMHIyobTJfCHW/el4HxQHJOdoQgQ5vHE7b 2aV5xkch1vwl/G8ipXni/rLkhSXs9k4mTq+v4Ejuas/0cQw3jwZ08uiIK9EM7iB/+VYo4g 3wNTJeEHV97RZBxe8KMeexb83tA4bhXCZOHDJkNywvtgTwXgGUmPuGOIOpnTwL90L4eX5a txe9JlAMr0LiFEJjzQMEZGY4WXzyVvnAOHeTUNHKL8H8obbh6jeUwgtEq/SUVg== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , BELOUARGA Mohamed Subject: [kirkstone][PATCH 8/9] ref-manual: add documentation of the variable SPDX_NAMESPACE_PREFIX Date: Mon, 25 Mar 2024 14:27:36 +0100 Message-Id: <20240325132737.3018073-9-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:51 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5006 From: Michael Opdenacker From: BELOUARGA Mohamed The documentation of the variable SPDX_NAMESPACE_PREFIX does not exist. This variable is used to change the prefix of some links in SPDX docs. Signed-off-by: BELOUARGA Mohamed Reviewed-by: Michael Opdenacker --- documentation/ref-manual/variables.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index aa1d5f393d..5e5b688878 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -7521,6 +7521,11 @@ system and gives an overview of their function and contents. image), compared to just using the :ref:`create-spdx ` class with no option. + :term:`SPDX_NAMESPACE_PREFIX` + This option could be used in order to change the prefix of ``spdxDocument`` + and the prefix of ``documentNamespace``. It is set by default to + ``http://spdx.org/spdxdoc``. + :term:`SPDX_PRETTY` This option makes the SPDX output more human-readable, using identation and newlines, instead of the default output in a From patchwork Mon Mar 25 13:27:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Opdenacker X-Patchwork-Id: 41452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD565C54E64 for ; Mon, 25 Mar 2024 13:27:51 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by mx.groups.io with SMTP id smtpd.web10.51978.1711373270623459568 for ; Mon, 25 Mar 2024 06:27:51 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=IywqdUjy; spf=pass (domain: bootlin.com, ip: 217.70.183.195, mailfrom: michael.opdenacker@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 2835360004; Mon, 25 Mar 2024 13:27:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1711373269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nEQh6ZQiSj81aC88Q/pZVh/CUwHNLwS9if6Qjepw4ys=; b=IywqdUjyodC4Mjz4rAPqvnrHdSeU/Vu+FnhCBTt2cxnBe/OnyHHMwGTevv4VkZNJqtedMv 9oX8vw+OVN6bO+Cu45oz/lNNvJvyfFzKSk5DIpx/WyavYSIxgODQal+ZKEDTYJnAPfsEhb DtjkkTl6un8WYMseIOx811jQxEDeb0aqMxKMP9C44Eh3G3Q6Ytai3CXH5jvt5AXT3JvArl XqraAwozQIgJmjgCFL8kuraxbKP+vUU/E3lYHDqOjA/ArlJYARISZie6LiuQzSidUp2KE5 9x9hBV+1hf+2f0S2QVBHcXMJdnlIF4NeTHNKjCvcWp1HYZZd84fGv1EceSgzZA== From: michael.opdenacker@bootlin.com To: docs@lists.yoctoproject.org Cc: Michael Opdenacker , Alexander Kanavin Subject: [kirkstone][PATCH 9/9] dev-manual: improve descriptions of 'bitbake -S printdiff' Date: Mon, 25 Mar 2024 14:27:37 +0100 Message-Id: <20240325132737.3018073-10-michael.opdenacker@bootlin.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> References: <20240325132737.3018073-1-michael.opdenacker@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: michael.opdenacker@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Mar 2024 13:27:51 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/docs/message/5007 From: Michael Opdenacker From: Alexander Kanavin Try to particularly emphasize that it can be used to find out why something rebuilds when it shouldn't. Signed-off-by: Alexander Kanavin Reviewed-by: Michael Opdenacker --- documentation/dev-manual/building.rst | 7 +++---- documentation/dev-manual/debugging.rst | 13 ++++++++++--- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/documentation/dev-manual/building.rst b/documentation/dev-manual/building.rst index 7fcac33b75..fe502690dd 100644 --- a/documentation/dev-manual/building.rst +++ b/documentation/dev-manual/building.rst @@ -775,10 +775,9 @@ your tunings to best consider build times and package feed maintenance. in the script for information on how to use the tool. - *BitBake's "-S printdiff" Option:* Using this option causes - BitBake to try to establish the closest signature match it can - (e.g. in the shared state cache) and then run ``bitbake-diffsigs`` - over the matches to determine the stamps and delta where these two - stamp trees diverge. + BitBake to try to establish the most recent signature match + (e.g. in the shared state cache) and then compare matched signatures + to determine the stamps and delta where these two stamp trees diverge. Building Software from an External Source ========================================= diff --git a/documentation/dev-manual/debugging.rst b/documentation/dev-manual/debugging.rst index 71b5807f5a..74f5772554 100644 --- a/documentation/dev-manual/debugging.rst +++ b/documentation/dev-manual/debugging.rst @@ -339,7 +339,10 @@ BitBake has determined by doing the following: :term:`BB_BASEHASH_IGNORE_VARS` information. -There is also a ``bitbake-diffsigs`` command for comparing two +Debugging signature construction and unexpected task executions +=============================================================== + +There is a ``bitbake-diffsigs`` command for comparing two ``siginfo`` or ``sigdata`` files. This command can be helpful when trying to figure out what changed between two versions of a task. If you call ``bitbake-diffsigs`` with just one file, the command behaves like @@ -356,8 +359,12 @@ BitBake command-line options:: .. note:: Two common values for `SIGNATURE_HANDLER` are "none" and "printdiff", which - dump only the signature or compare the dumped signature with the cached one, - respectively. + dump only the signature or compare the dumped signature with the most recent one, + respectively. "printdiff" will try to establish the most recent + signature match (e.g. in the sstate cache) and then + compare the matched signatures to determine the stamps and delta + where these two stamp trees diverge. This can be used to determine why + tasks need to be re-run in situations where that is not expected. Using BitBake with either of these options causes BitBake to dump out ``sigdata`` files in the ``stamps`` directory for every task it would