diff mbox series

classes: rootfs-postcommands: set better sane time to systemd

Message ID 20241019021935.2105739-1-gael.portay+rtone@gmail.com
State New
Headers show
Series classes: rootfs-postcommands: set better sane time to systemd | expand

Commit Message

Gaël PORTAY Oct. 19, 2024, 2:19 a.m. UTC 
When systemd is started, it sets the system clock to epoch to ensure the
system clock is reasonably initialized if no working RTC.

As init process, systemd sets epoch very early to the more recent
timestamp of[1]:
 - the build time of systemd (-Dtime-epoch)
 - the modification time ("mtime") of /var/lib/systemd/timesync/clock
   (systemd-timesyncd)
 - the modification time ("mtime") of /usr/lib/clock-epoch (systemd)

The first epoch timestamp is is hard-coded at build-time by the systemd
recipe (using either SOURCE_DATE_EPOCH, git-tag, or NEWS modification
time[2]).

The second epoch timestamp is maintained at run-time if the system runs
systemd-timesyncd.

This implements the third epoch timestamp at image build-time, from a
newly introduced rootfs post-process command that is very similar to
update_timestamp.

It touches the file /usr/lib/clock-epoch with the modification time set
to either the current timestamp (now) or to a reproducible timestamp if
REPRODUCIBLE_TIMESTAMP_ROOTFS is set.

Note: That newly introduced rootfs post-process command is automatically
set if systemd is set in the distro features.

[1]: https://github.com/systemd/systemd/commit/863098fdc9cd91e4f760085356ac02c4b7ba6df1
[2]: https://github.com/systemd/systemd/blob/v256/meson.build#L804-L825

Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
---
Hello,

With this patch, one can have a better sane default time set by systemd
very very very early in the boot (i.e. even before systemd outputs its
version!).

	# dmesg
	(...)
	[    3.347641] Run /sbin/init as init process
	[    3.350435]   with arguments:
	[    3.350440]     /sbin/init
	[    3.350443]   with environment:
	[    3.350447]     HOME=/
	[    3.350451]     TERM=linux
	[    3.431275] EXT4-fs (mmcblk2p2): re-mounted. Opts: (null)
	[    3.523997] EXT4-fs (mmcblk2p4): recovery complete
	[    3.527828] EXT4-fs (mmcblk2p4): mounted filesystem with ordered data mode. Opts: (null)
	[    3.694440] systemd[1]: System time before build time, advancing clock.
	[    3.714755] systemd[1]: systemd 250.5+ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK -SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid)

Thus, one can do time-based related things offline or before the system
clock is set by the network (if trusting the time does not matter that
much for sure).

This happens if the system has no hardware clock or if it is way too far
in the past and if using an old versions of systemd; systemd v250.5 sets
the system time to April 28, 2022 5:53:22 PM (1651168402).

Also, please set REPRODUCIBLE_TIMESTAMP_ROOTFS to any appropriate value
if enabling reproducible-builds (the default). Otherwise, bitbake sets
the timestamp to Friday, March 9, 2018 12:34:56 PM (1520598896).

If you have a better idea on how to implement it, please tell; in the
meanwhile...

Kind Regards,
Gaël
 meta/classes-recipe/rootfs-postcommands.bbclass | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Comments

Alexander Kanavin Oct. 21, 2024, 10:25 a.m. UTC | #1 
Can this be done from systemd recipe itself? Items in
rootfs-postprocess list should not be recipe-specific.

Alex

On Sat, 19 Oct 2024 at 04:19, Gaël PORTAY via lists.openembedded.org
<gael.portay+rtone=gmail.com@lists.openembedded.org> wrote:
>
> When systemd is started, it sets the system clock to epoch to ensure the
> system clock is reasonably initialized if no working RTC.
>
> As init process, systemd sets epoch very early to the more recent
> timestamp of[1]:
>  - the build time of systemd (-Dtime-epoch)
>  - the modification time ("mtime") of /var/lib/systemd/timesync/clock
>    (systemd-timesyncd)
>  - the modification time ("mtime") of /usr/lib/clock-epoch (systemd)
>
> The first epoch timestamp is is hard-coded at build-time by the systemd
> recipe (using either SOURCE_DATE_EPOCH, git-tag, or NEWS modification
> time[2]).
>
> The second epoch timestamp is maintained at run-time if the system runs
> systemd-timesyncd.
>
> This implements the third epoch timestamp at image build-time, from a
> newly introduced rootfs post-process command that is very similar to
> update_timestamp.
>
> It touches the file /usr/lib/clock-epoch with the modification time set
> to either the current timestamp (now) or to a reproducible timestamp if
> REPRODUCIBLE_TIMESTAMP_ROOTFS is set.
>
> Note: That newly introduced rootfs post-process command is automatically
> set if systemd is set in the distro features.
>
> [1]: https://github.com/systemd/systemd/commit/863098fdc9cd91e4f760085356ac02c4b7ba6df1
> [2]: https://github.com/systemd/systemd/blob/v256/meson.build#L804-L825
>
> Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
> ---
> Hello,
>
> With this patch, one can have a better sane default time set by systemd
> very very very early in the boot (i.e. even before systemd outputs its
> version!).
>
>         # dmesg
>         (...)
>         [    3.347641] Run /sbin/init as init process
>         [    3.350435]   with arguments:
>         [    3.350440]     /sbin/init
>         [    3.350443]   with environment:
>         [    3.350447]     HOME=/
>         [    3.350451]     TERM=linux
>         [    3.431275] EXT4-fs (mmcblk2p2): re-mounted. Opts: (null)
>         [    3.523997] EXT4-fs (mmcblk2p4): recovery complete
>         [    3.527828] EXT4-fs (mmcblk2p4): mounted filesystem with ordered data mode. Opts: (null)
>         [    3.694440] systemd[1]: System time before build time, advancing clock.
>         [    3.714755] systemd[1]: systemd 250.5+ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK -SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid)
>
> Thus, one can do time-based related things offline or before the system
> clock is set by the network (if trusting the time does not matter that
> much for sure).
>
> This happens if the system has no hardware clock or if it is way too far
> in the past and if using an old versions of systemd; systemd v250.5 sets
> the system time to April 28, 2022 5:53:22 PM (1651168402).
>
> Also, please set REPRODUCIBLE_TIMESTAMP_ROOTFS to any appropriate value
> if enabling reproducible-builds (the default). Otherwise, bitbake sets
> the timestamp to Friday, March 9, 2018 12:34:56 PM (1520598896).
>
> If you have a better idea on how to implement it, please tell; in the
> meanwhile...
>
> Kind Regards,
> Gaël
>  meta/classes-recipe/rootfs-postcommands.bbclass | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>
> diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
> index 5f4d67f93c..cc74c75cfa 100644
> --- a/meta/classes-recipe/rootfs-postcommands.bbclass
> +++ b/meta/classes-recipe/rootfs-postcommands.bbclass
> @@ -22,6 +22,9 @@ ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'deb
>  # Create /etc/timestamp during image construction to give a reasonably sane default time setting
>  ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp "
>
> +# Create /usr/lib/clock-epoch during image construction to give a reasonably sane default time setting to systemd
> +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "rootfs_systemd_timestamp ", "", d)}'
> +
>  # Tweak files in /etc if read-only-rootfs is enabled
>  ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "read_only_rootfs_hook ", "",d)}'
>
> @@ -402,6 +405,19 @@ rootfs_update_timestamp () {
>         bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
>  }
>
> +# Can be used to create /usr/lib/clock-epoch during image construction to give a reasonably
> +# sane default time setting
> +rootfs_systemd_timestamp () {
> +       if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
> +               # Convert UTC into %4Y%2m%2d%2H%2M.%2S
> +               sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M.%2S`
> +       else
> +               sformatted=`date -u +%4Y%2m%2d%2H%2M.%2S`
> +       fi
> +       touch -m -t "$sformatted" ${IMAGE_ROOTFS}/usr/lib/clock-epoch
> +       bbnote "rootfs_systemd_timestamp: set /usr/lib/clock-epoch mtime to $sformatted"
> +}
> +
>  # Prevent X from being started
>  rootfs_no_x_startup () {
>         if [ -f ${IMAGE_ROOTFS}/etc/init.d/xserver-nodm ]; then
> --
> 2.47.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#206078): https://lists.openembedded.org/g/openembedded-core/message/206078
> Mute This Topic: https://lists.openembedded.org/mt/109095282/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Gaël PORTAY Oct. 21, 2024, 5:02 p.m. UTC | #2 
Hello Alex,

On Mon Oct 21, 2024 at 12:25 PM CEST, Alexander Kanavin wrote:
> Can this be done from systemd recipe itself? Items in
> rootfs-postprocess list should not be recipe-specific.
>

Sure it could be done in the recipe itself ;) but...

The recipe harcodes already a sane value at configure time thanks to the
meson option -Dtime-epoch and the environment variable $SOURCE_DATE_EPOC
(or the date the creation of the latest git-tag or the modification time
of the NEWS file). systemd sets the date somewhere in 2022 (kirkstone).

I have created a rootfs-postcommand to be able to set a better time **at
image creation** to keep the systemd package untouched (not rebuilt) by
updateing the variable $REPRODUCIBLE_TIMESTAMP_ROOTFS.

That variable is suffixed by _ROOTFS; and I guess that variable **SHOULD
NOT** be used by package recipes.

The rootfs_systemd_timestamp() is based on rootfs_update_timestamp(); it
touches the file and it leaves the file empty.

Also, to decrease the systemd-specific thing, I wonder if this is
acceptable:

	rootfs_update_timestamp () {
	        if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
	                # Convert UTC into %4Y%2m%2d%2H%2M%2S
	                sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
	        else
	                sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
	        fi
	        echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
	        bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
	+
	+       if [ -x /lib/systemd/systemd ]; then
	+           if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
	+                   # Convert UTC into %4Y%2m%2d%2H%2M.%2S
	+                   sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M.%2S`
	+           else
	+                   sformatted=`date -u +%4Y%2m%2d%2H%2M.%2S`
	+           fi
	+           touch -m -t "$sformatted" ${IMAGE_ROOTFS}/usr/lib/clock-epoch
	+           bbnote "rootfs_systemd_timestamp: set /usr/lib/clock-epoch mtime to $sformatted"
	+       }
	}

Or, linking the file instead:

	rootfs_update_timestamp () {
	        if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
	                # Convert UTC into %4Y%2m%2d%2H%2M%2S
	                sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
	        else
	                sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
	        fi
	        echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
	        bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
	+
	+       if [ -x /lib/systemd/systemd ]; then
	+           ln -sf /etc/timestamp /usr/lib/clock-epoch
	+       }
	}

Does it look better for you?

> Alex
>

Regards,
Gaël
Alexander Kanavin Oct. 21, 2024, 5:09 p.m. UTC | #3 
On Mon, 21 Oct 2024 at 19:02, Gaël PORTAY <gael.portay@gmail.com> wrote:

> Or, linking the file instead:
>
>         rootfs_update_timestamp () {
>                 if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
>                         # Convert UTC into %4Y%2m%2d%2H%2M%2S
>                         sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
>                 else
>                         sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
>                 fi
>                 echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
>                 bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
>         +
>         +       if [ -x /lib/systemd/systemd ]; then
>         +           ln -sf /etc/timestamp /usr/lib/clock-epoch
>         +       }
>         }
>
> Does it look better for you?

This does look better, yes. Will systemd follow the symlink and read
the mtime of the original file?

Alex
Gaël PORTAY Oct. 21, 2024, 6:02 p.m. UTC | #4 
On Monday 21 October 2024, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
> On Mon, 21 Oct 2024 at 19:02, Gaël PORTAY <gael.portay@gmail.com> wrote:
>
>> Or, linking the file instead:
>>
>>         rootfs_update_timestamp () {
>>                 if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
>>                         # Convert UTC into %4Y%2m%2d%2H%2M%2S
>>                         sformatted=`date -u -d
@${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
>>                 else
>>                         sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
>>                 fi
>>                 echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
>>                 bbnote "rootfs_update_timestamp: set /etc/timestamp to
$sformatted"
>>         +
>>         +       if [ -x /lib/systemd/systemd ]; then
>>         +           ln -sf /etc/timestamp /usr/lib/clock-epoch
>>         +       }
>>         }
>>
>> Does it look better for you?
>
> This does look better, yes. Will systemd follow the symlink and read
> the mtime of the original file?
>
> Alex
>

I have not tested it so far, but I guess so.

systemd stat() the file, so it should follows symlink.

I will give a try tomorrow.

Also, I wonder what is the best test to symlink the file:
- testing for the ${IMAGE_ROOTFS}/llib/systemd/systemd
- testing for systemd in DISTRO_FEATURES,
- testing for systemd in IMAGE_INSTALL?

https://github.com/systemd/systemd/blob/4d47aa8c0435ad7bcd45aea3a091c22027fc0aa8/src/core/clock-warp.c#L26

Gaël
Alexander Kanavin Oct. 21, 2024, 6:06 p.m. UTC | #5 
On Mon, 21 Oct 2024 at 20:02, Gaël PORTAY <gael.portay+rtone@gmail.com> wrote:

> Also, I wonder what is the best test to symlink the file:
> - testing for the ${IMAGE_ROOTFS}/llib/systemd/systemd
> - testing for systemd in DISTRO_FEATURES,
> - testing for systemd in IMAGE_INSTALL?
>
> https://github.com/systemd/systemd/blob/4d47aa8c0435ad7bcd45aea3a091c22027fc0aa8/src/core/clock-warp.c#L26

Maybe test for nothing and simply always symlink, leaving a comment
just above the line that does it?

Alex
Peter Kjellerstedt Oct. 22, 2024, 6:40 p.m. UTC | #6 
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of "Gaël PORTAY
> Sent: den 21 oktober 2024 19:02
> To: Alexander Kanavin <alex.kanavin@gmail.com>;
> gael.portay+rtone@gmail.com
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] classes: rootfs-postcommands: set better
> sane time to systemd
> 
> Hello Alex,
> 
> On Mon Oct 21, 2024 at 12:25 PM CEST, Alexander Kanavin wrote:
> > Can this be done from systemd recipe itself? Items in
> > rootfs-postprocess list should not be recipe-specific.
> >
> 
> Sure it could be done in the recipe itself ;) but...
> 
> The recipe harcodes already a sane value at configure time thanks to the
> meson option -Dtime-epoch and the environment variable $SOURCE_DATE_EPOC
> (or the date the creation of the latest git-tag or the modification time
> of the NEWS file). systemd sets the date somewhere in 2022 (kirkstone).
> 
> I have created a rootfs-postcommand to be able to set a better time **at
> image creation** to keep the systemd package untouched (not rebuilt) by
> updateing the variable $REPRODUCIBLE_TIMESTAMP_ROOTFS.

Can't you add a pkg_postinst:${PN} function to the systemd recipe and that 
way do it in the recipe while stile executing it at image creation time?

> 
> That variable is suffixed by _ROOTFS; and I guess that variable **SHOULD
> NOT** be used by package recipes.
> 
> The rootfs_systemd_timestamp() is based on rootfs_update_timestamp(); it
> touches the file and it leaves the file empty.
> 
> Also, to decrease the systemd-specific thing, I wonder if this is
> acceptable:
> 
> 	rootfs_update_timestamp () {
> 	        if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
> 	                # Convert UTC into %4Y%2m%2d%2H%2M%2S
> 	                sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
> 	        else
> 	                sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
> 	        fi
> 	        echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
> 	        bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
> 	+
> 	+       if [ -x /lib/systemd/systemd ]; then

Umm, why are you looking at the host's files?

> 	+           if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
> 	+                   # Convert UTC into %4Y%2m%2d%2H%2M.%2S
> 	+                   sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M.%2S`
> 	+           else
> 	+                   sformatted=`date -u +%4Y%2m%2d%2H%2M.%2S`
> 	+           fi
> 	+           touch -m -t "$sformatted" ${IMAGE_ROOTFS}/usr/lib/clock-epoch
> 	+           bbnote "rootfs_systemd_timestamp: set /usr/lib/clock-epoch mtime to $sformatted"
> 	+       }
> 	}
> 
> Or, linking the file instead:
> 
> 	rootfs_update_timestamp () {
> 	        if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
> 	                # Convert UTC into %4Y%2m%2d%2H%2M%2S
> 	                sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S`
> 	        else
> 	                sformatted=`date -u +%4Y%2m%2d%2H%2M%2S`
> 	        fi
> 	        echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp
> 	        bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
> 	+
> 	+       if [ -x /lib/systemd/systemd ]; then
> 	+           ln -sf /etc/timestamp /usr/lib/clock-epoch

This too is trying to create a link in the host's filesystem.

> 	+       }
> 	}
> 
> Does it look better for you?
> 
> > Alex
> >
> 
> Regards,
> Gaël

//Peter
Gaël PORTAY Oct. 22, 2024, 11:13 p.m. UTC | #7 
Hello Peter,

On Tue Oct 22, 2024 at 8:40 PM CEST, Peter Kjellerstedt wrote:
> > I have created a rootfs-postcommand to be able to set a better time **at
> > image creation** to keep the systemd package untouched (not rebuilt) by
> > updateing the variable $REPRODUCIBLE_TIMESTAMP_ROOTFS.
>
> Can't you add a pkg_postinst:${PN} function to the systemd recipe and that 
> way do it in the recipe while stile executing it at image creation time?
>

Well, that is certainly doable indeed; I have not think about it. Very
good catch, I will give a try, thanks.

To be honest, my first intention was to reuse the existing bits (i.e.
rootfs_update_timestamp) that is there already, and I wanted to make
this existing bits working with systemd as well :)

---

I am openning a parenthesis...

As I said, the variable $REPRODUCIBLE_TIMESTAMP_ROOTFS is to be used for
rootfs (i.e. image) only, am I right?

But it is used to compile the kernel... see:
 - https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel.bbclass#n371
 - https://git.yoctoproject.org/poky/tree/meta/classes-recipe/kernel.bbclass#n427
The change was introduced a long time ago (2017-08-16 00:03:15 +0100):
 - https://git.yoctoproject.org/poky/commit/?id=55e9485735ae8393b410f30973c785236dc402d2

I think the kernel should (now) default to SOURCE_DATE_EPOCH_FALLBACK if
SOURCE_DATE_EPOCH cannot be deduced from the git repository.

That fallback variable was introduced afterward:
 - https://git.yoctoproject.org/poky/commit/?id=7e9c2f33d4ea9f6449dd56d19ff4522a9ddc2df1

But this is another story; parenthesis closed.

---

The worst thing is I figured out today my work does not work on master
branch; there must be changed since kirkstone that break it.

On kirkstone, stat() /usr/lib/clock-time at run-time returns same mtime
timestamp value than the one set in $REPRODUCIBLE_TIMESTAMP_ROOTFS.

On master, stat() /usr/lib/clock-time at run-time **DOES NOT** returns
the mtime timestamp value set in $REPRODUCIBLE_TIMESTAMP_ROOTFS.

It returns the values set in $SOURCE_DATE_EPOCH_FALLBACK instead! And
for every timestamp values, see:

        root@qemux86-64:~# stat /usr/lib/clock-epoch
          File: /clock-epoch
          Size: 0               Blocks: 0          IO Block: 1024   regular empty file
        Device: fd00h/64768d    Inode: 17          Links: 1
        Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
        Access: 2011-04-05 23:00:00.000000000 +0000
        Modify: 2011-04-05 23:00:00.000000000 +0000
        Change: 2011-04-05 23:00:00.000000000 +0000

The variable REPRODUCIBLE_TIMESTAMP_ROOTFS was set to "4000000000":

	root@qemux86-64:~# cat /etc/timestamp
	20961002070640

According to https://www.epochconverter.com, timestamp 4000000000 is
Tuesday, October 2, 2096 7:06:40 AM:

	Supports Unix timestamps in seconds, milliseconds, microseconds and nanoseconds.
	Assuming that this timestamp is in seconds:
	GMT: Tuesday, October 2, 2096 7:06:40 AM
	Your time zone: Tuesday, October 2, 2096 9:06:40 AM GMT+02:00 DST
	Relative: In 72 years

Therefore, the REPRODUCIBLE_TIMESTAMP_ROOTFS is set properly, but the
reproducible-builds resets the timestamp to $SOURCE_DATE_EPOCH_FALLBACK.

Is it the expected behaviour? I mean having the file timestamp created
before the binary is compiled (certainly yes, reproducible-builds) and
before the sources are actually released (maybe no)? or is it a bug :/

I have to **REDO** the test to ensure what I am telling is true for the
kirkstone branch; but I would not be able to do it before next week or
two (unless I found some free time to test it).

Note: IIRC, the files are created at the real date on kirkstone branch,
but I am not sure (i.e. do not use any reproducible-build variables).

> > 	+
> > 	+       if [ -x /lib/systemd/systemd ]; then
>
> Umm, why are you looking at the host's files?
>

Oops, thas was a just snippet written on-the-fly for the post; So yes I
forgot to prefix the files ${IMAGE_ROOTFS}.

> > 	+       if [ -x /lib/systemd/systemd ]; then
> > 	+           ln -sf /etc/timestamp /usr/lib/clock-epoch
>
> This too is trying to create a link in the host's filesystem.

Indeed.

>
> //Peter

Regards,
Gaël
ChenQi Oct. 23, 2024, 2:40 a.m. UTC | #8 
The timestamps of all files in rootfs will be changed again by 
reproducible_final_image_task. To achieve what you what, what you need 
to do is touching an empty file, $D/usr/lib/clock-epoch, in systemd's 
do_install task.

And when you create /usr/lib/clock-epoch at rootfs time and changes its 
mtime to the current time when REPRODUCIBLE_TIMESTAMP_ROOTFS is not set, 
its mtime will be set again by reproducible_final_image_task to be the 
commit time of the latest commit. This means the touch action in rootfs 
time is useless.

Regards,
Qi

On 10/19/24 10:19, Gaël PORTAY wrote:
> When systemd is started, it sets the system clock to epoch to ensure the
> system clock is reasonably initialized if no working RTC.
>
> As init process, systemd sets epoch very early to the more recent
> timestamp of[1]:
>   - the build time of systemd (-Dtime-epoch)
>   - the modification time ("mtime") of /var/lib/systemd/timesync/clock
>     (systemd-timesyncd)
>   - the modification time ("mtime") of /usr/lib/clock-epoch (systemd)
>
> The first epoch timestamp is is hard-coded at build-time by the systemd
> recipe (using either SOURCE_DATE_EPOCH, git-tag, or NEWS modification
> time[2]).
>
> The second epoch timestamp is maintained at run-time if the system runs
> systemd-timesyncd.
>
> This implements the third epoch timestamp at image build-time, from a
> newly introduced rootfs post-process command that is very similar to
> update_timestamp.
>
> It touches the file /usr/lib/clock-epoch with the modification time set
> to either the current timestamp (now) or to a reproducible timestamp if
> REPRODUCIBLE_TIMESTAMP_ROOTFS is set.
>
> Note: That newly introduced rootfs post-process command is automatically
> set if systemd is set in the distro features.
>
> [1]: https://github.com/systemd/systemd/commit/863098fdc9cd91e4f760085356ac02c4b7ba6df1
> [2]: https://github.com/systemd/systemd/blob/v256/meson.build#L804-L825
>
> Signed-off-by: Gaël PORTAY <gael.portay+rtone@gmail.com>
> ---
> Hello,
>
> With this patch, one can have a better sane default time set by systemd
> very very very early in the boot (i.e. even before systemd outputs its
> version!).
>
> 	# dmesg
> 	(...)
> 	[    3.347641] Run /sbin/init as init process
> 	[    3.350435]   with arguments:
> 	[    3.350440]     /sbin/init
> 	[    3.350443]   with environment:
> 	[    3.350447]     HOME=/
> 	[    3.350451]     TERM=linux
> 	[    3.431275] EXT4-fs (mmcblk2p2): re-mounted. Opts: (null)
> 	[    3.523997] EXT4-fs (mmcblk2p4): recovery complete
> 	[    3.527828] EXT4-fs (mmcblk2p4): mounted filesystem with ordered data mode. Opts: (null)
> 	[    3.694440] systemd[1]: System time before build time, advancing clock.
> 	[    3.714755] systemd[1]: systemd 250.5+ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK -SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid)
>
> Thus, one can do time-based related things offline or before the system
> clock is set by the network (if trusting the time does not matter that
> much for sure).
>
> This happens if the system has no hardware clock or if it is way too far
> in the past and if using an old versions of systemd; systemd v250.5 sets
> the system time to April 28, 2022 5:53:22 PM (1651168402).
>
> Also, please set REPRODUCIBLE_TIMESTAMP_ROOTFS to any appropriate value
> if enabling reproducible-builds (the default). Otherwise, bitbake sets
> the timestamp to Friday, March 9, 2018 12:34:56 PM (1520598896).
>
> If you have a better idea on how to implement it, please tell; in the
> meanwhile...
>
> Kind Regards,
> Gaël
>   meta/classes-recipe/rootfs-postcommands.bbclass | 16 ++++++++++++++++
>   1 file changed, 16 insertions(+)
>
> diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
> index 5f4d67f93c..cc74c75cfa 100644
> --- a/meta/classes-recipe/rootfs-postcommands.bbclass
> +++ b/meta/classes-recipe/rootfs-postcommands.bbclass
> @@ -22,6 +22,9 @@ ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'deb
>   # Create /etc/timestamp during image construction to give a reasonably sane default time setting
>   ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp "
>   
> +# Create /usr/lib/clock-epoch during image construction to give a reasonably sane default time setting to systemd
> +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "rootfs_systemd_timestamp ", "", d)}'
> +
>   # Tweak files in /etc if read-only-rootfs is enabled
>   ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "read_only_rootfs_hook ", "",d)}'
>   
> @@ -402,6 +405,19 @@ rootfs_update_timestamp () {
>   	bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
>   }
>   
> +# Can be used to create /usr/lib/clock-epoch during image construction to give a reasonably
> +# sane default time setting
> +rootfs_systemd_timestamp () {
> +	if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
> +		# Convert UTC into %4Y%2m%2d%2H%2M.%2S
> +		sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M.%2S`
> +	else
> +		sformatted=`date -u +%4Y%2m%2d%2H%2M.%2S`
> +	fi
> +	touch -m -t "$sformatted" ${IMAGE_ROOTFS}/usr/lib/clock-epoch
> +	bbnote "rootfs_systemd_timestamp: set /usr/lib/clock-epoch mtime to $sformatted"
> +}
> +
>   # Prevent X from being started
>   rootfs_no_x_startup () {
>   	if [ -f ${IMAGE_ROOTFS}/etc/init.d/xserver-nodm ]; then
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#206078): https://lists.openembedded.org/g/openembedded-core/message/206078
> Mute This Topic: https://lists.openembedded.org/mt/109095282/7304865
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Qi.Chen@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Gaël PORTAY Oct. 23, 2024, 5:23 a.m. UTC | #9 
On Wed Oct 23, 2024 at 4:40 AM CEST, ChenQi wrote:
> The timestamps of all files in rootfs will be changed again by 
> reproducible_final_image_task. To achieve what you what, what you need 
> to do is touching an empty file, $D/usr/lib/clock-epoch, in systemd's 
> do_install task.
>
> And when you create /usr/lib/clock-epoch at rootfs time and changes its 
> mtime to the current time when REPRODUCIBLE_TIMESTAMP_ROOTFS is not set, 
> its mtime will be set again by reproducible_final_image_task to be the 
> commit time of the latest commit. This means the touch action in rootfs 
> time is useless.
>

Thanks for the clarification, I will give a try.

> Regards,
> Qi
>

Regards,
Gaël
diff mbox series

Patch

diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
index 5f4d67f93c..cc74c75cfa 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -22,6 +22,9 @@  ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains_any("IMAGE_FEATURES", [ 'deb
 # Create /etc/timestamp during image construction to give a reasonably sane default time setting
 ROOTFS_POSTPROCESS_COMMAND += "rootfs_update_timestamp "
 
+# Create /usr/lib/clock-epoch during image construction to give a reasonably sane default time setting to systemd
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "rootfs_systemd_timestamp ", "", d)}'
+
 # Tweak files in /etc if read-only-rootfs is enabled
 ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", "read_only_rootfs_hook ", "",d)}'
 
@@ -402,6 +405,19 @@  rootfs_update_timestamp () {
 	bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted"
 }
 
+# Can be used to create /usr/lib/clock-epoch during image construction to give a reasonably
+# sane default time setting
+rootfs_systemd_timestamp () {
+	if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then
+		# Convert UTC into %4Y%2m%2d%2H%2M.%2S
+		sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M.%2S`
+	else
+		sformatted=`date -u +%4Y%2m%2d%2H%2M.%2S`
+	fi
+	touch -m -t "$sformatted" ${IMAGE_ROOTFS}/usr/lib/clock-epoch
+	bbnote "rootfs_systemd_timestamp: set /usr/lib/clock-epoch mtime to $sformatted"
+}
+
 # Prevent X from being started
 rootfs_no_x_startup () {
 	if [ -f ${IMAGE_ROOTFS}/etc/init.d/xserver-nodm ]; then