diff mbox series

[1/6] overlayfs-etc: mount etc as overlayfs

Message ID 83e41a0b7b5e1dc41fa89ffb4d3a8105299f7b1c.1637301667.git.uvv.mail@gmail.com
State Accepted, archived
Commit 610ea808c8b5edb2826bda1f1c42a811bd4ba758
Headers show
Series [1/6] overlayfs-etc: mount etc as overlayfs | expand

Commit Message

Vyacheslav Yurkov Nov. 19, 2021, 6:15 a.m. UTC 
This class provides an image feature that mounts /etc as an overlayfs
file system. This is an extension for existing overlayfs class, which
doesn't support /etc

Signed-off-by: Alfred Schapansky <alfred.schapansky@avantys.de>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/classes/overlayfs-etc.bbclass | 93 ++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)
 create mode 100644 meta/classes/overlayfs-etc.bbclass

Comments

Ross Burton Dec. 9, 2021, 10:59 a.m. UTC | #1 
On Fri, 19 Nov 2021 at 06:15, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
> +# Class for setting up /etc in overlayfs
> +#
> +# In order to have /etc directory in overlayfs a special handling at early boot stage is required
> +# The idea is to supply a custom init script that mounts /etc before launching actual init program,
> +# because the latter already requires /etc to be mounted

Can you elaborate on the exact scenario that is being supported here?
Is it read-only / with /etc (and potentially more) in an overlayfs?

> +OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
> +OVERLAYFS_ETC_FSTYPE ?= "ext4"
> +OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"

I'm guessing these are not universal defaults, but the values that you use.

If you can't provide default values which will actually work out of
the box, just set them to "" and the class can abort if they're not
set.

Ross
Richard Purdie Dec. 9, 2021, 11:08 a.m. UTC | #2 
On Fri, 2021-11-19 at 07:15 +0100, Vyacheslav Yurkov wrote:
> This class provides an image feature that mounts /etc as an overlayfs
> file system. This is an extension for existing overlayfs class, which
> doesn't support /etc
> 
> Signed-off-by: Alfred Schapansky <alfred.schapansky@avantys.de>
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
>  meta/classes/overlayfs-etc.bbclass | 93 ++++++++++++++++++++++++++++++
>  1 file changed, 93 insertions(+)
>  create mode 100644 meta/classes/overlayfs-etc.bbclass
> 
> diff --git a/meta/classes/overlayfs-etc.bbclass b/meta/classes/overlayfs-etc.bbclass
> new file mode 100644
> index 0000000000..78caf0211b
> --- /dev/null
> +++ b/meta/classes/overlayfs-etc.bbclass
> @@ -0,0 +1,93 @@
> +# Class for setting up /etc in overlayfs
> +#
> +# In order to have /etc directory in overlayfs a special handling at early boot stage is required
> +# The idea is to supply a custom init script that mounts /etc before launching actual init program,
> +# because the latter already requires /etc to be mounted
> +#
> +# The configuration must be machine specific. You should at least set these two variables if you
> +# are not happy with default values:
> +#   OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
> +#   OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
> +#
> +# To control more mount options you should consider also setting file system type and mount options:
> +#   OVERLAYFS_ETC_FSTYPE ?= "ext4"
> +#   OVERLAYFS_ETC_MOUNT_OPTIONS ?= "defaults"
> +#
> +# The class provides two options for /sbin/init generation
> +# 1. Default option is to rename original /sbin/init to /sbin/init.orig and place generated init under
> +#    original name, i.e. /sbin/init. It has an advantage that you won't need to change any kernel
> +#    parameters in order to make it work, but it poses a restriction that package-management can't
> +#    be used, becaause updating init manager would remove generated script
> +# 2. If you are would like to keep original init as is, you can set
> +#    OVERLAYFS_ETC_USE_ORIG_INIT_NAME = "0"
> +#    Then generated init will be named /sbin/preinit and you would need to extend you kernel parameters
> +#    manually in your bootloader configuration.
> +#
> +# Regardless which mode you choose, update and migration strategy of configuration files under /etc
> +# overlay is out of scope of this class
> +
> +ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "overlayfs-etc", "create_overlayfs_etc_preinit;", "", d)}'
> +IMAGE_FEATURES_CONFLICTS_overlayfs-etc = "package-management"
> +
> +OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
> +OVERLAYFS_ETC_FSTYPE ?= "ext4"
> +OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
> +OVERLAYFS_ETC_USE_ORIG_INIT_NAME ?= "1"
> +OVERLAYFS_ETC_MOUNT_OPTIONS ?= "defaults"
> +
> +python create_overlayfs_etc_preinit() {
> +    PreinitTemplate = """#!/bin/sh
> +
> +echo "PREINIT: Start"
> +
> +PATH=/sbin:/bin:/usr/sbin:/usr/bin
> +mount -o remount,rw /
> +
> +mkdir -p /proc
> +mkdir -p /sys
> +mkdir -p /run

Sorry about the delay replying on this. Something has been bothering me about
the code and I haven't been able to articulate it. I think Ross is right about
part of it, the defaults are very device specific and need to be invalid by
default which checks to ensure they're set.

The second thing which I don't really like is this mix of shell and python, it
is very confusing to read. I'd much rather we put template files into
meta/lib/files/ and then read them from there for the variable substitutions. We
do this for other scripts. I appreciate the overlayfs class merged without that
but I think we need to fix this there as well.

Adding the tests in this series is really great through, thanks for that!

Cheers,

Richard
Vyacheslav Yurkov Dec. 9, 2021, 3:13 p.m. UTC | #3 
On 09.12.2021 11:59, Ross Burton wrote:
> On Fri, 19 Nov 2021 at 06:15, Vyacheslav Yurkov <uvv.mail@gmail.com> wrote:
>> +# Class for setting up /etc in overlayfs
>> +#
>> +# In order to have /etc directory in overlayfs a special handling at early boot stage is required
>> +# The idea is to supply a custom init script that mounts /etc before launching actual init program,
>> +# because the latter already requires /etc to be mounted
> Can you elaborate on the exact scenario that is being supported here?
> Is it read-only / with /etc (and potentially more) in an overlayfs?

Correct, that's a read-only / with only /etc in an overlayfs. For other 
directories (not under /etc) there's a systemd based solution in 
overlayfs.bbclass.

>> +OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
>> +OVERLAYFS_ETC_FSTYPE ?= "ext4"
>> +OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
> I'm guessing these are not universal defaults, but the values that you use.
>
> If you can't provide default values which will actually work out of
> the box, just set them to "" and the class can abort if they're not
> set.
>
> Ross

Thanks for the feedback. Will fix that.

Vyacheslav
Vyacheslav Yurkov Dec. 9, 2021, 3:19 p.m. UTC | #4 
On 09.12.2021 12:08, Richard Purdie wrote:
> The second thing which I don't really like is this mix of shell and python, it
> is very confusing to read. I'd much rather we put template files into
> meta/lib/files/ and then read them from there for the variable substitutions. We
> do this for other scripts. I appreciate the overlayfs class merged without that
> but I think we need to fix this there as well.

Thanks for the feedback, Richard.
I see only meta/files. I guess you meant this directory.

Will do the same for overlayfs.bbclass too.

> Adding the tests in this series is really great through, thanks for that!
>
> Cheers,
>
> Richard

Regards,
Vyacheslav
Richard Purdie Dec. 9, 2021, 5:01 p.m. UTC | #5 
On Thu, 2021-12-09 at 16:19 +0100, Vyacheslav Yurkov wrote:
> On 09.12.2021 12:08, Richard Purdie wrote:
> > The second thing which I don't really like is this mix of shell and python, it
> > is very confusing to read. I'd much rather we put template files into
> > meta/lib/files/ and then read them from there for the variable substitutions. We
> > do this for other scripts. I appreciate the overlayfs class merged without that
> > but I think we need to fix this there as well.
> 
> Thanks for the feedback, Richard.
> I see only meta/files. I guess you meant this directory.

I did, yes. Not sure where the lib came from! :)

> Will do the same for overlayfs.bbclass too.

Thanks!

Cheers,

Richard
diff mbox series

Patch

diff --git a/meta/classes/overlayfs-etc.bbclass b/meta/classes/overlayfs-etc.bbclass
new file mode 100644
index 0000000000..78caf0211b
--- /dev/null
+++ b/meta/classes/overlayfs-etc.bbclass
@@ -0,0 +1,93 @@ 
+# Class for setting up /etc in overlayfs
+#
+# In order to have /etc directory in overlayfs a special handling at early boot stage is required
+# The idea is to supply a custom init script that mounts /etc before launching actual init program,
+# because the latter already requires /etc to be mounted
+#
+# The configuration must be machine specific. You should at least set these two variables if you
+# are not happy with default values:
+#   OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
+#   OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
+#
+# To control more mount options you should consider also setting file system type and mount options:
+#   OVERLAYFS_ETC_FSTYPE ?= "ext4"
+#   OVERLAYFS_ETC_MOUNT_OPTIONS ?= "defaults"
+#
+# The class provides two options for /sbin/init generation
+# 1. Default option is to rename original /sbin/init to /sbin/init.orig and place generated init under
+#    original name, i.e. /sbin/init. It has an advantage that you won't need to change any kernel
+#    parameters in order to make it work, but it poses a restriction that package-management can't
+#    be used, becaause updating init manager would remove generated script
+# 2. If you are would like to keep original init as is, you can set
+#    OVERLAYFS_ETC_USE_ORIG_INIT_NAME = "0"
+#    Then generated init will be named /sbin/preinit and you would need to extend you kernel parameters
+#    manually in your bootloader configuration.
+#
+# Regardless which mode you choose, update and migration strategy of configuration files under /etc
+# overlay is out of scope of this class
+
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "overlayfs-etc", "create_overlayfs_etc_preinit;", "", d)}'
+IMAGE_FEATURES_CONFLICTS_overlayfs-etc = "package-management"
+
+OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
+OVERLAYFS_ETC_FSTYPE ?= "ext4"
+OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
+OVERLAYFS_ETC_USE_ORIG_INIT_NAME ?= "1"
+OVERLAYFS_ETC_MOUNT_OPTIONS ?= "defaults"
+
+python create_overlayfs_etc_preinit() {
+    PreinitTemplate = """#!/bin/sh
+
+echo "PREINIT: Start"
+
+PATH=/sbin:/bin:/usr/sbin:/usr/bin
+mount -o remount,rw /
+
+mkdir -p /proc
+mkdir -p /sys
+mkdir -p /run
+mkdir -p /var/run
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+
+[ -z "$CONSOLE" ] && CONSOLE="/dev/console"
+
+mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}
+if mount -n -t {OVERLAYFS_ETC_FSTYPE} -o {OVERLAYFS_ETC_MOUNT_OPTIONS} {OVERLAYFS_ETC_DEVICE} {OVERLAYFS_ETC_MOUNT_POINT}
+then
+    mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper
+    mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/work
+    mount -n -t overlay -o upperdir={OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper,lowerdir=/etc,workdir={OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/work {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper /etc || echo "PREINIT: Mounting etc-overlay failed!"
+else
+    echo "PREINIT: Mounting </data> failed!"
+fi
+
+echo "PREINIT: done; starting </sbin/init>"
+exec {SBIN_INIT_NAME}
+"""
+
+    useOrigInit = oe.types.boolean(d.getVar('OVERLAYFS_ETC_USE_ORIG_INIT_NAME'))
+    preinitPath = oe.path.join(d.getVar("IMAGE_ROOTFS"), d.getVar("base_sbindir"), "preinit")
+    initBaseName = oe.path.join(d.getVar("base_sbindir"), "init")
+    origInitNameSuffix = ".orig"
+
+    args = {
+        'OVERLAYFS_ETC_MOUNT_POINT': d.getVar('OVERLAYFS_ETC_MOUNT_POINT'),
+        'OVERLAYFS_ETC_MOUNT_OPTIONS': d.getVar('OVERLAYFS_ETC_MOUNT_OPTIONS'),
+        'OVERLAYFS_ETC_FSTYPE': d.getVar('OVERLAYFS_ETC_FSTYPE'),
+        'OVERLAYFS_ETC_DEVICE': d.getVar('OVERLAYFS_ETC_DEVICE'),
+        'SBIN_INIT_NAME': initBaseName + origInitNameSuffix if useOrigInit else initBaseName
+    }
+
+    if useOrigInit:
+        # rename original /sbin/init
+        origInit = oe.path.join(d.getVar("IMAGE_ROOTFS"), initBaseName)
+        bb.debug(1, "rootfs path %s, init path %s, test %s" % (d.getVar('IMAGE_ROOTFS'), origInit, d.getVar("IMAGE_ROOTFS")))
+        bb.utils.rename(origInit, origInit + origInitNameSuffix)
+        preinitPath = origInit
+
+    with open(preinitPath, 'w') as f:
+        f.write(PreinitTemplate.format(**args))
+    os.chmod(preinitPath, 0o755)
+}