diff mbox series

gstreamer1.0-plugins-bad: handle padded buffers in wl_shm buffer creation

Message ID 20260430114715.990400-1-tdarote@qti.qualcomm.com
State New
Headers show
Series gstreamer1.0-plugins-bad: handle padded buffers in wl_shm buffer creation | expand

Commit Message

Tushar Darote April 30, 2026, 11:47 a.m. UTC 
Use GstBuffer metadata when constructing wl_shm buffers so stride and size reflect the actual buffer layout.
This fixes incorrect rendering with DMA-backed buffers that include padding or custom strides.

Signed-off-by: Tushar Darote <tdarote@qti.qualcomm.com>
---
 ...added-buffers-in-wl_shm-buffer-creat.patch | 178 ++++++++++++++++++
 .../gstreamer1.0-plugins-bad_1.28.2.bb        |   1 +
 2 files changed, 179 insertions(+)
 create mode 100644 meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch

Comments

patchtest@automation.yoctoproject.org April 30, 2026, 1 p.m. UTC | #1 
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/gstreamer1.0-plugins-bad-handle-padded-buffers-in-wl_shm-buffer-creation.patch

FAIL: test Signed-off-by presence: A patch file has been added without a Signed-off-by tag: '0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch' (test_patch.TestPatch.test_signed_off_by_presence)

PASS: test CVE tag format (test_patch.TestPatch.test_cve_tag_format)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test Upstream-Status presence (test_patch.TestPatch.test_upstream_status_presence_format)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
diff mbox series

Patch

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch
new file mode 100644
index 0000000000..949da1b7e4
--- /dev/null
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch
@@ -0,0 +1,178 @@ 
+From f8bc21dae1056be483645594ea79131888d8b1ae Mon Sep 17 00:00:00 2001
+From: Tushar Darote <tdarote@qti.qualcomm.com>
+Date: Thu, 30 Apr 2026 16:49:57 +0530
+Subject: [PATCH] wayland: handle padded buffers in wl_shm buffer creation
+
+Use GstBuffer metadata when constructing wl_shm buffers so stride and size reflect the actual buffer layout.
+This fixes incorrect rendering with DMA-backed buffers that include padding or custom strides.
+
+Upstream-Status: Submitted [https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/11370]
+---
+ ext/gtk/gstgtkwaylandsink.c              |  4 +-
+ ext/wayland/gstwaylandsink.c             |  4 +-
+ gst-libs/gst/wayland/gstwlshmallocator.c | 57 +++++++++++++++++-------
+ gst-libs/gst/wayland/gstwlshmallocator.h |  2 +-
+ gst-libs/gst/wayland/gstwlwindow.c       |  2 +-
+ 5 files changed, 48 insertions(+), 21 deletions(-)
+
+diff --git a/ext/gtk/gstgtkwaylandsink.c b/ext/gtk/gstgtkwaylandsink.c
+index 0660b30..6d1adce 100644
+--- a/ext/gtk/gstgtkwaylandsink.c
++++ b/ext/gtk/gstgtkwaylandsink.c
+@@ -1302,7 +1302,7 @@ handle_shm:
+   if (!wbuf && gst_wl_display_check_format_for_shm (priv->display,
+           &priv->render_info)) {
+     if (gst_buffer_n_memory (buffer) == 1 && gst_is_fd_memory (mem))
+-      wbuf = gst_wl_shm_memory_construct_wl_buffer (mem, priv->display,
++      wbuf = gst_wl_shm_memory_construct_wl_buffer (buffer, priv->display,
+           &priv->render_info);
+ 
+     /* If nothing worked, copy into our internal pool */
+@@ -1329,7 +1329,7 @@ handle_shm:
+       /* attach a wl_buffer if there isn't one yet */
+       if (G_UNLIKELY (!wlbuffer)) {
+         mem = gst_buffer_peek_memory (to_render, 0);
+-        wbuf = gst_wl_shm_memory_construct_wl_buffer (mem, priv->display,
++        wbuf = gst_wl_shm_memory_construct_wl_buffer (to_render, priv->display,
+             &priv->render_info);
+ 
+         if (G_UNLIKELY (!wbuf))
+diff --git a/ext/wayland/gstwaylandsink.c b/ext/wayland/gstwaylandsink.c
+index 443db24..2490b6e 100644
+--- a/ext/wayland/gstwaylandsink.c
++++ b/ext/wayland/gstwaylandsink.c
+@@ -1148,7 +1148,7 @@ handle_shm:
+   if (!wbuf && gst_wl_display_check_format_for_shm (self->display,
+           &self->render_info)) {
+     if (gst_buffer_n_memory (buffer) == 1 && gst_is_fd_memory (mem))
+-      wbuf = gst_wl_shm_memory_construct_wl_buffer (mem, self->display,
++      wbuf = gst_wl_shm_memory_construct_wl_buffer (buffer, self->display,
+           &self->video_info);
+ 
+     /* If nothing worked, copy into our internal pool */
+@@ -1174,7 +1174,7 @@ handle_shm:
+       /* attach a wl_buffer if there isn't one yet */
+       if (G_UNLIKELY (!wlbuffer)) {
+         mem = gst_buffer_peek_memory (to_render, 0);
+-        wbuf = gst_wl_shm_memory_construct_wl_buffer (mem, self->display,
++        wbuf = gst_wl_shm_memory_construct_wl_buffer (to_render, self->display,
+             &self->video_info);
+ 
+         if (G_UNLIKELY (!wbuf))
+diff --git a/gst-libs/gst/wayland/gstwlshmallocator.c b/gst-libs/gst/wayland/gstwlshmallocator.c
+index 671f2b7..4452b96 100644
+--- a/gst-libs/gst/wayland/gstwlshmallocator.c
++++ b/gst-libs/gst/wayland/gstwlshmallocator.c
+@@ -76,32 +76,58 @@ gst_wl_shm_validate_video_info (const GstVideoInfo * vinfo)
+ }
+ 
+ struct wl_buffer *
+-gst_wl_shm_memory_construct_wl_buffer (GstMemory * mem, GstWlDisplay * display,
++gst_wl_shm_memory_construct_wl_buffer (GstBuffer * buf, GstWlDisplay * display,
+     const GstVideoInfo * info)
+ {
++  GstMemory *mem;
++  GstVideoMeta *vmeta;
++  GstVideoInfo vinfo;
+   gint width, height, stride;
+-  gsize offset, size, memsize, maxsize;
++  gsize plane_offset = 0;
++  gsize mem_offset = 0;
++  gsize size, memsize, maxsize;
+   enum wl_shm_format format;
+   struct wl_shm_pool *wl_pool;
+   struct wl_buffer *wbuffer;
+ 
+-  if (!gst_wl_shm_validate_video_info (info)) {
+-    GST_DEBUG_OBJECT (display, "Unsupported strides and offsets.");
++  /* Ensure the buffer has exactly one memory block */
++  if (gst_buffer_n_memory (buf) != 1)
++    return NULL;
++
++  /* Retrieve the memory and ensure it is FD backed */
++  mem = gst_buffer_peek_memory (buf, 0);
++  if (!gst_is_fd_memory (mem))
+     return NULL;
++
++  /* Start from caps-derived video info */
++  vinfo = *info;
++
++  /* Override layout using GstVideoMeta if present */
++  vmeta = gst_buffer_get_video_meta (buf);
++  if (vmeta) {
++    vinfo.width = vmeta->width;
++    vinfo.height = vmeta->height;
++    vinfo.stride[0] = vmeta->stride[0];
++    vinfo.offset[0] = vmeta->offset[0];
+   }
+ 
+-  width = GST_VIDEO_INFO_WIDTH (info);
+-  height = GST_VIDEO_INFO_HEIGHT (info);
+-  stride = GST_VIDEO_INFO_PLANE_STRIDE (info, 0);
+-  size = GST_VIDEO_INFO_SIZE (info);
+-  format = gst_video_format_to_wl_shm_format (GST_VIDEO_INFO_FORMAT (info));
++  /* Validate the final video layout */
++  if (!gst_wl_shm_validate_video_info (&vinfo)) {
++    GST_DEBUG_OBJECT (display, "Unsupported strides and offsets.");
++    return NULL;
++  }
+ 
+-  memsize = gst_memory_get_sizes (mem, &offset, &maxsize);
+-  offset += GST_VIDEO_INFO_PLANE_OFFSET (info, 0);
++  width = GST_VIDEO_INFO_WIDTH (&vinfo);
++  height = GST_VIDEO_INFO_HEIGHT (&vinfo);
++  stride = GST_VIDEO_INFO_PLANE_STRIDE (&vinfo, 0);
++  plane_offset = GST_VIDEO_INFO_PLANE_OFFSET (&vinfo, 0);
+ 
+-  g_return_val_if_fail (gst_is_fd_memory (mem), NULL);
++  /* wl_shm requires the actual memory footprint of the buffer */
++  size = plane_offset + (stride * height);
++  format = gst_video_format_to_wl_shm_format (GST_VIDEO_INFO_FORMAT (&vinfo));
++  memsize = gst_memory_get_sizes (mem, &mem_offset, &maxsize);
+   g_return_val_if_fail (size <= memsize, NULL);
+-  g_return_val_if_fail (gst_wl_display_check_format_for_shm (display, info),
++  g_return_val_if_fail (gst_wl_display_check_format_for_shm (display, &vinfo),
+       NULL);
+ 
+   GST_DEBUG_OBJECT (display, "Creating wl_buffer from SHM of size %"
+@@ -110,8 +136,9 @@ gst_wl_shm_memory_construct_wl_buffer (GstMemory * mem, GstWlDisplay * display,
+ 
+   wl_pool = wl_shm_create_pool (gst_wl_display_get_shm (display),
+       gst_fd_memory_get_fd (mem), memsize);
+-  wbuffer = wl_shm_pool_create_buffer (wl_pool, offset, width, height, stride,
+-      format);
++  wbuffer =
++      wl_shm_pool_create_buffer (wl_pool, mem_offset + plane_offset, width,
++      height, stride, format);
+   wl_shm_pool_destroy (wl_pool);
+ 
+   return wbuffer;
+diff --git a/gst-libs/gst/wayland/gstwlshmallocator.h b/gst-libs/gst/wayland/gstwlshmallocator.h
+index 4309bd7..38f3ba5 100644
+--- a/gst-libs/gst/wayland/gstwlshmallocator.h
++++ b/gst-libs/gst/wayland/gstwlshmallocator.h
+@@ -33,7 +33,7 @@ GST_WL_API
+ void gst_wl_shm_init_once (void);
+ 
+ GST_WL_API
+-struct wl_buffer * gst_wl_shm_memory_construct_wl_buffer (GstMemory * mem,
++struct wl_buffer * gst_wl_shm_memory_construct_wl_buffer (GstBuffer * buf,
+     GstWlDisplay * display, const GstVideoInfo * info);
+ 
+ G_END_DECLS
+diff --git a/gst-libs/gst/wayland/gstwlwindow.c b/gst-libs/gst/wayland/gstwlwindow.c
+index 3c5c3cc..b055755 100644
+--- a/gst-libs/gst/wayland/gstwlwindow.c
++++ b/gst-libs/gst/wayland/gstwlwindow.c
+@@ -986,7 +986,7 @@ gst_wl_window_update_borders (GstWlWindow * self)
+     gst_buffer_memset (buf, 0, 0, info.size);
+ 
+     wlbuf =
+-        gst_wl_shm_memory_construct_wl_buffer (gst_buffer_peek_memory (buf, 0),
++        gst_wl_shm_memory_construct_wl_buffer (buf,
+         priv->display, &info);
+ 
+     g_object_unref (alloc);
+-- 
+2.34.1
+
diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb
index cdf3a20dff..72f2cc7342 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_1.28.2.bb
@@ -9,6 +9,7 @@  SRC_URI = "https://gstreamer.freedesktop.org/src/gst-plugins-bad/gst-plugins-bad
            file://0001-fix-maybe-uninitialized-warnings-when-compiling-with.patch \
            file://0002-avoid-including-sys-poll.h-directly.patch \
            file://0004-opencv-resolve-missing-opencv-data-dir-in-yocto-buil.patch \
+           file://0005-wayland-handle-padded-buffers-in-wl_shm-buffer-creat.patch \
            "
 SRC_URI[sha256sum] = "6467e3964828f4d7d08bfe1fbb4d76287a1c8fa76674e59e101a149c020fefd7"