[2/3] contributor-guide: add section about why we use mailing lists

From: Michael Opdenacker <michael.opdenacker@bootlin.com>

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../contributor-guide/submit-change.rst       | 41 ++++++++++++++++++-
 1 file changed, 40 insertions(+), 1 deletion(-)

On Wed, 9 Aug 2023 at 16:25, Michael Opdenacker via lists.yoctoproject.org
<michael.opdenacker=bootlin.com@lists.yoctoproject.org> wrote:

> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  .../contributor-guide/submit-change.rst       | 41 ++++++++++++++++++-
>  1 file changed, 40 insertions(+), 1 deletion(-)
>
> diff --git a/documentation/contributor-guide/submit-change.rst
> b/documentation/contributor-guide/submit-change.rst
> index 2555767102..573491ecbc 100644
> --- a/documentation/contributor-guide/submit-change.rst
> +++ b/documentation/contributor-guide/submit-change.rst
> @@ -8,10 +8,49 @@ Because the system is extremely configurable and
> flexible, we recognize
>  that developers will want to extend, configure or optimize it for their
>  specific uses.
>
> +.. _ref-why-mailing-lists:
> +
> +Contributing through mailing lists --- Why not using web-based workflows?
> +=========================================================================
> +
> +Both Yocto Project and OpenEmbedded have many key components that are
> +maintained by patches being submitted on mailing lists. We appreciate this
> +approach does look a little old fashioned when other workflows are
> available
> +through web technology such as GitHub, GitLab and others. Since we are
> often
> +asked this question, we’ve decided to document the reasons for using
> mailing
> +lists.


Great to read those lines, I was one of those who asked this question ;)


>
>
+One significant factor is that we value peer review. When a change is
> proposed
> +to many of the core pieces of the project, it helps to have many eyes of
> review
> +go over them. Whilst there is ultimately one maintainer who needs to make
> the
> +final call on accepting or rejecting a patch, the review is made by many
> eyes
> +and the exact people reviewing it are likely unknown to the maintainer.
> It is
> +often the surprise reviewer that catches the most interesting issues!
> +
> +This is in contrast to the "GitHub" style workflow where either just a
> +maintainer makes that review, or review is specifically requested from
> +nominated people. We believe there is significant value added to the
> codebase
> +by this peer review and that moving away from mailing lists would be to
> the
> +detriment of our code.
>

For notifications, I don't see why the web flow would not allow to send
notifications
for changes to a mailing list (I think it can also be made per user profile
to receive
notifications for a whole project, an issue, a particular project ... etc)


> +We also need to acknowledge that many of our developers are used to this
> +mailing list workflow and have worked with it for years, with tools and
> +processes built around it. Changing away from this would result in a loss
> +of key people from the project, which would again be to its detriment.
>

I'd like to know some of these flow that are plugged on mail, can we insert
some
examples in the documentation ?


> +The projects are acutely aware that potential new contributors find the
> +mailing list approach off-putting and would prefer a point-and-click web
> GUI.
>

I find the term "point-and-click" a little patronizing,
github/gitlab/others web tools
are not just point-and-click machine but highly customizable tool aims at
smoothen
the developer work (in term of tracking, viewing diff, forking) and to
provide
enhanced user experience.


> +Since we don’t believe that can work for us, the project is aiming to
> ensure
> +`patchwork <https://patchwork.yoctoproject.org/>` is available to help
> track
> +patch status and also looking at how tooling can provide more feedback to
> users
> +about patch status. We are looking at tools such as ``patchtest`` to
> +test user contributions before they hit the mailing lists and also at
> better
> +documenting how to use such workflows since we recognise that whilst this
> was
> +common knowledge a decade ago, it might not be as familiar now.
>

I didn't know patchtest, I'll look at it. I think a big improvement (at
least for me) in the review
process would be to some kind of colorized webview of a patch towards the
target branch.
Sorry, but , unless I miss some cool feature of my gmail client, I find
that view (screenshot join)

far more readable that raw diff by mail, e.g:

diff --git a/meta/classes-recipe/cargo.bbclass
b/meta/classes-recipe/cargo.bbclass
index 3ef0bbbb44..8c0b92df8d 100644
--- a/meta/classes-recipe/cargo.bbclass
+++ b/meta/classes-recipe/cargo.bbclass
@@ -39,7 +39,12 @@ MANIFEST_PATH ??= "${S}/${CARGO_SRC_DIR}/Cargo.toml"

 RUSTFLAGS ??= ""
 BUILD_MODE = "${@['--release', ''][d.getVar('DEBUG_BUILD') == '1']}"
-CARGO_BUILD_FLAGS = "-v --offline --target ${RUST_HOST_SYS} ${BUILD_MODE}
--manifest-path=${MANIFEST_PATH}"
+# --frozen flag will prevent network access (which is required since only
+# the do_fetch step is authorized to access network)
+# and will require an up to date Cargo.lock file.
+# This force the package being built to already ship a Cargo.lock, in the
end
+# this is what we want, at least, for reproducibility of the build.
+CARGO_BUILD_FLAGS = "-v --frozen --target ${RUST_HOST_SYS} ${BUILD_MODE}
--manifest-path=${MANIFEST_PATH}"

 # This is based on the content of CARGO_BUILD_FLAGS and generally will
need to
 # change if CARGO_BUILD_FLAGS changes.

On Wed, 2023-08-09 at 17:58 +0200, Frédéric Martinsons wrote:
> On Wed, 9 Aug 2023 at 16:25, Michael Opdenacker via
> lists.yoctoproject.org
> <michael.opdenacker=bootlin.com@lists.yoctoproject.org> wrote:
> 
> > +One significant factor is that we value peer review. When a change
> > is proposed
> > +to many of the core pieces of the project, it helps to have many
> > eyes of review
> > +go over them. Whilst there is ultimately one maintainer who needs
> > to make the
> > +final call on accepting or rejecting a patch, the review is made
> > by many eyes
> > +and the exact people reviewing it are likely unknown to the
> > maintainer. It is
> > +often the surprise reviewer that catches the most interesting
> > issues!
> > +
> > +This is in contrast to the "GitHub" style workflow where either
> > just a
> > +maintainer makes that review, or review is specifically requested
> > from
> > +nominated people. We believe there is significant value added to
> > the codebase
> > +by this peer review and that moving away from mailing lists would
> > be to the
> > +detriment of our code.
> > 
> 
> 
> For notifications, I don't see why the web flow would not allow to
> send notifications for changes to a mailing list (I think it can also
> be made per user profile to receive notifications for a whole
> project, an issue, a particular project ... etc)

You can make a web interface send notices to an email list, sure, that
isn't hard. Someone then replies giving feedback on the mailing list to
the patch. What happens to that feedback from the web point of view?
This "next step" is where the challenge is.

Patchwork is an attempt to make that match up and it basically shows
how hard this is to get right and pair the worlds up like that.
>  
> > +We also need to acknowledge that many of our developers are used
> > to this
> > +mailing list workflow and have worked with it for years, with
> > tools and
> > +processes built around it. Changing away from this would result in
> > a loss
> > +of key people from the project, which would again be to its
> > detriment.
> 
> I'd like to know some of these flow that are plugged on mail, can we
> insert some examples in the documentation ?

You mean document how mailing list patch work flow works?

> > +The projects are acutely aware that potential new contributors
> > find the
> > +mailing list approach off-putting and would prefer a point-and-
> > click web GUI.
> > 
> 
> 
> I find the term "point-and-click" a little patronizing,
> github/gitlab/others web tools
> are not just point-and-click machine but highly customizable tool
> aims at smoothen the developer work (in term of tracking, viewing
> diff, forking) and to provide enhanced user experience.

I agree we should tweak the wording.

>  
> > +Since we don’t believe that can work for us, the project is aiming
> > to ensure
> > +`patchwork <https://patchwork.yoctoproject.org/>` is available to
> > help track
> > +patch status and also looking at how tooling can provide more
> > feedback to users
> > +about patch status. We are looking at tools such as ``patchtest``
> > to
> > +test user contributions before they hit the mailing lists and also
> > at better
> > +documenting how to use such workflows since we recognise that
> > whilst this was
> > +common knowledge a decade ago, it might not be as familiar now.
> > 
> 
> 
> I didn't know patchtest, I'll look at it. I think a big improvement
> (at least for me) in the review
> process would be to some kind of colorized webview of a patch towards
> the target branch.

If you make a pull request from the contrib repos, you can see
colorized diffs using the web interfaces. You can also see that locally
with git itself or tools like gitk too. I do use both the cgit
interface and gitk commands at different times.

> Sorry, but , unless I miss some cool feature of my gmail client, I
> find that view (screenshot join) far more readable that raw diff by
> mail, e.g:

It does depend upon what you're used to. Personally, I can hand edit or
even hand write diff files :/. I'm not saying that is a good idea or a
good thing but it is what it is.

Cheers,

Richard

Le mer. 9 août 2023, 19:57, Richard Purdie <
richard.purdie@linuxfoundation.org> a écrit :

> On Wed, 2023-08-09 at 17:58 +0200, Frédéric Martinsons wrote:
> > On Wed, 9 Aug 2023 at 16:25, Michael Opdenacker via
> > lists.yoctoproject.org
> > <michael.opdenacker=bootlin.com@lists.yoctoproject.org> wrote:
> >
> > > +One significant factor is that we value peer review. When a change
> > > is proposed
> > > +to many of the core pieces of the project, it helps to have many
> > > eyes of review
> > > +go over them. Whilst there is ultimately one maintainer who needs
> > > to make the
> > > +final call on accepting or rejecting a patch, the review is made
> > > by many eyes
> > > +and the exact people reviewing it are likely unknown to the
> > > maintainer. It is
> > > +often the surprise reviewer that catches the most interesting
> > > issues!
> > > +
> > > +This is in contrast to the "GitHub" style workflow where either
> > > just a
> > > +maintainer makes that review, or review is specifically requested
> > > from
> > > +nominated people. We believe there is significant value added to
> > > the codebase
> > > +by this peer review and that moving away from mailing lists would
> > > be to the
> > > +detriment of our code.
> > >
> >
> >
> > For notifications, I don't see why the web flow would not allow to
> > send notifications for changes to a mailing list (I think it can also
> > be made per user profile to receive notifications for a whole
> > project, an issue, a particular project ... etc)
>
> You can make a web interface send notices to an email list, sure, that
> isn't hard. Someone then replies giving feedback on the mailing list to
> the patch. What happens to that feedback from the web point of view?
> This "next step" is where the challenge is.
>

Yes I understand that.


> Patchwork is an attempt to make that match up and it basically shows
> how hard this is to get right and pair the worlds up like that.
> >
> > > +We also need to acknowledge that many of our developers are used
> > > to this
> > > +mailing list workflow and have worked with it for years, with
> > > tools and
> > > +processes built around it. Changing away from this would result in
> > > a loss
> > > +of key people from the project, which would again be to its
> > > detriment.
> >
> > I'd like to know some of these flow that are plugged on mail, can we
> > insert some examples in the documentation ?
>
> You mean document how mailing list patch work flow works?
>

No, I do know (or I think I know) how the patch flow works. I talked about
the text that says "with tools [...] built around it", I wouls like to know
what are these tools.


> > > +The projects are acutely aware that potential new contributors
> > > find the
> > > +mailing list approach off-putting and would prefer a point-and-
> > > click web GUI.
> > >
> >
> >
> > I find the term "point-and-click" a little patronizing,
> > github/gitlab/others web tools
> > are not just point-and-click machine but highly customizable tool
> > aims at smoothen the developer work (in term of tracking, viewing
> > diff, forking) and to provide enhanced user experience.
>
> I agree we should tweak the wording.
>
> >
> > > +Since we don’t believe that can work for us, the project is aiming
> > > to ensure
> > > +`patchwork <https://patchwork.yoctoproject.org/>` is available to
> > > help track
> > > +patch status and also looking at how tooling can provide more
> > > feedback to users
> > > +about patch status. We are looking at tools such as ``patchtest``
> > > to
> > > +test user contributions before they hit the mailing lists and also
> > > at better
> > > +documenting how to use such workflows since we recognise that
> > > whilst this was
> > > +common knowledge a decade ago, it might not be as familiar now.
> > >
> >
> >
> > I didn't know patchtest, I'll look at it. I think a big improvement
> > (at least for me) in the review
> > process would be to some kind of colorized webview of a patch towards
> > the target branch.
>
> If you make a pull request from the contrib repos, you can see
> colorized diffs using the web interfaces. You can also see that locally
> with git itself or tools like gitk too. I do use both the cgit
> interface and gitk commands at different times.
>

Yes of course, I can see my own patches the way I want. But what about
contributions from other that I'd like to read more carefully? I can pick
these patches and apply in my local checkout and see them "graphically" but
it is an additional step that may be skipped if we directly have them
somewhere. I agreed that patchwork have a better look interfaces than mail,
but as you said earlier, making the second step (i.e commenting a line,
suggesting changes... Etc) is not made in such interface.

>
> > Sorry, but , unless I miss some cool feature of my gmail client, I
> > find that view (screenshot join) far more readable that raw diff by
> > mail, e.g:
>
> It does depend upon what you're used to. Personally, I can hand edit or
> even hand write diff files :/. I'm not saying that is a good idea or a
> good thing but it is what it is.
>

Yes, I'm aware that it is a matter of taste and habits. I only want to gave
my point of view. Thank you very much for your answers.


> Cheers,
>
> Richard
>
>

On 8/9/23 16:25, Michael Opdenacker via lists.yoctoproject.org wrote:
> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
> 
> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>   .../contributor-guide/submit-change.rst       | 41 ++++++++++++++++++-
>   1 file changed, 40 insertions(+), 1 deletion(-)
> 
> diff --git a/documentation/contributor-guide/submit-change.rst b/documentation/contributor-guide/submit-change.rst
> index 2555767102..573491ecbc 100644
> --- a/documentation/contributor-guide/submit-change.rst
> +++ b/documentation/contributor-guide/submit-change.rst
> @@ -8,10 +8,49 @@ Because the system is extremely configurable and flexible, we recognize
>   that developers will want to extend, configure or optimize it for their
>   specific uses.
>   
> +.. _ref-why-mailing-lists:
> +
> +Contributing through mailing lists --- Why not using web-based workflows?
> +=========================================================================
> +
> +Both Yocto Project and OpenEmbedded have many key components that are
> +maintained by patches being submitted on mailing lists. We appreciate this
> +approach does look a little old fashioned when other workflows are available
> +through web technology such as GitHub, GitLab and others. Since we are often
> +asked this question, we’ve decided to document the reasons for using mailing
> +lists.
> +
> +One significant factor is that we value peer review. When a change is proposed
> +to many of the core pieces of the project, it helps to have many eyes of review
> +go over them. Whilst there is ultimately one maintainer who needs to make the
> +final call on accepting or rejecting a patch, the review is made by many eyes
> +and the exact people reviewing it are likely unknown to the maintainer. It is
> +often the surprise reviewer that catches the most interesting issues!
> +
> +This is in contrast to the "GitHub" style workflow where either just a
> +maintainer makes that review, or review is specifically requested from
> +nominated people. We believe there is significant value added to the codebase
> +by this peer review and that moving away from mailing lists would be to the
> +detriment of our code.
> +
> +We also need to acknowledge that many of our developers are used to this
> +mailing list workflow and have worked with it for years, with tools and
> +processes built around it. Changing away from this would result in a loss
> +of key people from the project, which would again be to its detriment.
> +
> +The projects are acutely aware that potential new contributors find the
> +mailing list approach off-putting and would prefer a point-and-click web GUI.
> +Since we don’t believe that can work for us, the project is aiming to ensure
> +`patchwork <https://patchwork.yoctoproject.org/>` is available to help track
> +patch status and also looking at how tooling can provide more feedback to users
> +about patch status. We are looking at tools such as ``patchtest`` to
> +test user contributions before they hit the mailing lists and also at better
> +documenting how to use such workflows since we recognise that whilst this was
> +common knowledge a decade ago, it might not be as familiar now.
> +
>   Finding a Suitable Mailing List
>   ===============================
>   
> -The Yocto Project uses a mailing list and a patch-based workflow that is

Oops

>   similar to the Linux kernel but contains important differences. In
>   general, there is a mailing list through which you can submit patches. You
>   should send patches to the appropriate mailing list so that they can be
> 
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#4121): https://lists.yoctoproject.org/g/docs/message/4121
> Mute This Topic: https://lists.yoctoproject.org/mt/100643892/3616718
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [daniel.ammann@bytesatwork.ch]
> -=-=-=-=-=-=-=-=-=-=-=-
>

HI Daniel,

On 10.08.23 at 09:42, Daniel Ammann wrote:
> On 8/9/23 16:25, Michael Opdenacker via lists.yoctoproject.org wrote:
>> From: Michael Opdenacker <michael.opdenacker@bootlin.com>
>>
>> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> ---
>>   .../contributor-guide/submit-change.rst       | 41 ++++++++++++++++++-
>>   1 file changed, 40 insertions(+), 1 deletion(-)
>>
>> diff --git a/documentation/contributor-guide/submit-change.rst 
>> b/documentation/contributor-guide/submit-change.rst
>> index 2555767102..573491ecbc 100644
>> --- a/documentation/contributor-guide/submit-change.rst
>> +++ b/documentation/contributor-guide/submit-change.rst
>> @@ -8,10 +8,49 @@ Because the system is extremely configurable and 
>> flexible, we recognize
>>   that developers will want to extend, configure or optimize it for 
>> their
>>   specific uses.
>>   +.. _ref-why-mailing-lists:
>> +
>> +Contributing through mailing lists --- Why not using web-based 
>> workflows?
>> +========================================================================= 
>>
>> +
>> +Both Yocto Project and OpenEmbedded have many key components that are
>> +maintained by patches being submitted on mailing lists. We 
>> appreciate this
>> +approach does look a little old fashioned when other workflows are 
>> available
>> +through web technology such as GitHub, GitLab and others. Since we 
>> are often
>> +asked this question, we’ve decided to document the reasons for using 
>> mailing
>> +lists.
>> +
>> +One significant factor is that we value peer review. When a change 
>> is proposed
>> +to many of the core pieces of the project, it helps to have many 
>> eyes of review
>> +go over them. Whilst there is ultimately one maintainer who needs to 
>> make the
>> +final call on accepting or rejecting a patch, the review is made by 
>> many eyes
>> +and the exact people reviewing it are likely unknown to the 
>> maintainer. It is
>> +often the surprise reviewer that catches the most interesting issues!
>> +
>> +This is in contrast to the "GitHub" style workflow where either just a
>> +maintainer makes that review, or review is specifically requested from
>> +nominated people. We believe there is significant value added to the 
>> codebase
>> +by this peer review and that moving away from mailing lists would be 
>> to the
>> +detriment of our code.
>> +
>> +We also need to acknowledge that many of our developers are used to 
>> this
>> +mailing list workflow and have worked with it for years, with tools and
>> +processes built around it. Changing away from this would result in a 
>> loss
>> +of key people from the project, which would again be to its detriment.
>> +
>> +The projects are acutely aware that potential new contributors find the
>> +mailing list approach off-putting and would prefer a point-and-click 
>> web GUI.
>> +Since we don’t believe that can work for us, the project is aiming 
>> to ensure
>> +`patchwork <https://patchwork.yoctoproject.org/>` is available to 
>> help track
>> +patch status and also looking at how tooling can provide more 
>> feedback to users
>> +about patch status. We are looking at tools such as ``patchtest`` to
>> +test user contributions before they hit the mailing lists and also 
>> at better
>> +documenting how to use such workflows since we recognise that whilst 
>> this was
>> +common knowledge a decade ago, it might not be as familiar now.
>> +
>>   Finding a Suitable Mailing List
>>   ===============================
>>   -The Yocto Project uses a mailing list and a patch-based workflow 
>> that is
>
> Oops
>
>>   similar to the Linux kernel but contains important differences. In
>>   general, there is a mailing list through which you can submit 
>> patches. You
>>   should send patches to the appropriate mailing list so that they 
>> can be
>>
>>
Oops indeed. Fixed directly in master-next.
Thanks for the review.
Cheers
Michael.

On 09.08.23 at 19:57, Richard Purdie wrote:
>>
>> I find the term "point-and-click" a little patronizing,
>> github/gitlab/others web tools
>> are not just point-and-click machine but highly customizable tool
>> aims at smoothen the developer work (in term of tracking, viewing
>> diff, forking) and to provide enhanced user experience.
> I agree we should tweak the wording.


I changed the text to a more neutral "would prefer a web-based GUI".
Done directly in master-next.
Thanks for the review!
Michael.