| Message ID | 20220511143613.25002-1-akash.hadke@kpit.com |
|---|---|
| State | New |
| Headers | show |
| Series | [poky,master,1/3] cve_check.py: Add new method get_ignored_cves | expand |
On Wed, May 11, 2022 at 4:37 PM akash hadke via lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org> wrote: > > Add new method get_ignored_cves in cve_check.py > to get ignored CVEs from recipe by excluding distro-wide > ignored CVEs from meta/conf/distro/include/cve-extra-exclusions.inc > > While calling this method use below code to get argument values > paths = d.getVar('PATH').split(':') > cves = d.getVar('CVE_CHECK_IGNORE').split() > Hello Akash, While looking into this patch set I'm wondering what is your use case. It seems to be to get a list of ignored and patched CVEs. This is already available from the cve-check output or from the create-spdx output after some parsing. With the new JSON format for cve-check it becomes very easy. If you could elaborate more on the way you plan to use this data, I'm pretty sure we can come with a simple post-processing script to do the same. BTW Why do assume people always include meta/conf/distro/include/cve-extra-exclusions.inc ? We don't do that at Oniro and we use our own judgement on outstanding CVEs. Regards, Marta
Hello Marta, Actually, I wanted to add the ignored and patched CVEs in buildhistory and for that purpose, I am exporting variables CVE_IGNORED and CVE_PATCHED with those values. I don't want to use cve-check.bbclass as it checks for the CVEs from the NVD database, and I only want to get ignored and patched CVEs from the recipe. Regarding meta/conf/distro/include/cve-extra-exclusions.inc if any project includes it then CVEs that are ignored in cve-extra-exclusions.inc will get shown for each recipe in the CVE_CHECK_IGNORED list even though the CVEs are not related to that component recipe. Hence, I have did the changes to exclude CVEs from cve-extra-exclusions.inc Best Regards, Akash ________________________________ From: Marta Rybczynska <rybczynska@gmail.com> Sent: 17 May 2022 14:42 To: Akash Hadke <Akash.Hadke@kpit.com> Cc: OE-core <openembedded-core@lists.openembedded.org>; Ranjitsinh Rathod <Ranjitsinh.Rathod@kpit.com>; Akash Hadke <hadkeakash4@gmail.com> Subject: Re: [OE-core] [poky][master][PATCH 1/3] cve_check.py: Add new method get_ignored_cves Caution: This email originated from outside of the KPIT. Do not click links or open attachments unless you recognize the sender and know the content is safe. On Wed, May 11, 2022 at 4:37 PM akash hadke via lists.openembedded.org <akash.hadke=kpit.com@lists.openembedded.org> wrote: > > Add new method get_ignored_cves in cve_check.py > to get ignored CVEs from recipe by excluding distro-wide > ignored CVEs from meta/conf/distro/include/cve-extra-exclusions.inc > > While calling this method use below code to get argument values > paths = d.getVar('PATH').split(':') > cves = d.getVar('CVE_CHECK_IGNORE').split() > Hello Akash, While looking into this patch set I'm wondering what is your use case. It seems to be to get a list of ignored and patched CVEs. This is already available from the cve-check output or from the create-spdx output after some parsing. With the new JSON format for cve-check it becomes very easy. If you could elaborate more on the way you plan to use this data, I'm pretty sure we can come with a simple post-processing script to do the same. BTW Why do assume people always include meta/conf/distro/include/cve-extra-exclusions.inc ? We don't do that at Oniro and we use our own judgement on outstanding CVEs. Regards, Marta
On Tue, May 17, 2022 at 1:42 PM Akash Hadke <Akash.Hadke@kpit.com> wrote: > > Hello Marta, > > Actually, I wanted to add the ignored and patched CVEs in buildhistory and for that purpose, I am exporting variables CVE_IGNORED and CVE_PATCHED with those values. I don't want to use cve-check.bbclass as it checks for the CVEs from the NVD database, and I only want to get ignored and patched CVEs from the recipe. Hello again Akash, What you'd like to do is to see the difference in ignored and patched CVEs in buildhistory? Do I get it right? > > Regarding meta/conf/distro/include/cve-extra-exclusions.inc if any project includes it then CVEs that are ignored in cve-extra-exclusions.inc will get shown for each recipe in the CVE_CHECK_IGNORED list even though the CVEs are not related to that component recipe. Hence, I have did the changes to exclude CVEs from cve-extra-exclusions.inc I think I understand the idea. The point I'm making is that if someone does not include the cve-extra-exclusions.inc in their distro, the code will still use it and filter out CVEs they still see when doing cve-check. Kind regards, Marta > > Best Regards, > Akash > ________________________________ > From: Marta Rybczynska <rybczynska@gmail.com> > Sent: 17 May 2022 14:42 > To: Akash Hadke <Akash.Hadke@kpit.com> > Cc: OE-core <openembedded-core@lists.openembedded.org>; Ranjitsinh Rathod <Ranjitsinh.Rathod@kpit.com>; Akash Hadke <hadkeakash4@gmail.com> > Subject: Re: [OE-core] [poky][master][PATCH 1/3] cve_check.py: Add new method get_ignored_cves > > Caution: This email originated from outside of the KPIT. Do not click links or open attachments unless you recognize the sender and know the content is safe. > > On Wed, May 11, 2022 at 4:37 PM akash hadke via lists.openembedded.org > <akash.hadke=kpit.com@lists.openembedded.org> wrote: > > > > Add new method get_ignored_cves in cve_check.py > > to get ignored CVEs from recipe by excluding distro-wide > > ignored CVEs from meta/conf/distro/include/cve-extra-exclusions.inc > > > > While calling this method use below code to get argument values > > paths = d.getVar('PATH').split(':') > > cves = d.getVar('CVE_CHECK_IGNORE').split() > > > > Hello Akash, > While looking into this patch set I'm wondering what is your use case. > It seems to be to get a list > of ignored and patched CVEs. This is already available from the > cve-check output or from the create-spdx > output after some parsing. With the new JSON format for cve-check it > becomes very easy. If you could > elaborate more on the way you plan to use this data, I'm pretty sure > we can come with a simple > post-processing script to do the same. > > BTW Why do assume people always include > meta/conf/distro/include/cve-extra-exclusions.inc ? > We don't do that at Oniro and we use our own judgement on outstanding CVEs. > > Regards, > Marta
Hi Marta, If you see the code from the succeeding patch to this https://lists.openembedded.org/g/openembedded-core/message/165502 here I have checked if cve-extra-exclusions.inc is included or not. If it is not included then the code will not get executed.
On Tue, 2022-05-17 at 11:42 +0000, akash hadke via lists.openembedded.org wrote: > Actually, I wanted to add the ignored and patched CVEs in > buildhistory and for that purpose, I am exporting variables > CVE_IGNORED and CVE_PATCHED with those values. I don't want to use > cve-check.bbclass as it checks for the CVEs from the NVD database, > and I only want to get ignored and patched CVEs from the recipe. I'd really prefer to have one cve handling class where we can configure it to get the data different people need rather than multiple difference cve classes which are going to confuse people. Could we have a way to disable NVD data from the cve-check class? Cheers, Richard
Hi Richard, I tried modifying the cve-check.bbclass but did not able to get the solution for disabling the NVD data, because when we inherit cve-check it executes the cve_check task that checks CVEs from NVD DB. So I am also not sure how to disable it hence I used the separate bbclass.
On Wed, 2022-05-18 at 02:46 -0700, akash hadke via lists.openembedded.org wrote: > Hi Richard, > > I tried modifying the cve-check.bbclass but did not able to get the > solution for disabling the NVD data, because when we inherit cve- > check it executes the cve_check task that checks CVEs from NVD DB. So > I am also not sure how to disable it hence I used the separate > bbclass. I guess the task dependencies are a bit of an issue there. There are probably ways to make those configurable, it would just take a little work. I really do want to encourage us to work together on common cve tooling rather than having several partial implementations so I can't take this patch series. Cheers, Richard
On Wed, May 18, 2022 at 12:33 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Wed, 2022-05-18 at 02:46 -0700, akash hadke via > lists.openembedded.org wrote: > > Hi Richard, > > > > I tried modifying the cve-check.bbclass but did not able to get the > > solution for disabling the NVD data, because when we inherit cve- > > check it executes the cve_check task that checks CVEs from NVD DB. So > > I am also not sure how to disable it hence I used the separate > > bbclass. > > I guess the task dependencies are a bit of an issue there. There are > probably ways to make those configurable, it would just take a little > work. > > I really do want to encourage us to work together on common cve tooling > rather than having several partial implementations so I can't take this > patch series. > I plan to cut those dependencies and make different stages optional. In my case the first use is to be able to run multiple cve-checks with the same database (guaranteed without updates) or run cve-check with some given known database (for testing). We can add an option to make the actual check with the database optional. Exporting the data to the buildhistory looks like a useful thing to do, too. I can draft the split after YPS. Kind regards, Marta
diff --git a/meta/lib/oe/cve_check.py b/meta/lib/oe/cve_check.py index dc7d2e2826..d96d47b737 100644 --- a/meta/lib/oe/cve_check.py +++ b/meta/lib/oe/cve_check.py @@ -163,3 +163,41 @@ def cve_check_merge_jsons(output, data): return output["package"].append(data["package"][0]) + +def get_ignored_cves(paths, cves): + """ + Get ignored CVEs from the recipe and exlude the CVEs from + meta/conf/distro/include/cve-extra-exclusions.inc + + While calling this method use below code to get argument values + paths = d.getVar('PATH').split(':') + cves = d.getVar('CVE_CHECK_IGNORE').split() + """ + import os + + cve_extra_exclusion_inc_file = "../meta/conf/distro/include/cve-extra-exclusions.inc" + for path in paths: + check_for_correct_file_path = os.path.join(path, cve_extra_exclusion_inc_file) + if os.path.isfile(check_for_correct_file_path): + inc_file = check_for_correct_file_path + else: + continue + + cve_check_ignored = set() + ignored_cves = set() + with open(inc_file) as f: + lines = f.readlines() + for line in lines: + if line.strip(): + # Ignore the comments from cve-extra-exclusions.inc + if not re.search("^#", line): + cve_match = re.findall(r'CVE\-\d{4}\-\d+', line) + for cve in cve_match: + ignored_cves.add(cve) + + for cve in cves: + if cve not in ignored_cves: + cve_check_ignored.add(cve) + ignored_cves_from_recipe = " ".join(cve_check_ignored) + + return ignored_cves_from_recipe
Add new method get_ignored_cves in cve_check.py to get ignored CVEs from recipe by excluding distro-wide ignored CVEs from meta/conf/distro/include/cve-extra-exclusions.inc While calling this method use below code to get argument values paths = d.getVar('PATH').split(':') cves = d.getVar('CVE_CHECK_IGNORE').split() Signed-off-by: Akash Hadke <akash.hadke@kpit.com> Signed-off-by: Akash Hadke <hadkeakash4@gmail.com> --- meta/lib/oe/cve_check.py | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+)