mbox series

[RFC,v2,0/4] Allow choosing the NVD feed

Message ID 20241212065435.15582-1-marta.rybczynska@ygreky.com
Headers show
Series Allow choosing the NVD feed | expand

Message

Marta Rybczynska Dec. 12, 2024, 6:54 a.m. UTC
This series is allowing choice of the NVD feed to use, you can
configure them using the NVD_DB_VERSION variable in local.conf

Available feeds:
- NVD2 (default) - the current NVD API v2 feed
- NVD1 - the old NVD feed (deprecated, but still working)
- FKIE - the NVD feed restoration from FKIE-CAD

Marta Rybczynska (4):
  cve-update-db-native: restore
  cve-update-db-native: update structure
  cve-update-db-native: add the fkie source
  cve-check: allow feed choice

 meta/classes/cve-check.bbclass                |   8 +-
 .../recipes-core/meta/cve-update-db-native.bb | 405 ++++++++++++++++++
 2 files changed, 411 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-core/meta/cve-update-db-native.bb

Comments

Ross Burton Dec. 16, 2024, 2:59 p.m. UTC | #1
On 12 Dec 2024, at 06:54, Marta Rybczynska via lists.openembedded.org <rybczynska=gmail.com@lists.openembedded.org> wrote:
> 
> This series is allowing choice of the NVD feed to use, you can
> configure them using the NVD_DB_VERSION variable in local.conf
> 
> Available feeds:
> - NVD2 (default) - the current NVD API v2 feed
> - NVD1 - the old NVD feed (deprecated, but still working)
> - FKIE - the NVD feed restoration from FKIE-CAD
> 
> Marta Rybczynska (4):
>  cve-update-db-native: restore
>  cve-update-db-native: update structure
>  cve-update-db-native: add the fkie source
>  cve-check: allow feed choice

If I select FKIE this fails to fetch:

File: '/home/rosbur01/Yocto/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 301, function: update_db_nvdjson
     0297:    import json
     0298:    bb.warn(str(jsondata)[:1000])
     0299:    root = json.loads(jsondata)
     0300:
 *** 0301:    for elt in root['CVE_Items']:
     0302:        if not elt['impact']:
     0303:            continue
     0304:
     0305:        accessVector = None
Exception: KeyError: 'CVE_Items'

Adding a dump of the JSON shows that the keys are lowercase, which the parser is not expecting:

WARNING: cve-update-db-native-1.0-r0 do_fetch: b'{\n  "timestamp": "2024-12-16T01:00:10+00:00",\n  "cve_count": 2392,\n  "feed_name": "CVE-2002",\n  "source": "fkie-cad/nvd-json-data-feeds",\n  "cve_items”:

Ross
Ross Burton Dec. 16, 2024, 3:01 p.m. UTC | #2
On 16 Dec 2024, at 14:59, Ross Burton <Ross.Burton@arm.com> wrote:
> 
> On 12 Dec 2024, at 06:54, Marta Rybczynska via lists.openembedded.org <rybczynska=gmail.com@lists.openembedded.org> wrote:
>> 
>> This series is allowing choice of the NVD feed to use, you can
>> configure them using the NVD_DB_VERSION variable in local.conf
>> 
>> Available feeds:
>> - NVD2 (default) - the current NVD API v2 feed
>> - NVD1 - the old NVD feed (deprecated, but still working)
>> - FKIE - the NVD feed restoration from FKIE-CAD
>> 
>> Marta Rybczynska (4):
>> cve-update-db-native: restore
>> cve-update-db-native: update structure
>> cve-update-db-native: add the fkie source
>> cve-check: allow feed choice
> 
> If I select FKIE this fails to fetch:
> 
> File: '/home/rosbur01/Yocto/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 301, function: update_db_nvdjson
>     0297:    import json
>     0298:    bb.warn(str(jsondata)[:1000])
>     0299:    root = json.loads(jsondata)
>     0300:
> *** 0301:    for elt in root['CVE_Items']:
>     0302:        if not elt['impact']:
>     0303:            continue
>     0304:
>     0305:        accessVector = None
> Exception: KeyError: 'CVE_Items'
> 
> Adding a dump of the JSON shows that the keys are lowercase, which the parser is not expecting:
> 
> WARNING: cve-update-db-native-1.0-r0 do_fetch: b'{\n  "timestamp": "2024-12-16T01:00:10+00:00",\n  "cve_count": 2392,\n  "feed_name": "CVE-2002",\n  "source": "fkie-cad/nvd-json-data-feeds",\n  "cve_items”:

Nope I’m wrong, the correct bug is "if you typo the NVD_DB_VERSION then weird things happen”.  I suggest either validating it early, or always checking for equality instead of using else.  I accidentally put KFIE and it fell into a bad combination of cases.

Ross
Marta Rybczynska Dec. 16, 2024, 8:38 p.m. UTC | #3
On Mon, 16 Dec 2024, 16:01 Ross Burton, <Ross.Burton@arm.com> wrote:

> On 16 Dec 2024, at 14:59, Ross Burton <Ross.Burton@arm.com> wrote:
> >
> > On 12 Dec 2024, at 06:54, Marta Rybczynska via lists.openembedded.org
> <rybczynska=gmail.com@lists.openembedded.org> wrote:
> >>
> >> This series is allowing choice of the NVD feed to use, you can
> >> configure them using the NVD_DB_VERSION variable in local.conf
> >>
> >> Available feeds:
> >> - NVD2 (default) - the current NVD API v2 feed
> >> - NVD1 - the old NVD feed (deprecated, but still working)
> >> - FKIE - the NVD feed restoration from FKIE-CAD
> >>
> >> Marta Rybczynska (4):
> >> cve-update-db-native: restore
> >> cve-update-db-native: update structure
> >> cve-update-db-native: add the fkie source
> >> cve-check: allow feed choice
> >
> > If I select FKIE this fails to fetch:
> >
> > File: '/home/rosbur01/Yocto/poky/meta/recipes-core/meta/
> cve-update-db-native.bb', lineno: 301, function: update_db_nvdjson
> >     0297:    import json
> >     0298:    bb.warn(str(jsondata)[:1000])
> >     0299:    root = json.loads(jsondata)
> >     0300:
> > *** 0301:    for elt in root['CVE_Items']:
> >     0302:        if not elt['impact']:
> >     0303:            continue
> >     0304:
> >     0305:        accessVector = None
> > Exception: KeyError: 'CVE_Items'
> >
> > Adding a dump of the JSON shows that the keys are lowercase, which the
> parser is not expecting:
> >
> > WARNING: cve-update-db-native-1.0-r0 do_fetch: b'{\n  "timestamp":
> "2024-12-16T01:00:10+00:00",\n  "cve_count": 2392,\n  "feed_name":
> "CVE-2002",\n  "source": "fkie-cad/nvd-json-data-feeds",\n  "cve_items”:
>
> Nope I’m wrong, the correct bug is "if you typo the NVD_DB_VERSION then
> weird things happen”.  I suggest either validating it early, or always
> checking for equality instead of using else.  I accidentally put KFIE and
> it fell into a bad combination of cases.
>

Thanks for testing Ross! I'm fixing that.

Kind regards,
Marta

>