| Message ID | 20220429063226.22192-2-rybczynska@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/2] cve-update-db-native: update the CVE database once a day only | expand |
Hi Marta, This explains why the CVE database update seemed to happen far more frequently than it should. Thanks for digging into it. On Fri, Apr 29, 2022 at 2:32 AM Marta Rybczynska <rybczynska@gmail.com> wrote: > > Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force > the database update, if the default update frequency is too low. Just an idea, maybe instead of a boolean, the variable could specify the interval, eg: # Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60" Regards, Ralph
On Fri, Apr 29, 2022 at 5:53 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote: > Hi Marta, > > This explains why the CVE database update seemed to happen far more > frequently than it should. Thanks for digging into it. > > On Fri, Apr 29, 2022 at 2:32 AM Marta Rybczynska <rybczynska@gmail.com> > wrote: > > > > Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force > > the database update, if the default update frequency is too low. > > Just an idea, maybe instead of a boolean, the variable could specify > the interval, eg: > > # Interval between CVE database updates, in seconds. > # Set to "0" to to force an update of the database. > CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60" > > This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly. Regards, Marta
On Mon, May 2, 2022 at 9:23 AM Marta Rybczynska <rybczynska@gmail.com> wrote: > > On Fri, Apr 29, 2022 at 5:53 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote: >> >> # Interval between CVE database updates, in seconds. >> # Set to "0" to to force an update of the database. >> CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60" >> > > This is a good idea, thank you Ralph, I like it. I'll be sending a v2 > shortly. Thanks for this. I've tested it locally, on the fist run, the CVE database was fetched (it took quite a while, for some reason). On subsequent runs, no fetch occurs, so this seems to be working. I then set CVE_DB_UPDATE_INTERVAL = "3600" in my local.conf, and ran the build again. As it had been over an hour since the first build, the database was downloaded again. The timestamp on nvdcve_1.1.db did not change (as noted in one of your commit descriptions). So, it seems to be working correctly. I will re-test tomorrow (eg. after 24 hours) with the interval set back to default. One minor point of confusion is that the log still shows "NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Started" even when the download is skipped. This is of course understandable when looking at the python code, the check is within the do_fetch function. There is probably no simple way to avoid this from being displayed. And most users won't notice anyway. But it initially confused me about what was happening. Regards, Ralph
On Mon, May 2, 2022 at 12:43 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote: > > On Mon, May 2, 2022 at 9:23 AM Marta Rybczynska <rybczynska@gmail.com> wrote: > > > > On Fri, Apr 29, 2022 at 5:53 PM Ralph Siemsen <ralph.siemsen@linaro.org> wrote: > >> > >> # Interval between CVE database updates, in seconds. > >> # Set to "0" to to force an update of the database. > >> CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60" > >> > > > > This is a good idea, thank you Ralph, I like it. I'll be sending a v2 > > shortly. > > Thanks for this. I've tested it locally, on the fist run, the CVE > database was fetched (it took quite a while, for some reason). On > subsequent runs, no fetch occurs, so this seems to be working. > > I then set CVE_DB_UPDATE_INTERVAL = "3600" in my local.conf, and ran > the build again. As it had been over an hour since the first build, > the database was downloaded again. The timestamp on nvdcve_1.1.db did > not change (as noted in one of your commit descriptions). > > So, it seems to be working correctly. I will re-test tomorrow (eg. > after 24 hours) with the interval set back to default. > > One minor point of confusion is that the log still shows "NOTE: recipe > cve-update-db-native-1.0-r0: task do_fetch: Started" even when the > download is skipped. This is of course understandable when looking at > the python code, the check is within the do_fetch function. There is > probably no simple way to avoid this from being displayed. And most > users won't notice anyway. But it initially confused me about what was > happening. yeah you can avoid it unless another logic outside of this function is invoked. perhaps we can add a diagnostic inside the do_fetch to spill out the state information and informing like "no fetch needed" or "database uptodate" or somesuch > > Regards, > Ralph > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165172): https://lists.openembedded.org/g/openembedded-core/message/165172 > Mute This Topic: https://lists.openembedded.org/mt/90771095/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index af39480dda..d89f79f310 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -13,6 +13,7 @@ deltask do_install deltask do_populate_sysroot NVDCVE_URL ?= "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-" +FORCE_CVE_DB_UPDATE ?= "0" python () { if not bb.data.inherits_class("cve-check", d): @@ -44,11 +45,14 @@ python do_fetch() { os.remove(db_file) # The NVD database changes once a day, so no need to update more frequently + # Allow the user to force-update try: import time - if time.time() - os.path.getmtime(db_file) < (24*60*60): - bb.debug(2, "Recently updated, skipping") - return + if d.getVar("FORCE_CVE_DB_UPDATE") == "0": + if time.time() - os.path.getmtime(db_file) < (24*60*60): + bb.debug(2, "Recently updated, skipping") + return + except OSError: pass
Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force the database update, if the default update frequency is too low. Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com> --- meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)