Rake task registers database path for rake-cleaning

[jasonkarns]

Rake ships with clean and clobber tasks out of the box. Now the audit Task can be (optionally) registered for cleaning.

It defaults to off so no change from current behavior. If opted-in, the audit-database path is added to rake's CLOBBER collection.

Clean is meant to remove temporary and generated files that are the result of "normal" application runs.
Clobber is intended to be a more exhaustive clean that returns the application's state to a "fresh cloned" state.

With these conventions, the ruby advisory db fits more cleanly (ha) into the "clobber" category.

prior art: Bundler's gem tasks itself:
https://github.com/ruby/rubygems/blob/master/bundler/lib/bundler/gem_tasks.rb#L3-L4

Bundler's tasks register the pkg path for clobbering unconditionally, but I think that makes sense given the limited scope of the pkg path. Since the ruby advisory db is written to a path outside of one's project, I believe the task should be more conservative in its deletion. Hence making it opt-in.

This PR does not yet have tests. If this enhancement is a viable candidate, I can add tests and documentation.

[postmodern]

Bundler::Audit::Database.path defaults to ~/.local/share/ which is meant for persistent app data, not temporary files. It is unwise to delete ruby-advisory-db, which would cause bundler-audit to have to re-download the entire Git repository on the next run instead of updating it.

[jasonkarns]

Right. Temp files would be cleaned by rake clean. I'm not proposing that it be added to the CLEAN collection.

But the database is a generated file. It's expected that after rake clobber, the project is back to a just-cloned state. This is necessary for testing, validating CI changes locally, debugging a potentially-corrupted download, or simply purging the repo back to pristine so one can replay scripts "from zero".

Per rake:

# [:clobber] Clobber all generated and non-source files in a project.
#            The task depends on :clean, so all the clean files will
#            be deleted as well as files in the CLOBBER file list.
#            The intent of this task is to return a project to its
#            pristine, just unpacked state.

Clobber is run very rarely, so the advisory db would still persist effectively all the time. But when a "full reset" is needed, that's what clobber is for.

[postmodern]

Right. Temp files would be cleaned by rake clean. I'm not proposing that it be added to the CLEAN collection.

Then the keyword argument should be named clobber: instead of clean:?

But the database is a generated file. It's expected that after rake clobber, the project is back to a just-cloned state. This is necessary for testing, validating CI changes locally, debugging a potentially-corrupted download, or simply purging the repo back to pristine so one can replay scripts "from zero".

The ruby-advisory-db is not really meant to be rolled back. git will also already handle failed or corrupted downloads. Could you help me understand how you would use this feature in a real-world project or scenario?

[jasonkarns]

Then the keyword argument should be named clobber: instead of clean:?

Perhaps. Though both rake clean and rake clobber are provided by require "rake/clean". Clobber is more exhaustive but still under the larger feature umbrella of "cleaning". I wouldn't be against that as the option name though. The make world has similar tiers of "cleaning" but as far as I've seen, it's all generalized under "cleaning" (despite having different tasks for different degrees of "clean"). 1 2

The immediate use case I had, was building out our rake lint:ci primary task for CircleCI. It was in a legacy codebase that had configured a custom bundle:audit task before bundler-audit gained its own rake task. They were running the CLI directly, but the way the task deps were wired up, it wasn't guaranteed to download before auditing. Over a few runs, we'd occasionally see it run audit and then update the db. (Presumably because the pre-reqs weren't wired up correctly and rake was also running in multitask mode.)

Regardless, as part of updating to latest bundler-audit and moving to bundler-audit's built-in tasks, I needed to consistently iterate by simulating a fresh CI environment. My first inclination in all cases of running rake tasks, is to expect rake clobber to restore to a clean slate. As I dug deeper and discovered this wasn't the case, I wired it up to CLOBBER myself so I had a tight feedback loop. (And our rake environment preserves the convention that rake tasks can build/write things; clean wipes transient scratch/cache/tmp files, and clobber reverts to a pristine state.)

Other folks on the team have been using the clobber task to revert to pristine state in order to do performance auditing. (clobber, then run tasks with profiling to gather performance data).

I don't know of any other use cases off hand. That's just what we've been using recently.

I should pre-empt the obvious point: doing a simple rm on the directory works. However, as the bundle-audit task is only one of many that we're running (tasks upon tasks all built up with different pre-req trees), the entire point of rake clean/clobber is for devs to not need to know which files they need to go remove. A dev running lint:ci, shouldn't need to know that includes rubocop's html and junit reports, brakeman's json, html, and junit reports. Or that after spec:ci, you have simplecov coverage, and rspec reports, and the "next-failures" scratch file. Whenever cleaning or clobbering is necessary, they don't need to know which (and where) the multitude of files exist at, they just run the corresponding tasks.

A few examples of other files we have in our CLOBBER list (distinct from CLEAN):

• Rails' local-disk active-storage for test env
• our Rails encrypted-credentials master keys (whose re-fetching is scripted via a rake task that pulls from heroku)