BFG: git Repo Cleaner

As mentioned in github docs, BFG is the prefered way to scrub secrets, credentials, PHI, and other sensitive information accidentally included in a git repo. (avoid git -A and you're unlikely to end up in this situation)

On rhea us the command: java -jar /opt/ni_tools/bfg-git-repo-cleaner/bfg-1.14.0.jar

Here an ssh key is accidentally included and upload to github.

You might try removing the file as you would any other source file

git rm sshkey*
git commit -m 'remove secrets'

Though it's gone from the top of the repository, the file still exists in the history! For most things that are to be version controlled, this is ideal. You want to be able to go back and time and see what was included.

Run bfg to delete the file. In the example, we cd to the

cd /Volumes/Hera/Projects/7TBrainMech/scripts/eeg/Shane
java -jar /opt/ni_tools/bfg-git-repo-cleaner/bfg-1.14.0.jar  --delete-files sshkey
git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push --force

Add the names of secret files to a line in .gitignore. git -A will ignore those files and git will warn you (and require git add -f) if you try to manually add them.

Github is aggressively mirrored by separate entities. A published secret then removed has still likely been captured externally. You should regenerate any compromised credentials in addition to scrubbing them from the repo.

See blog.acolyer.org's breakdown of 10.14722/ndss.2019.23418 [2019] for a scary insight into extracing secrets from github's searches API: 30 mins to scan all of github and find 100,000s leaked credentials.