File fingerprint finder

[selfup]

Thoughts

New mode: -m fff File Fingerprint Finder: search by SHA2-256 hash!

Example use:

$ known_hash="de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd"
$ go run main.go -m fff -d $HOME/Documents -k $known_hash
/home/selfup/Documents//dotfiles/mac/.bash_profile

Probably going to stick to SHA2-256 for now. Will add 512 later? SHA2-256 seems pretty standard and collisions shouldn't be an issue for quite some time. Performance still needs to be evaluated but I don't think I can avoid having to run it on all files. Maybe link to a vuln database to see if I can start scanning file systems for known malware. Could be interesting!

Caveat will always be that I skip files that cannot be read with current user permissions. I will not throw an error. This is by design. You should know your permissions and which files are going to be in scope.

Test on a decent amount of files

Some of the files in this directory are quite large so this "test" is not the best for speed but can be a good indicator of running sha256 on large files.

Out of the 48,588 files:

• 133 are 10MB or more
• 11 of the 133 are 100MB or more

scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME/Documents/ | wc -l
133
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME/Documents/ | wc -l
11

Here's the fff run using time

scnnr (file-fingerprint-finder) $ go run main.go -d $HOME/Documents | wc -l
48588
scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k de4f51f97fa690026e225798ff294cd182b93847aaa46fe1e32b848eb9e985bd -d $HOME/Documents/                                    
/home/selfup/Documents//dotfiles/mac/.bash_profile

real    0m24.897s
user    0m26.849s
sys     0m7.355s

Test on a large amount of files

Running this on a $HOME with 240k+ files:

scnnr (file-fingerprint-finder) $ go run main.go -d $HOME | wc -l
242948
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 10MB -d $HOME | wc -l
227
scnnr (file-fingerprint-finder) $ go run main.go -m fsf -s 100MB -d $HOME | wc -l
20

Here is the result:

scnnr (file-fingerprint-finder) $ time go run main.go -m fff -k 93a2f45f1a1f2ebdc0cca3019d42aae9631c016b65e553c041865ff0c6692858 -d $HOME
/home/selfup/.tmux.conf

real    0m36.652s
user    1m6.782s
sys     0m17.100s