Find duplicates in bash

find_dupes.sh

 #! /bin/bash
OUTF=rem-duplicates.sh;
echo "#! /bin/sh" > $OUTF;
echo ""                >> $OUTF;
find "$@" -type f -print0 | xargs -0 -n1 md5sum | sort --key=1,32 | uniq -w 32 -d --all-repeated=separate | sed -r 's/^[0-9a-f]*( )*//;s/([^a-zA-Z0-9./_-])/\\\1/g;s/(.+)/#rm \1/' >> $OUTF;
chmod a+x $OUTF

Run the script:

./find_dupes.sh /path/to/check .\

Which will output a file called rem-duplicates.sh which can then be edited as you wish.

I stole this from here. Credit to Marco Fioretti, who in turn used code from J. Elonen