Page 1 of 1

Clean up duplicate files

Posted: Thu Jan 14, 2010 11:54 am
by andrebaros
These may be the first of many questions...

I finally set up my Amahi Home Server and it was super easy... now to use it.

1. I'm using it to consolidate files from 4 old machines into one place, especially photos and music. What I'm ending up with is a LOT of duplicates because I ripped the same CD to multiple machines or downloaded photos to more than one place... over the years it's added up and was a big reason for me to set up the server. What is a good duplicate file cleanup app I can run on Fedora?

Thanks
Andre

Re: Clean up duplicate files

Posted: Fri Jan 15, 2010 1:46 am
by moredruid
I personally use fslint
you can install it with:
yum install fslint

great tool, is quite resource intensive since it checks duplicates on the basis of filenames and content (by sha and md5 summing)

Re: Clean up duplicate files

Posted: Fri Jan 15, 2010 2:20 am
by cpg
NICE TOOL!!

i did not know about this disk deduplication tool!

i am on the road on a slow connection, and i was hoping the command line tool would be included, alas, it does not seem to be included, even if yum info advertises it.

i will be putting it through it's paces just as soon as i can!

thanks for sharing!

Re: Clean up duplicate files

Posted: Sun Jan 17, 2010 3:19 am
by hfxrzw
Team, sounds like a very usefull app for the HDA. How would one go about getting it installed/useable on the server? Thanks, Rene.

Re: Clean up duplicate files

Posted: Sun Jan 24, 2010 5:03 pm
by abateman
It's not the best program in the world, but if you're after a command-line one you can try fdupes: http://en.wikipedia.org/wiki/Fdupes

I had a similar situation (combining a couple/few different hard drives) and it did a good job of picking up al the duplicates.