Finding duplicate File names in a directory tree

Just a little one liner. Essentially I have a nested file system structure with a lot of subfolders and I wanted to know if there are duplicate file names.

This does the trick for me:

find . -iname "*.txt" | xargs -I {} basename {} | uniq -d -i 

So what this does:

  1. Call find to find all files that match *.txt (case insensitive) in the current folder and all subfolders
  2. Call basename to turn /path/to/file.txt into file.txt. We have to use xargs since basename does not seem to be able to read from stdin
  3. Call uniq to check the list for duplicates. -d means that we only show duplicates (default is to show unique values) and -i performs a case insensitive comparison

Sidenote: As much as I love working with Windows, it just does not stand a chance against a long chain of simple, small, well-defined UNIX tools that are piped together. As much as PowerShell is a step in the right direction, it's not the same because the philosophy of having tons of very limited and specialized tools doesn't seem to exist in the Windows universe - most command line tools do too much. Also, UNIX has a 30 year head start against PowerShell.

Comments (3)

NathanApril 21st, 2011 at 14:07

Just a quick correction:
the 'uniq' command only compares continuous duplicates (matches found directly next to each other). Therefore, you need to add a 'sort' command before calling 'uniq'. The end-product looks like this:
find . -iname "*.txt" | xargs -I {} basename {} | sort | uniq -d -i

MirkoJune 10th, 2011 at 13:30

ls -include *.txt -recurse | group name | ?{$_.count -gt 1}

AussieOSXJune 14th, 2012 at 03:53

The FIND code resulted in an error on error on OSX 10.7

The error message is "xargs: unterminated quote"

At a guess, the might be caused by spaces in file name or directory names. I tried to modify the code to allow for spaces but couldnt get it to work.

If you know how to fix this error, please post an update. Thank you

P.S I tried Mirko's code, but it did not work on OSX 10.7 for me