Finding duplicate File names in a directory tree

Just a little one liner. Essentially I have a nested file system structure with a lot of subfolders and I wanted to know if there are duplicate file names.

This does the trick for me:

find . -iname "*.txt" | xargs -I {} basename {} | uniq -d -i 

So what this does:

  1. Call find to find all files that match *.txt (case insensitive) in the current folder and all subfolders
  2. Call basename to turn /path/to/file.txt into file.txt. We have to use xargs since basename does not seem to be able to read from stdin
  3. Call uniq to check the list for duplicates. -d means that we only show duplicates (default is to show unique values) and -i performs a case insensitive comparison

Sidenote: As much as I love working with Windows, it just does not stand a chance against a long chain of simple, small, well-defined UNIX tools that are piped together. As much as PowerShell is a step in the right direction, it’s not the same because the philosophy of having tons of very limited and specialized tools doesn’t seem to exist in the Windows universe – most command line tools do too much. Also, UNIX has a 30 year head start against PowerShell.

Connected Web Parts are bugged in SharePoint 2010

Update: This is an intentional design change, see the Microsoft answer in this thread.

Just stumbled upon a bug in SharePoint 2010. Imagine this scenario: You have two lists and add them to a Site as DataForm WebParts. You then configure the Web Parts so that one of them sends Filter values to the other one and uncheck the option for “Send first row to connected Web Parts when page loads”. You refresh the Page and notice that the list that receives the Filter (the Consumer) does not show any items because the list that provides the Filter (the Provider) does not have any items selected.

In SharePoint 2007, not selecting an item in the Provider List would cause the consumer List to display all items instead, which was (at least for us) highly desirable. In SharePoint 2010, you have to select an Item in the Provider to provide a filter and there is no way to have the Consumer List display all items anymore.

The reason why I think this is a bug is the result of an extended Debugging session that eventually led me to the class Microsoft.SharePoint.WebPartPages.DataFormWebPart and specifically it’s method ApplyReceivedFilterData. I’m not going to paste the source of this class, but the culprit is this line: if (dictionary2[str] != null)

This is supposed to only add Filters that have a value and skip the ones that are null. The problem is that dictionary2[str] is not null but DBNull, which then causes SharePoint 2010 to process the value anyway and generate a nasty <Query><Where><IsNull><FieldRef Name="FieldToReceiveFilter" /></IsNull></Where></Query> in the SelectCommand of the underlying DataSource

Unfortunately, DataFormWebPart.ApplyReceivedFilterData is not virtual, so sub-classing it does not seem to be an option. Not sure if there is a good workaround for this. I verified this behavior in both RTM and December 2010 Hotfixed versions of SharePoint 2010.

Edit: Just checked the SharePoint 2007 DataFormWebPart.ApplyReceivedFilterData() in Reflector and lo and behold, they did check for DBNull in 2007:

        Label_0046:
            if (current[str] is DBNull)
            {
                goto Label_006D;
                // Note by myself: Label_006D is the call to MoveNext, aka. continue
            }

A little Eggtimer script using Growl

The quasi-standard notification system for Mac OS X is Growl, and one of the extras included in the download is the growlnotify app that lets you push notifications from shell scripts. Then, there is the at command which allows to execute a job at a given time. It’s essentially another way to modify the crontab, that doesn’t use the crontab directly.

So with a notification system and a scheduler at our disposal, I thought it would be good to write a little eggtimer script so that I can specify “Remove Pizza from oven in 10 minutes”. I’m sure there are tons of graphical tools available, but I prefer a simple shell script.

The growl side is easy, this command needs to be executed:

growlnotify -t "Reminder at 03:54" -m "Remove Pizza from oven" -s -a iCal

The -t parameter specifies the title (even though the manpage incorrectly states “this does nothing”), -m is the message, -s means it’s sticky (so it doesn’t disappear unless explicitly closed) and -a specifies the icon to use, in this case the iCal icon.

The at command is used to create jobs that run at a specified time. However, it requires another “service” to run, called atrun. By default, this is disabled. Check the atrun manpage for the command to enable it, on my OS X 10.6.6 it’s

sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist

I found the syntax of at a bit weird as it kept complaining about garbled or incomplete time. It actually needs to read a whole input from stdin rather than just accepting a single command. So the invocation looks like this:

at now + 10 minutes << eol
growlnotify -t "Reminder at 03:54" -m "Remove Pizza from oven" -s -a iCal
eol

That works, but of course it's WAY too much to type for us lazy people 🙂 So let's wrap this in another shellscript that accepts two parameters: A number for "Remind in X minutes" and a message:

#! /bin/zsh
function verifyminutes () {
  echo -n $1 | grep -Eq "^[0-9]+$"
  echo $?
}
function gettargettime() {
  date -v +$1M "+%H:%M" 
}
function getmessage() {
  # function to collapse arguments $2..$n together (skipping $1)
  # I'm pretty sure there is a better way of doing this
  local FIRST_PARAMETER=0
  for i;
  do {
    if [ "$FIRST_PARAMETER" -eq 0 ]; then
      FIRST_PARAMETER=1
      continue
    fi 
    echo -n $i" "; 
  }
  done
  echo "\b"
}

local ISNUMBER=`verifyminutes $1`
[[ -z $1 || -z $2 || "$ISNUMBER" -gt 0 ]] && {
  echo "usage: $0 minutes message"
  exit 1 
}

local TARGET_TIME=`gettargettime $1`
MESSAGE=`getmessage $@`
echo "Reminder set at $TARGET_TIME: $MESSAGE"

at now + $1 minutes >/dev/null 2>&1 << eol
growlnotify -t "Reminder at $TARGET_TIME" -m "$MESSAGE" -s -a iCal
eol

exit 0

Usage of this script is simple, just call eggtimer 10 Remove Pizza from oven and 10 minutes later you will get a Growl notification.

PS: If you are wondering about the getmessage function: Normally if you have spaces in an argument, you need to enclose the entire message in quotes. I wanted to get rid of the quotes, hence I'm collapsing all parameters after the first one together.

Eye-Fi Pro X2

A few weeks ago I decided that it was time to retire my trusty old Canon Powershot A410 Camera. It’s still a good camera (although low-light sucks), but I wanted something with a bit more zoom and resolution, and preferably RAW and manual Shutter/Aperture Settings. I ended up buying a Panasonic Lumix DMC-FZ35.

Now, the camera is nice, but has one major drawback: A non-standard USB Port which requires the special Panasonic USB Cable – this sucks. Also, I got a 16 GB SDHC Card but my existing USB SD Card Reader is an old one that does not support SDHC. Also, removing the card from the camera for syncing every time isn’t that great either as the camera does a complete reboot every time I open the battery cover to remove the card.

Scott Hanselman had a posting about his digital camera history and mentioned a special SD Card with Wi-Fi connectivity, made by Eye-Fi. Looking at their website, they have four different cards, but only one that supports RAW. I bought the Pro X2 on Amazon for about $99.

The card comes with a USB SDHC Card Reader, and the card conveniently contains the installation software for Windows and Mac OS X. You need to configure it on a Mac/PC first so that it connects to your Wi-Fi. Registration requires internet access, as you need to create an account with Eye-Fi and register your card with it – I’m not too impressed with that, as I think it’s unnecessary. I understand that certain premium features like automatic Geotagging should be restricted to an account, but I feel that the card should ‘just work’ without any account. I just hope Eye-Fi never goes out of business.

This account allows for some nice features though, if you choose to do so: You can automatically upload pictures to your account, even through hotspots. So if you are on the road and the card connects to a hotspot, it can automatically upload your pictures to Eye-Fi and you can then download it to your PC when you are at home again. Also, it can automatically upload photos to popular photo sharing sites like Picara, Flickr or Facebook.

On my Mac I use Picasa as my photo application, because iPhoto does not support the RAW format of my camera (neither does Preview). One of the nice things is that Eye-Fi Center allows me to configure separate directories for JPG and RAW photos (my Camera is set to save pictures as both RAW and JPEG), so I can put RAW into my Picasa watch folder. Unfortunately, automatic Geotagging does not work for RAW files, only for JPG so I might need to write a script that copies that info from the JPG to the RAW at some point.

The other great feature is Endless Memory. Basically if the card is filled to a certain percentage, it will automatically delete older photos, but only ones that were transferred to the PC before.

Overall, the card works just as I expect it to be: magically. The little Eye-Fi Helper app that runs in the background of my Mac automatically picks up pictures if the card is in my network, puts the JPG in one and the RAW files in another folder where Picasa picks it up. No more proprietary cable or removing the card all the time, it just works.

The only downside I can see is that 8 GB isn’t exactly a lot nowadays, but I can use my old 16 GB card as an emergency spare in situations where space really becomes a problem.