Cron and SSH public key authentication

I’ve got a bit of an odd problem. I’ve setup a shell script to rsync my home folder over to my file server via SSH. I’m using public key auth so I don’t have to enter my password for the file serve every time the script runs. When I run the script manually it works fine. However, when the cron task I setup to run the same shell script jumps into action I get a nice mail message from cron stating the server rejected the connection request due to too many authentication failures (I have sshd on the server setup to say no after two failures). I have the key in ~/.ssh, does cron run as a different user on 10.5? Do I need to move my keys to /etc?

The server is running Slackware Linux 12.2. The client is running OS X 10.5.6.

I used to use this in 10.4 to do backups, and it worked fine. I kind of let it lapse in 10.5 due to Time Machine, but I decided I’d like just a plain old copy of my files both off site and on another machine.

I’ve Googled and I can’t seem to find anything …