3

I have been testing rsync and found a weird bug when copying files and folders from a partition to another:

If I do this:

rsync -avE --delete '/Volumes/disk1/origin/' '/Volumes/disk2/destination/'

It copies/syncronizes correctly.

The next time I use rsync for the same syncronization, some modification dates in files (not folders!) become incorrect (are changed to the current date and time), even though I have used the -a in the rsync command which should preserve it.

The most weird thing is that if I redo it, the dates that were wrong are correct now, which means rsync is changing the modification dates every second time it is run, and when it changes the dates, it is always to the same files, I don't see a pattern other than just affecting files and the same files.

What am I doing wrong and can this be fixed?

This is with OS X 10.9.5, using the terminal, rsync 2.6.9

jackJoe
  • 658
  • 1
    What are the filesystems on these two volumes? FAT has a notorious design issue where modification times can only be represented at a 2-second resolution. rsync doesn't always handle this well. – Chris Harrington Jun 16 '15 at 15:39
  • @ChrisHarrington both are with Mac OS Extended (Journaled), and with a standard GUID Partition Table. – jackJoe Jun 16 '15 at 16:12
  • 3
    See comments here rsync 2.6.9 has a known bug on modification times - get a newer version – mmmmmm Jun 16 '15 at 16:58
  • @Mark I suspected that rsync might have to be updated, wich I have done for a test (updated to version 3.1.0) but the creation date (in the case of this new version) is not retained... So part of the problem is solved, now I have a new one. Maybe I need to use different options and not just the -avE? – jackJoe Jun 16 '15 at 17:42
  • @jackJoe creation date always changes when a file gets a new inode ( well, a Catalog node ID in hfs). I'm assuming that you are running a 64bit kernel in which case birthtime should be preserved. Did you compile rsync yourself ? – fd0 Jun 17 '15 at 17:13
  • @fd0 yep, 64bit kernel, and I compiled the new rsync. Copying with the finder (drag and drop) retains everything as desired. Also this new issue is that the creation date becomes the same as the modification date (in this new case the modification date is correct). – jackJoe Jun 17 '15 at 20:33
  • @jackJoe I can reproduce some of your observations. For clarity, 64bit timestamps consist of access-modification-creation-birthtime. In Finder terms birthtime is creation time. An initial run of rsync -av SOURCE DESTINATION preserves access modification and birthtime- creation time becomes the time rsync created the inode. If files are modified in the SOURCE folder on the next run of rsync the birthtime of the DESTINATION file is changed to the creation time of the SOURCE file. - OS X 10.6, 64bit with rsync 3.1.1. I think this is an issue with OS X not rsync, will look in to it further. – fd0 Jun 18 '15 at 12:08

1 Answers1

2

Let me correct my comment: A 64bit timestamp consist of access-modification-change-birthtime.

From man 2 stat the following system calls change the respective times.

The time-related fields of struct stat are as follows:

 st_atime         Time when file data last accessed.  Changed by the mknod(2), utimes(2) and read(2) system calls.

 st_mtime         Time when file data last modified.  Changed by the mknod(2), utimes(2) and write(2) system calls.

 st_ctime         Time when file status was last changed (inode data modification).  Changed by the chmod(2), chown(2), link(2), mknod(2), rename(2),
                  unlink(2), utimes(2) and write(2) system calls.

 st_birthtime     Time of file creation. Only set once when the file is created. This field is only available in the 64 bit inode variants. On filesys-
                  tems where birthtime is not available, this field holds the ctime instead.

Tools such as cp,ditto, and pax can preserve OS X metadata when they are called to copy files. These tools will not preserve birthtime if the modification time is newer than the original file's birthtime. Birthtime of the new file is set to the modification time of the original file.

If you compile rsync with the patches fileflags, crtimes, hfs-compression then rsync can handle OS X metadata and preserve the original file's birthtime on the new file.

So, you would call rsync like this.

rsync -avXN --delete SOURCE DESTINATION

I suggest that you have a long read of the rsync manual and understand the options that I have suggested before you attempt to apply them.

fd0
  • 10,708
  • Following your suggestion and this tutorial to upgrade rsync https://selfsuperinit.com/2014/01/04/an-updated-rsync-3-1-0-for-mavericks/ I used the rsync -avXN --delete SOURCE DESTINATION and it works great, all the modification dates, creation dates, etc, are retained, so problem solved, thanks! – jackJoe Jun 22 '15 at 15:49
  • -N is not an option. Did you mean -n for dry run? However, this still does not work for me. macOS Sierra 10.12 with rsync 3.1.2 (manually updated). Creation date on the destination is copying the modification date on the source. – lukejanicke Oct 30 '16 at 11:07
  • @lukejanicke- Did you apply the patches that I mentioned? If you did then the -N option should be available. From rsync -h:-N, --crtimes preserve create times (newness). – fd0 Oct 30 '16 at 11:26
  • @jackJoe The link you shared is broken. I assumed it was just a tutorial to upgrade. I manually upgraded to rsync 3.1.2 and there is still no -N in the options. Did that tutorial contain anything more than upgrade instructions? – lukejanicke Oct 30 '16 at 12:39
  • I found this http://romej.com/2016/04/rsync-backups-on-os-x/ which explains how to include the patches, which I didn’t realise were seperate. – lukejanicke Oct 30 '16 at 13:51