My words on free/open source software

Sunday, October 09, 2011

Using rsync to migrate data between Linux and Apple Mac OS X

I got a new MacBook Air running Mac OS X (10.7.1) "Lion", and I managed to migrate my archive data (around 1.3TB) from an old Linux laptop running Debian Testing "Wheezy." The process was not smooth and I hit several blocks. Here I document several points that you need to pay attention to when migrating data across operating systems:


Use the latest rsync (3.0.8 from MacPorts was used here). The old version of rsync is buggy on Mac. The version of rsync on Linux doesn't matter much because rsync on Linux has been quite stable since long ago.


Disable "Ignore ownership on this volume" (you can use Finder or from command line). If not rsync won't be able to set file permission correctly. I think if you don't instruct rsync to preserve permission and mode then you might not need to do this but I'm not very sure.


Disable antivirus software like McAfee because they interfere with rsync's time preservation.


Use rsync's "--iconv=UTF8-MAC,UTF-8" option (only available after rsync 3.0), because Mac's HFS+ decompose UTF-8 before storing it so the filename would be different from that of Linux if no conversion is done right.

If you saw "iconv_open("UTF-8", "UTF8") failed", try swapping the parameters of iconv.


This is not directly related to rsync nor Mac, but a general rule for handling archive data: use another piece of software to do the verification. Your rsync might be buggy, the OS might be buggy, the disk or USB cable might be kaput, you might have used the wrong options or settings. Therefore you should always use another piece of software other than rsync to verify the data before and after the migration. I used AIDE to do file checksum and time verification.

Wednesday, February 16, 2011

openSUSE Build Service (OBS): Break the Link Between a Branched Package and Its Source

We are using openSUSE Build Service (OBS) for the development of MeeGo. I have an old package, which was branched from a Trunk package long ago. But every time the source pakcage in Trunk is updated my package would be broken because the changes between them are so large that it's become meaningless to keep the link between them any more. So I think I should just remove the link between them.

The simplest way may be just removing my package and start a new package from scratch, however, this action not only removes the package history but will also reset the cumulative release version number, which is unacceptable since that would make it impossible for client machines to follow updates to this package.

I have carried out a long search but failed to find a hint to this issue, and after some trail-and-error I think I have found a way:

  1. Check out the project's unexpanded files by using "osc co -u PRJ/PKG"

  2. Remove the "_link" file by running "osc rm _link"

  3. Copy in the latest package files since the link is now broken

  4. Check-in the changes

I've done a test and the steps above worked. However, it seems that by those steps the release version number will also be reset.

Update: darix from freenode/#opensuse-buildservice said 'osc in git has "osc detachbranch" exactly for that.'

About Me

My photo
Santa Cruz, California, United States