Rsync

Linux Support
1

Rsync has been regularly used to backup files in the home directory on a Linux desktop to a FAT32 USB stick.

Recently rsync has stalled due to problems with Service Worker and Local Storage files being transferred from
the home directory to the stick.

These files were deleted and rsync then rerun.

Rsync then stalled and produced many error messages of the form :-

rsync: chgrp
failed:
Operation not permitted (1)

where represents a filename.

The rsync command is of the form :-

rsync -v -a --max-size=4GB --delete .
.

where represents the home directory and represents
the mount point.

I realise that FAT32 knows nothing about permissions, etc.,
but the above command has worked successfully on previous occasions.

Why are the above error messages appearing ?

Do they have anything to do with the Service Worker and Local Storage files which had been created on the
USB stick in the previous run of rsync ?

What is the best thing to do to get rsync working okay again ?

N.B. I don’t want to change the rsync command in any way.

Would it be okay to delete all the files currently on the stick and then rerun rsync ?

2

Hi,

I wonder if you have any large files in your home directories and if you’re exceeding the available space or your defined RSYNC limit or just the limitations of FAT32 (different file types have difference specifications/limitations). As you say the RSYNC command looks OK and I can run it on Debian10 to a VFAT USB (but I have almost nothing in my HOME, so that’s far from a comparable test)

Can you confirm you HOME partition and file type - “df -Th ~” and also check if there are any files larger than or close to 4gb (being the limit for FAT32)

If you have a spare USB, I’d format it to whatever your HOME is, which should reduce those incompatibility risks.

Is that possible? - let me know the results…

Regards deleting; if it’s solely a backup for you “home” and you intend to recreate it anyway - then that sounds fine (BUT, since I don’t know your requirements/intentions, I’ll strongly recommend you copy that data and ensure its safe first before doing anything that may risk your data).

…but on paper using RSYNC to copy to an USB drive, that should be fine…

3

Freelance,

you give your command as:

rsync -v -a --max-size=4GB --delete .
.

I note three points:

  1. you have two lines here - is that how you issue the command?
  2. You show two full stops (.). They have a particular meaning in Linux (it means your current directory) so the first line would copy your Home directory (if that’s where you are) to the home directory.
    I am guessing that your actual command is
rsync -v -a  --max-size=4GB --delete <home directory> <mount point>

Is that the case?

  1. Do you really want to back up the whole of your Home directory? Including all the system files? Mine comes to 64GB. If you do, you might be better off using the tar command which saves the whole lot as a single file which can be compressed, too. Its rather fiddly to use which is why I use rsync but I back up only the important directories (Documents, Pictures, etc) and you can put them all on the same line like so:
rsync -av --delete --max-size=4GB Documents Pictures Music .bin etc  <mount-point>

Keith

4

Thanks very much for your replies.

My answers to your questions, Keith :-

(1) The rsync command is entered on the computer as a single line
(2) Both full stops are intentionally part of the command
(3) The whole of the home directory is required to be backed up.

I am now suggesting a different procedure to try and overcome the rsync hang :-

Rsync has very recently been run okay on a new FAT32 stick, using the command i previously reported
(the -a option is retained).

(a) I propose to put all the files on the current new stick
into a new directory (using mv . backup-dir) outside of my
home directory and then run rsync again on the stick.

Presumably the new run of rsync would delete backup-dir
(as this doesn’t exist in my home directory) and then just
copy over all the home directory files again to the stick.

Would rsync run okay on the same stick using step (a) on a periodic basis ?

5

Hi Freelance.

I am confident that the use of full stops as shown in your previous post won’t work, but as you are proposing a new method I’ll address that.

I propose to put all the files on the current new stick into a new directory (using mv *.* backup-dir) outside of my home directory and then run rsync again on the stick.
The [b]mv[/b] command will [u]delete[/u] the files from the source (while transferring them to the destination). You need to use the [b]cp[/b] or [b]rsync[/b] command. You don't need [b]cp *.*[/b] - [b]cp *[/b] will do.
Presumably the new run of rsync would delete backup-dir (as this doesn't exist in my home directory) and then just copy over all the home directory files again to the stick.
No: When you backup a complete [u]directory[/u], rsync will remove from the backed-up directory any files that have been deleted from the corresponding source directory (hence the [u]sync[/u]). "backup-dir" will remain on the USB stick.
Would rsync run okay on the same stick using step (a) on a periodic basis ?
Yes, if you use [b]rsync[/b].

Keith

6

Hi,
Just a couple of comments on pattern matching, as Keith pointed out, although Windows pattern matching “looks” like Linux/bash pattern matching, there are some subtle differences. If you try the following;


echo *
echo .*
echo *.*
  • You will see that “*” matches all files (that do NOT begin with ‘.’)
    • “.*” will match all files that DO begin with “.”
    • And “.” will match all files that contain a “.” (but do not begin with a dot)
      :slight_smile: