5

What's the proper way to log any errors or warnings when performing a quiet rsync?

This is what I currently run from my crontab:

gsutil -m -q rsync -r -C /mount1/share/folder gs://my-bucket-1/folder/ > /mount2/share/folder/gsutil.log

Since the log file is always completely empty and I'm uploading terabytes of data I'm starting to think that maybe even errors and warnings are being supressed.

fredrik
  • 8,747
  • 13
  • 64
  • 117
  • 1
    Errors are still reported with `-q` but they should get sent to stderr instead of stdout. Can you redirect stderr as well? In general though, you shouldn't receive many errors, if any. – jterrace Dec 09 '14 at 16:34
  • Isn't it just too good to be true not to get any errors? ;) How about `command 2>&1 >> gsutil.log`. Is that what you mean? It's hard to know if it is really working since there is absolutely nothing being logged to the file. – fredrik Dec 10 '14 at 13:03
  • If you want progress information, why not remove -q? – jterrace Dec 10 '14 at 15:06
  • If possible, I only want errors and warnings logged since I'm transferring a large amount of files. The log would get enormous if everything (INFO level) would get logged. – fredrik Dec 10 '14 at 15:08
  • I simply removed the -q option, and noticed that nothing gets logged to that file I'm piping output to. All INFO level messages are printed in my terminal. I wonder what I'm doing wrong... – fredrik Dec 10 '14 at 15:50
  • I think you've got your redirect set up wrong. You want `command >> gsutil.log 2>&1`. – jterrace Dec 10 '14 at 16:38
  • 1
    Ah, yes – you are right about `command >> gsutil.log 2>&1`. Thank you. Still not getting anything in the log when using -q though... but I guess that's a good thing! :) – fredrik Dec 11 '14 at 14:55

1 Answers1

8

After having realized that this is related to how you pipe stdout and/or stderr to files in general, the answer really lies within this existing thread: How to redirect both stdout and stderr to a file

So a simple solution to log as much as possible into one single log file could be something like:

gsutil -d rsync [src] [dst] &> [logfile]

...where -d enables debug output. I found this to be the only way to show files which were affected by an error such as CommandException: 3 files/objects could not be copied. Please note that -d exposes authentication credentials.

Community
  • 1
  • 1
fredrik
  • 8,747
  • 13
  • 64
  • 117