How to copy folders with 'gsutil'?

Question

I've read the documentation on the gsutil cp command, but still don't understand how to copy folders to keep the same permissions. I tried this command:

gsutil cp gs://bucket-name/folder1/folder_to_copy gs://bucket-name/folder1/new_folder

But it resulted with the error:

CommandException: No URLs matched

Although, when I tried it with slashes in the end of each name, it didn't show any error:

gsutil cp gs://bucket-name/folder1/folder_to_copy/ gs://bucket-name/folder1/new_folder/

However, there was no new folder in the bucket when I checked with gsutil ls. What am I doing wrong?

Answer 1

You should use the -r option to copy a folder and its contents recursively:

gsutil cp -r gs://bucket-name/folder1/folder_to_copy gs://bucket-name/folder1/new_folder

Note that this will only work if folder_to_copy contains files. This is due to the fact Cloud Storage doesn't really have "folders" as one would expect in a typical GUI, it instead provides the illusion of a hierarchical file tree atop the "flat" name space, as explained here. In other words, the files within a folder are simply objects that have the folder prefix appended to them. Therefore, when you're doing gsutil cp, it expects actual objects to be copied and not empty directories which is something the CLI does not understand.

Another approach would be to simply use rsync instead, which tolerates the use of empty folders and also synchronizes the contents between source and destination folders:

gsutil rsync -r gs://bucket-name/folder1/folder_to_copy gs://bucket-name/folder1/new_folder

If you also want to preserve the ACL (permissions) of the objects, use the -p option:

gsutil rsync -p -r gs://bucket-name/folder1/folder_to_copy gs://bucket-name/folder1/new_folder