GCP How to copy files automatically from Project A to Project B every monday?

Question

GCP is a new thing for me but i want to know if it's possible to copy a specific file (e.g : myFiles.csv) from a bucket in the project A to a bucket in the project B every monday at 6.00 am ?

I need it because myFiles.csv is overwritten every Monday and i need to share it with the project B.

Can you have a look at a transfer service - https://cloud.google.com/storage-transfer/docs/overview are there any reasons it cannot be used in your case? — al-dann, Oct 14 '22 at 07:47

Mazlum Tosun · Accepted Answer · 2022-10-20T22:32:53.490

1

You can use Storage transfer : https://cloud.google.com/storage-transfer/docs/create-transfers#google-cloud-console

With this service, you can select the source and destination bucket and scheduling options, every day in your case.

Source bucket project GCP A :

In this example, I selected a folder team_league in a bucket called mazlum_dev

In the prefix field, I added the name of the file I want to transfer input_team_slogans.json

You have to put your file name for your job.

Destination bucket project B :

You have to select the output folder of your destination bucket.

Sheduling options :

You can also use the GCloud sdk if needed with gsutil :

gsutil cp gs://your_bucket_project_a/your_file gs://your_bucket_project_b/output/

But you have to find a way to cron this script every day, that's why I recommend the first solution because everything is native and integrated for your need.

edited Oct 20 '22 at 22:32

answered Oct 14 '22 at 09:15

Mazlum Tosun

5,761
1
9
23

Your answer is off to a good start but does not show how to copy a specific file as specified in the question. – John Hanley Oct 14 '22 at 12:31
Thanks John, I didn't want my bucket names appear in the image, but you are right, I am going to add this detail in my answer. – Mazlum Tosun Oct 14 '22 at 12:55
@JohnHanley I updated my answer, don't hesitate if you have other propositions. Thanks again – Mazlum Tosun Oct 14 '22 at 13:19
Thank you @Mazlum Tosun It seems that it's exactly what i need. I will try this solution by monday at the latest. – LearnLearnLearn Oct 14 '22 at 13:24
Thank you @Mazlum tosun, it's the perfect solution that i needed ! – LearnLearnLearn Oct 17 '22 at 07:49
Great :), I am glad it's worked for you – Mazlum Tosun Oct 17 '22 at 08:04

Hemanth Kumar · Answer 2 · 2022-10-14T09:20:14.497

Follow below steps :

Click on this Web console link Storage > Transfer to create a new transfer. Then Select the source bucket you want to copy from ex.Project A. So once you go to the destination part of the transfer form, you can write/paste the target bucket (Ex. Project B) right in its text input. Even if that bucket is from another project. It will show you a green icon once the target has been verified as being an existing bucket. You can continue the form again to finalize your setup.

Once you start the transfer from the form, you can follow its progress by hitting the refresh button on top of the console.

As the bucket identifiers are globally unique (this is key to the solution).

Refer this SO Link for more information.

GCP How to copy files automatically from Project A to Project B every monday?

2 Answers2