1

This new question is just because my old question was closed. So I can't even answer it.

Javascript fetch using a download URL from GCS

I seldom get into situation where I have to consider CORS so I forget about it. But here is what is happening.

  1. I get a download URL for a file on GCS, using fileRef.getDownloadURL().

  2. This URL works when opening it in a Chrome tab.

  3. However trying to use JavaScript fetch fails because of (bad) CORS.

  4. In principle this could be cured with gsutil cors set ... (see the question above).

  5. However because I am using the test environment (firebase serve, localhost) it does not seem possible to set a value for the origin in the request header that is accepted by gsutil.

So if someone knows a solution to 5 above I would be grateful. (I can use a workaround, of course, but I would prefer not.)

BTW, I tried to use the suggested proxy server, but it seems to be blocked here.

EDIT: I have also filed a bug report to Google. But it would be nice if I was wrong and this was not a bug in the interplay between Firebase local testing and GCS.

Leo
  • 4,136
  • 6
  • 48
  • 72
  • 3
    Far from being an expert, I know a bit about CORS. Issues like these are usually solved by letting your own server act as proxy for the download. CORS only comes into play when you try and fetch something from client-side JS, but as far as I can tell you have a backend in play here which should be able to download the file no problem. –  Jun 24 '19 at 21:06
  • @ChrisG Yes, I can use Node.js on the Firebase side to download the file to the server and then send it along to the client. But I would rather avoid it. Inefficient. More job. More mistakes... – Leo Jun 24 '19 at 21:17
  • 1
    So rather than implement a relatively simple, well-known solution, you want to implement a hypothetical, potentially complicated one? As far as I can tell, the `gsutil` route will only work for an actual host, not your `localhost` testing environment. And btw, what are these files in the first place? Why do you need to fetch them? –  Jun 24 '19 at 21:43
  • @ChrisG Because the proxy server I tried did not work (see my comment to the accepted answer). And yes, I need to fetch the files because they will be used locally in a PWA. – Leo Jun 25 '19 at 11:59

1 Answers1

1

Your question is: How to fetch content from GCS using a Javascript script from client side.

The question is a duplicate, since this has already been answered here. Please take a moment to actually read the answers, you will see is the same issue you are facing. You have an alternative that you can use without having to modify the bucket's cors policy or fetching from server side:

  1. Using a Heroku proxy server (I have tried this one using the online demo and it worked for me)

notes:

  • This solution's implementation is shown on the question I refer at the beginning along with their explanation.
  • Be sure that the object you are trying to access is public.
  • You could also use the "{ mode: 'no-cors'}" and it will retrieve the content but you won't be able to access to it through the client script since the response will be "opaque".
Mayeru
  • 1,044
  • 7
  • 12
  • Thanks, yes that proxy server actually works for me too. (The one I tried before didn't. And as I said I did read the answers. But I have no idea why this proxy works and the other did not. If you have, please tell me.) – Leo Jun 25 '19 at 11:57
  • 2
    Relying on a 3rd party service that could stop working at any point in the future doesn't sound like a suitable solution to me. This is a kludge at best. Plus, if a question is a duplicate, please mark it as such rather than answering it. –  Jun 25 '19 at 12:29
  • 1
    Leo, I could not say, I haven't look at the internal code of them. @Chris, on the link I provided, the person who answered also provides simple steps to launch your own Heroku proxy server so you don't rely on the proxy server. You are correct, the question should be marked as duplicate, but it already was and it caused OP to create a new question (because they didn't explain why it was considered duplicate) hence, the reason why I posted. But I will do edit my answer to mark it as CW because I forgot to do so before. – Mayeru Jun 25 '19 at 14:31
  • @ChrisG I explained why it was not simply a duplicate. There were two reasons: 1. The answer did not work. 2. I wanted to be sure if this was a bug in Firebase or not. – Leo Jun 25 '19 at 22:26
  • @Mayeru It was the same kind of proxy server as the one that you pointed to. (So it is not the code.) – Leo Jun 25 '19 at 22:27
  • @ChrisG And you are right, of course. Relying on a 3rd party service is not good. (That is perhaps a third reason why I do not think it was a duplicate. But someone insisted it was.) – Leo Jun 25 '19 at 22:29
  • Actually I think this is just a bug in Firebase/GCS. Someone did not think really careful about download URL:s (which are marked with a security access token) and CORS. The answers here unfortunately probably makes it much more difficult to get this point through to Google Devs. (I don't have much time for it.) – Leo Jun 25 '19 at 22:31
  • @Leo This isn't a bug, not even remotely. The fact that client-side JS can't download arbitrary files from other servers is well known. CORS limitations were implemented to protect the user, not to annoy the developer. Again: you have a server which can easily download and forward resources from your GCS. This is a non-problem, period. –  Jun 25 '19 at 23:06
  • @ChrisG Sorry, this does not seem to be your area. – Leo Jun 25 '19 at 23:46
  • 1
    @Leo Yes, my bad; my area is constructive problem-solving, and this is clearly way outside that. I won't bother you any more. –  Jun 26 '19 at 00:22
  • @ChrisG Excuse me for being a bit rude. You missed the problem here. Google/GCS is protecting me from myself by using CORS in a way no one did expect. The download URL I get from GCS is with credentials and ment to be used by me, the user, any way I want. However the use of CORS prevents that. Which is surely not what Google/GCS intended. – Leo Jun 26 '19 at 09:51
  • 1
    @Leo No problem; this doesn't change my answer to your problem though. [This page](https://cloud.google.com/storage/docs/reference/libraries) lists the client libraries, and strongly supports my suggestion that you're only supposed to download something off GCS as intended, i.e. from your *server*, in which case CORS is a non-issue. The fact that you want to authenticate a client-side download implies that anybody visiting your site can grab the private credentials you're supposed to use. Google simply expects you to route downloads through your server, as usual. –  Jun 26 '19 at 11:41
  • @ChrisG The URL goes to the data file produced for the user. Only the user can get the download URL (from the server) so there are no additional security issues. The URL contains an access token, without that token it does not work. It makes no sense to me that Google blocks me from using it in the web browser, they could just send an "Allow-Control-Allow-Origin: *" when the URL includes the access token. But yes, I can send the file through the Node.js server but that is an unnecessary overhead (since the data is going to be used in the PWA, in the users web browser). – Leo Jun 26 '19 at 16:54
  • @ChrisG Here is the doc for getDownloadURL: https://firebase.google.com/docs/reference/android/com/google/firebase/storage/StorageReference.html#getDownloadUrl() – Leo Jun 26 '19 at 17:34
  • 1
    @Leo Sorry, I have exhausted all the ways I can tell you that you're wrong without flatly saying so. The very doc you've linked to is part of the Android API docs, i.e. *not* client-side JS. I frankly don't care that you think Google is wrong; they aren't. Setting the ACAO header and putting the token in the URL is precisely what would enable *anybody* with the URL to download the file, the very thing Google wants to prevent by using the access token in the first place. Think about all the time you've wasted trying to bash your head through this wall. At which point is it enough? –  Jun 26 '19 at 20:23
  • @ChrisG Ah, thanks for pointing out that this was the wrong documentation. This made me search for the web docs instead and I found this: https://firebase.google.com/docs/storage/web/download-files And what did I do? I believe I did exactly what it says. Now I tried again. And it works. o.O So we were both right and wrong. – Leo Jun 26 '19 at 21:20
  • Maybe I should say one more thing. I believe I must have made some typo, but I am not quite sure. However some days ago I noticed a rather similar access problem in a PWA using direct links to Google Photos. Since this is hard coded (no privacy issues there) I know the problem was there. I thought about it, googled to see if I was doing something that Google tries to prevent. And then I started to debug it. Saw the issue some times, but now I have not seen it any more. – Leo Jun 26 '19 at 21:33