2

I am attempting to read a csv form the source https://stats.oecd.org/Index.aspx?DataSetCode=WILD_LIFE.

Let x be the url above.

 <script type="text/javascript">  
  var data_threatened = d3.csv(x)
   .then(function(data){
    console.log(data);
  })  
  </script>

However, when I run this script on my local host, I receive the following message:

Access to fetch at 'https://stats.oecd.org/Index.aspx?DataSetCode=WILD_LIFE' from origin 'http://localhost:8888' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Uncaught (in promise) TypeError: Failed to fetch

Is there a way around this block?

Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
Xenon
  • 311
  • 5
  • 12
  • 5
    If that server does not respond with the headers, your only choice is to use a server proxy of your own. You are running into a browser security feature that cannot be worked around. – Pointy Mar 04 '19 at 22:34
  • 1) Where can I read more on using server proxy's (in the context of my problem) so that I can better understand what you mean? 2) What does this look like in JS? @Pointy – Xenon Mar 04 '19 at 22:52
  • 1
    Basically you need code on a server you control that your page can access. Your server-side code will turn around and make HTTP requests to your target site. Nothing will care about CORS headers in that context. Code can be in any programming language you want. – Pointy Mar 04 '19 at 23:01
  • 1
    If you wanna stick to JS I can provide a simple [nodejs](https://nodejs.org) example or you can use something like https://github.com/ccoenraets/cors-proxy – rafaelcastrocouto Mar 04 '19 at 23:33
  • See the *How to use a CORS proxy to get around “No Access-Control-Allow-Origin header” problems* section of the answer at https://stackoverflow.com/questions/43871637/no-access-control-allow-origin-header-is-present-on-the-requested-resource-whe/43881141#43881141 – sideshowbarker Mar 05 '19 at 06:19

1 Answers1

1

Fetching the file and serving it from the same origin as the script runs would be the most easy. eg. PHP cares little about CORS (assuming that this file occasionally changes). In case it doesn't change, even manually placing it there would suffice.

To let a server-side script fetch the file and then serve it as same-origin, without caching it:

<?php
    $url = "https://stats.oecd.org/Index.aspx?DataSetCode=WILD_LIFE";
    header("Content-type: text/csv");
    echo file_get_contents($url);
?>

Any server-side scripting language should be capable of doing so; I used PHP just to provide an example. One could make WILD_LIFE a variable, so that it could be used to fetch any data-set.

Martin Zeitler
  • 1
  • 19
  • 155
  • 216
  • It responds with Uncaught TypeError: d3.csv(...).header is not a function, and then spits out the same cors information. @Martin Zeitler – Xenon Mar 04 '19 at 22:33
  • 1
    You cannot add the header from the client side; it's got to be done at the server. – Pointy Mar 04 '19 at 22:33
  • agreed. but the documentation is explicit about it... and the client blocks it, not the server. – Martin Zeitler Mar 04 '19 at 22:34
  • 2
    Yes, the client blocks it, but it does so because the HTTP response does not contain the required header. – Pointy Mar 04 '19 at 22:35
  • 2
    @MartinZeitler The documentation you link says this: "By default, most browsers do not allow cross-domain requests. To enable cross-domain requests, **have the server** set the header Access-Control-Allow-Origin: *" Bold mine. – Daedalus Mar 04 '19 at 22:36
  • 1
    fetching the file and serving it from the same origin as the script runs would be the most easy. eg. PHP cares little about CORS (assuming that this file occasionally changes). – Martin Zeitler Mar 04 '19 at 22:36
  • This doc you liked is not the official one, and it's outdated: D3 uses promises (look at OP's question) now, not XHR. – Gerardo Furtado Mar 04 '19 at 22:42
  • @GerardoFurtado that link had been removed ages before you commented. voting to delete a workable answer is also questionable... because it aims at providing a solution, despite not in JS. it's in every case rather likely to work out, than to ask at `OECD` to have the required header added. – Martin Zeitler Mar 04 '19 at 22:47
  • When you say let a server-side script fetch the file and then serve it as same origin, are you referring to the use of a proxy so that the request header origin aligns with the response access-control-allow-origin header? @MartinZeitler – Xenon Mar 04 '19 at 23:10
  • When you say One could make WILD_LIFE a variable, so that it could be used to fetch any data-set, what do you mean? Were you referring to the query string parameter? @MartinZeitler – Xenon Mar 05 '19 at 00:40
  • @MichaelRamage when you proxify the request with a remote sever, you have to set the `Access-Control-Allow-Origin` header, but when requesting it with your localhost, then not (because it's already "same-origin"). and what I've meant with the variable is, that a single such script can be used to fetch just any data-set (instead of hard-coding it, which would require 1 local script per remote file). the `CSV` could even be parsed and transformed, in case it needs adjustments to suit `d3`; see https://en.wikipedia.org/wiki/Extract,_transform,_load (the transform is optional). – Martin Zeitler Mar 05 '19 at 00:47
  • @MichaelRamage `ETL` is not exactly the same - but exactly the same principle applies - because fixing the headers counts towards transformation, which prevents loading the data into a chart (instead of a database). – Martin Zeitler Mar 05 '19 at 01:01