How can I get data from a JSON file that is on a different domain with no headers?

Question

I am trying to get data from AWS JSON. It is not on my domain and I can't get the data. The error that chrome gives is below Could anyone review my code and give me some tips?

Second, I was using Insomnia just using the link and was able to get the specific data point that I wanted using $.0.Ed0320. How do I translate this into JavaScript.

I have tried using a xmlHttpRequest.

Here is the JSON:

[
  {
    "Ed0320": "8.010886",
    "TmStamp": "2019-08-07 15:15:00",
    "Ed0340": "21.15973",
    "Ed0305": "0.2966875",
    "Ed0313": "3.344086"
  },
  {
    "Ed0320": "6.761719",
    "TmStamp": "2019-08-07 15:10:00",
    "Ed0340": "17.47292",
    "Ed0305": "0.2349026",
    "Ed0313": "2.789934"
  }
]

Here is my XML:

function reqListener() {
     // parse the the data
     var data = JSON.parse(this.responseText)
     // create an array from the required data (result.Stats -> watch the capital 'S')
     var mappedData = data[0];
     // display data
     document.getElementById('demo').innerHTML = mappedData.join()
}
function loadData(url) {
     var oReq = new XMLHttpRequest();
     oReq.addEventListener("load", reqListener);
     oReq.open("GET", url);
     oReq.send();
}

Chrome gives the error of Access to XMLHttpRequest at <fileName> from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https.

function calcUVI(Ed0305, Ed0313, Ed0320, Ed0340){
                total305=(Ed0305*0.8058)
                total313=(Ed0313*0.0887)
                total320=(Ed0320*0.0324)
                total340=(Ed0340*0.0131)
                UVI = total305 + total313 + total320 + total340
                return UVI
            }

Also, I would like to change the url based on this function below. Is there a way to input this into the fetch?

function changeText(){
                var d = new Date();
                var year = d.getFullYear();
                var month = d.getMonth() + 1;
                var day = d.getDate();
                if (month <= 9){
                    month = '0'+month;
                }
                if (day <= 9){
                    day = '0'+day;
                }
                var dateTotal = year + month + day;
                url = "https://cors-escape.herokuapp.com/https://tepfsail50.execute-api.us-west-2.amazonaws.com/v1/report/metguv?rptdate=" + dateTotal;
                console.log(url);
            }

Thank you in advance for your help. I am really new to all of this.

Answer 1

1. Fetching the data

I'm not quite sure, what you want to achieve, but here's a snippet, that

1. Makes a request to cors-anywhere.herokuapp.com to "proxy" the query of the "target" URL

2. Returns a Promise from the queried URL, so we can wait for it to resolve in other parts of the code

3. Filter and map the resulting data - I didn't understand what data would you like to get from the resulting array and how did you want to handle that data

// self calling function to wrap the await in
// an async function (so the await is not "top level")
(async function() {
  // waiting for the returned Promise to resolve
  // if you console.log() the data constant, then you'll
  // see that it's the full dataset (in JSON format) that you
  // got from simply calling the AWS URL
  const data = await fetchURL()

  // filtering the array for objects that have a key of "Ed0320"
  const filtered = data.filter(e => typeof e['Ed0320'] !== "undefined")
  // mapping the values of all "Ed0320" from the array
  const mapped = data.map(e => e['Ed0320'])

  // simple output, so you can check the values
  console.log(filtered);
  console.log(mapped)
})();

// fetch request using cors-anywhere
function fetchURL() {
  // creating the URL to be called:
  // first part is cors-anywhere, second part is
  // the real URL to be queried
  const url = 'https://cors-anywhere.herokuapp.com/https://tepfsail50.execute-api.us-west-2.amazonaws.com/v1/report/metguv?rptdate=20190807'

  // returning a Promise, as fetch() is an asynchron
  // function, so it's not resolved immediately
  return new Promise((resolve, reject) => {
    // using fetch() instead of the XMLHttpRequest()
    // fetch returns a Promise, so you can use .then() and
    // .catch() chained after it
    fetch(url)
      .then(r => {
        resolve(r.json())
      })
      .catch(err => {
        reject(err)
      })
  })
}

So, your code had CORS issues (CORS: Cross-Origin Resource Sharing), that had to be handled before you could get the data. I used the cors-anywhere on Heroku to overcome this obstacle. You can set your cors-anywhere service by installing the module from npm (here: https://www.npmjs.com/package/cors-anywhere).

2. EDIT: Processing the data

The answer is edited as the question was expanded with more information.

This new snippet does calculate the UVI based on the results from the API (the URL is generated dynamically based on current date).

Please note:

• you can make all your functions shorter (like don't create variables for values that you only need for one calculation, where you only return the result - e.g. current date string or the UVI calculation)

• the "creation" of the date string is modified: I used string interpolation and ('0' + d.getDate()).slice(-2) to force two digits long months and days

• when you create (design) a function try to keep it simple - it should do only one thing (so don't create a function that generates current date string and attaches it to the query string. Maintenance will be much easier this way.

• the calcUVI() function receives its parameters via destructuring the array element it's called with. Get familiar with destructuring - it's not that hard, and makes your code easier to read and understand. (You can start here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment)

(async function() {
  // modified fetchURL() - the URL parameter is passed to
  // the API query function
  const data = await fetchURL(urlToQuery())

  // calling calcUVI() function with the first data
  // entry of the data array
  const UVI = calcUVI(data[0])

  // displaying result in the console
  console.log("UVI:", UVI)
})();

function fetchURL(url) {
  return new Promise((resolve, reject) => {
    // the url parameter is received when the function
    // is called - it's not a constant anymore
    fetch(url)
      .then(r => {
        resolve(r.json())
      })
      .catch(err => {
        reject(err)
      })
  })
}

// creating and returning the API URL dynamically -
// date string is added
function urlToQuery() {
  // the query URL base is here - and the date string
  // at the end is created dynamically
  return `https://cors-anywhere.herokuapp.com/https://tepfsail50.execute-api.us-west-2.amazonaws.com/v1/report/metguv?rptdate=${getDateString()}`
}

// creating and returning the date string
function getDateString() {
  const d = new Date();
  return `${d.getFullYear()}${('0' + (d.getMonth() + 1)).slice(-2)}${('0' + d.getDate()).slice(-2)}`
}

// calcUVI receives the parameters destructured from
// the first element of the array (it's called in
// the main async function)
function calcUVI({
  Ed0305,
  Ed0313,
  Ed0320,
  Ed0340
}) {
  // no need to store these elements as variables, as the single
  // Ed... values are not used anymore in the process (a
  // total is returned, so just add them up)
  return Ed0305 * 0.8058 + Ed0313 * 0.0887 + Ed0320 * 0.0324 + Ed0340 * 0.0131
}