0

I have two API urls to call the first one is https://jsonplaceholder.typicode.com/todos I need to get this first url to retrieve the id. After retrieving, call the second url which is https://jsonplaceholder.typicode.com/todos/(id). I am using promise-based for this approach, but my problem is.

How to achieve this in fast retrieving the large number of data?

Note: I am using only Plain JavaScript and CDN for axios.


export const getData = () => {

    const API = `https://jsonplaceholder.typicode.com/todos`;

    return axios.get(API, {
        headers: {
            "accept": "application/json;odata=verbose"
        }
    }).then(res => {
         const data = [];

          const requests = res.map(val => {
            const id = val.id;
            var obj = {};

            const url = `https://jsonplaceholder.typicode.com/todos/(id)`;
            return axios.get(url).then(res => {

                obj['Result'] = res;

            });
        });
            return Promise.all(requests).then(() => {
            return data;
        });
    });
}

This code is working but it was slow getting the data and I need some suggestions for best concepts.

junreyd
  • 47
  • 9
  • How many items are in the first `res` array? – jfriend00 Aug 20 '19 at 02:43
  • FYI, your code shows `return posts`, but never shows putting anything into `posts`. – jfriend00 Aug 20 '19 at 02:44
  • Also, it doesn't look like your second API call is returning any data that isn't in the first API call or is this just make believe APIs here? – jfriend00 Aug 20 '19 at 02:48
  • @jfriend00 for your first question I have 3000 items and still counting in future in res array, FYI return posts is just only a variable not post method (I had already update my question), API is the same in first and second I need to get the first API to get the ID after that call the second API for the final result. Please let me know if this make sense of my question to you. – junreyd Aug 20 '19 at 03:12
  • Well, I don't know if this makes a big difference in your case or not, but you don't want to be launching 3000 requests to the same target host all at once. That will consume a lot of resources on both your node server and on the target server and very few target servers can actually productively work on 3000 requests at once. You should experiment with running between 5 and 25 requests at once to see what is fastest. – jfriend00 Aug 20 '19 at 03:13
  • See these answers for how to run only N requests at a time: [Promise all consumes all my ram](https://stackoverflow.com/questions/46654265/promise-all-consumes-all-my-ram/46654592#46654592) and [How to control how many promises access network at same time](https://stackoverflow.com/questions/41028790/javascript-how-to-control-how-many-promises-access-network-in-parallel/41028877#41028877). – jfriend00 Aug 20 '19 at 03:19

1 Answers1

0

The fastest way would be to not perform all the AJAX calls from a web browser.

Web browsers cap the number of simultaneous requests around 6–10 (from this post), so if you can perform a request a get a response in 200ms, you're still looking at a full minute of client-side requests.

If instead you built a server-side solution to aggregate the data, you could query your custom endpoint to retrieve larger chunks of data at a time.

If that isn't an option for you, then either way, the browser request limit will probably be your bottleneck.

romellem
  • 5,792
  • 1
  • 32
  • 64