Get large amount of data using ajax

Question

I have the following ajax code which am using to get the user details from the server. Am generating the html table from the server side and return the structure as a string. This works fine but i get undefined when user records are too many. What is limitation of data that can be passed in ajax ? Is there an alternative i can use to generate html table in the client side ?

 var param = {};
    param.CompanyID = $('[id*=txtCoID]').val();
    $.ajax({
        type: 'POST',
        url: 'AjaxAsync.aspx/BindActiveUsers',
        beforeSend: function () { },
        data: '{P: ' + JSON.stringify(param) + '}',
        contentType: 'application/json; charset=utf-8',
        //processData: false,
        //timeout: 1000000,
        //async: true,
        //cache: false,
        dataType: 'json',
        success: function (rsp) {
            document.getElementById('dvUsers').innerHTML = rsp.d;
        },
        error: function (error) {
            document.getElementById('dvUsers').innerHTML = error.d;
        }
    });

which variable exactly is showing as undefined? For instance, I know for a fact that, if your code is going into the "error" callback for any reason, then `error.d` does not exist - check the jQuery ajax docs for what fields are actually present in that object. If you _are_ going into the "error" callback, what's the error being returned by the server (you can see it in the browser's network tab, in the developer tools)? And also, how many records is "too many", exactly? How have you determined this? — ADyson, May 03 '17 at 11:00
@ADyson The `undefined` is in the 'error' but when i add a limit to the records server is returning, code runs error free — bwanamaina, May 03 '17 at 11:41
ok so try this in your "error" function: `document.getElementById('dvUsers').innerHTML = error.status + " - " + error.statusText;` (as per http://api.jquery.com/jquery.ajax/). If you can find out the error being reported, you might be able to do something to fix it, rather than having to impose a limit on the number of records returned. Like I said, you can also find out the error from your browser's console and/or network tab - did you do this? If you didn't know this before, please try it - it's a vital debugging skill. — ADyson, May 03 '17 at 12:09
ok so then it's a server-side error (that's what the 500 status code means - you can google the list of HTTP status codes easily). You'd have to set breakpoints or add error logging to your server code to find out what is going on. — ADyson, May 03 '17 at 13:38
@ADyson server side code now is fine, when i step into the server side code it works perfect and returns the data, but ajax on the client side now displays `0 - undefined` any idea ? — bwanamaina, May 04 '17 at 13:39
no, mainly because I don't know what you've changed in order to get to that situation. I would check your console for errors as well btw. — ADyson, May 04 '17 at 13:41
I only changed the config file as advised in answer below,i used to get error 500 coz of timeout, am using an aspx project — bwanamaina, May 04 '17 at 13:47
any errors in the browser console? Any other clues in the network tab entry for the ajax request (e.g. a response code, or some other info in the response body?) Or maybe the request did not fire for some reason? All I am describing here is basic debugging which you should do every time you encounter this kind of problem. — ADyson, May 04 '17 at 13:52
No errors in browser console, suggest the request is fired coz when i limit records to be returned by server, my code works perfect, am generating a html table with around 3000 rows. — bwanamaina, May 04 '17 at 13:55
"suggest the request is fired". Guessing is no use. Please verify whether the request is fired by using your network tab, and if it is, also provide details of the code and response body returned (if any), as I previously requested. — ADyson, May 04 '17 at 14:11
Since this is aspx you can easily also set breakpoints and/or log any exceptions on the server side so you can see whether the response is generated correctly by the server. It will help us isolate the point at which the problem is happening. — ADyson, May 04 '17 at 14:13
Maybe you should have a look at [this](https://stackoverflow.com/questions/5225597/set-timeout-for-ajax-jquery) SO question. — Jeroen Heier, May 27 '17 at 04:24

score 2 · Accepted Answer · edited May 16 '19 at 00:52

The problem was caused by the request timeout but not the size of the data. Since I was using ajax updatepanel in aspx project I added AsyncPostBackTimeOut='300000000' to my ToolkitScriptManager tag and added

<system.web.extensions>
 <scripting>
  <webServices>
    <jsonSerialization maxJsonLength="300000000" />
  </webServices>
 </scripting>
</system.web.extensions>

to my Web.config file as documented here. Now I can load the data without any problem though it's taking some time depending with the number of records returned. Thanks for your help.

score 1 · Answer 2 · answered May 03 '17 at 10:59

1

There's no maximum data size imposed by HTTP. The limitation you're facing is imposed by your web-server so you'll need to increase the cap.

Another solution could be to partially build the HTML table and create a button that when pressed loads the rest (or use an EventListener to capture the page scroll position and keep loading while the user scrolls). If this approach meets your requirements, you shouldn't face any kind of data limitation.

answered May 03 '17 at 10:59

LunarParks

148
6

1

Talking of partially building the html table, how will i append the content to the already loaded data to end up with one table ? – bwanamaina May 03 '17 at 11:12
Upvoted this solution because it eases the load on your server and provides faster access to small amounts of the data, assuming it confines to your requirements. See this question regarding appending rows http://stackoverflow.com/questions/171027/add-table-row-in-jquery. – Matt Goodrich May 03 '17 at 11:15

score 1 · Answer 3 · edited May 23 '17 at 11:54

According to this question and this question, there isn't a limit to the amount of data transmitted in an AJAX request. The browser or your server could be imposing limitations. Have you tested your application with multiple browsers? Which type of server? Can you attempt to find the approximate amount of data where the request fails?

This question reveals there should be configuration values to allow the massive transmission. One answer proposes converting your data into a JSON string, which you have already done.

Regarding an alternative way to generate the table, my initial thought was to break the request into multiple requests (possibly based on the number of user records). Perhaps you could fall-back on this method if you can't resolve the problem. @Arnau Fernández's suggestion to load the data in small chunks as the user scrolls is similar and clever. If you don't want the scroll and load behavior you could send all of the requests at once, which would still provide the security of eliminating the massive request with smaller ones.

I see the .aspx in your URL. If you're using asp.net, you could try the solution here, which adjusts a value in the web.config file. If this doesn't help, then more information about your technologies would be helpful.

Pasting their configuration for reference:

<configuration>
    <system.web.extensions>
        <scripting>  
             <webServices>                                                   
                 <jsonSerialization maxJsonLength="1000000" />                 
             </webServices>
        </scripting>
    </system.web.extensions>
</configuration>

Let me try this option and see whether it will work for me. – bwanamaina May 03 '17 at 11:30 — bwanamaina, May 03 '17 at 11:30

score 0 · Answer 4 · answered Aug 09 '20 at 23:09

        JavaScriptSerializer js = new JavaScriptSerializer();
        js.MaxJsonLength = Int32.MaxValue;
        string str = js.Serialize(YourList);

when the json string size is big, it can't send the data to client side, so you have to increase the size like "Int32.MaxValue". No need to configure web.config file.

Get large amount of data using ajax

4 Answers4