6

I am building an autocomplete functionality and realized the amount of time taken between the client and server is too high (in the range of 450-700ms)

autocomplete response time

My first stop was to check if this is result of server delay.

enter image description here

But as you can see these Nginx logs are almost always 0.001 milliseconds (request time is the last column). It’s hardly a cause of concern.

So it became very evident that I am losing time between the server and the client. My benchmarks are Google Instant's response times. Which almost often is in the range of 30-40 milliseconds. Magnitudes lower.

enter image description here

Although it’s easy to say that Google's has massive infrastructural capabilities to deliver at this speed, I wanted to push myself to learn if this is possible for someone who is not that level. If not 60 milliseconds, I want to shave off 100-150 milliseconds.

Here are some of the strategies I’ve managed to learn.

  1. Enable httpd slowstart and initcwnd
  2. Ensure SPDY if you are on https
  3. Ensure results are http compressed
  4. Etc.

What are the other things I can do here?

e.g

  • Does have a persistent connection help?
  • Should I reduce the response size dramatically?

Edit: Here are the ping and traceroute numbers. The site is served via cloudflare from a Fremont Linode machine.

    mymachine-Mac:c name$ ping site.com
    PING site.com (160.158.244.92): 56 data bytes
    64 bytes from 160.158.244.92: icmp_seq=0 ttl=58 time=95.557 ms
    64 bytes from 160.158.244.92: icmp_seq=1 ttl=58 time=103.569 ms
    64 bytes from 160.158.244.92: icmp_seq=2 ttl=58 time=95.679 ms
    ^C  
    --- site.com ping statistics --- 
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 95.557/98.268/103.569/3.748 ms
    mymachine-Mac:c name$ traceroute site.com
    traceroute: Warning: site.com has multiple addresses; using 160.158.244.92
    traceroute to site.com (160.158.244.92), 64 hops max, 52 byte packets
     1  192.168.1.1 (192.168.1.1)  2.393 ms  1.159 ms  1.042 ms
     2  172.16.70.1 (172.16.70.1)  22.796 ms  64.531 ms  26.093 ms
     3  abts-kk-static-ilp-241.11.181.122.airtel.in (122.181.11.241)  28.483 ms  21.450 ms  25.255 ms
     4  aes-static-005.99.22.125.airtel.in (125.22.99.5)  30.558 ms  30.448 ms  40.344 ms
     5  182.79.245.62 (182.79.245.62)  75.568 ms  101.446 ms  68.659 ms
     6  13335.sgw.equinix.com (202.79.197.132)  84.201 ms  65.092 ms  56.111 ms
     7  160.158.244.92 (160.158.244.92)  66.352 ms  69.912 ms  81.458 ms
    mymachine-Mac:c name$  site.com (160.158.244.92): 56 data bytes
Community
  • 1
  • 1
Quintin Par
  • 15,862
  • 27
  • 93
  • 146
  • 1
    What's the ping time from the client you've shown to your server? How many hops are in the traceroute? Have you tried this on a local network or on a single machine (with `localhost`?) If you're going edge to edge on the network (where one edge is your client and the other is a machine in a low-cost web hosting farm) you may have hit the latency limit already. – O. Jones May 31 '14 at 18:17
  • @OllieJones I've added the ping and traceroute details – Quintin Par Jun 01 '14 at 03:51
  • 1
    Aha, it's served via cloudflare. Your ping time is to the nearest cloudflare server. So, your dynamic menu queries travel from your client to a nearby cloudflare node, then through the cloudflare - to - customer relay to your server. You're carrying the cloudflare latency in your latency. You should turn off cloudflare to eliminate their latency from the mix for your queries if you're going to tackle this problem rationally. I suspect a premium cloudflare subscription is in your future if you want to serve these queries through them. (Is this intercontinental? If so, it will be slow.) – O. Jones Jun 01 '14 at 21:06
  • I tried bypassing cloudflare to the machine directly. Not much difference. @OllieJones – Quintin Par Jun 04 '14 at 16:21
  • May I have the server URL to check the times from here and the page loading sequence? – miguel-svq Jun 04 '14 at 17:46
  • 1) Is the server you running the script shared hosting? If so, you will never get a faster speed. Get a dedicated one. 2) Are your scripts efficient enough? Keep the script as little as can be. 3 Building an autocomplete for what? How does it search? – Codebeat Jun 11 '14 at 15:00

4 Answers4

3

I may well be wrong, but personally I smell a rat. Your times aren't justified by your setup; I believe that your requests ought to run much faster.

If at all possible, generate a short query using curl and intercept it with tcpdump on both the client and the server.

It could be a bandwidth/concurrency problem on the hosting. Check out its diagnostic panel, or try estimating the traffic.

You can try and save a response query into a static file, then requesting that file (taking care as not to trigger the local browser cache...), to see whether the problem might be in processing the data (either server or client side).

Does this slowness affect every request, or only the autocomplete ones? If the latter, and no matter what nginx says, it might be some inefficiency/delay in recovering or formatting the autocompletion data for output.

Also, you can try and serve a static response bypassing nginx altogether, in case this is an issue with nginx (and for that matter: have you checked out nginx' error log?).

LSerni
  • 55,617
  • 10
  • 65
  • 107
1

One approach I didn't see you mention is to use SSL sessions: you can add the following into your nginx conf to make sure that an SSL handshake (very expensive process) does not happen with every connection request:

ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

See "HTTPS server optimizations" here: http://nginx.org/en/docs/http/configuring_https_servers.html

Martin Konecny
  • 57,827
  • 19
  • 139
  • 159
  • This is all via http. – Quintin Par Jun 01 '14 at 03:42
  • I see - what about ping times? – Martin Konecny Jun 01 '14 at 03:43
  • 1
    I think the `$response_time` in your logs doesn't mean what you think it does. See this link: http://lincolnloop.com/blog/tracking-application-response-time-nginx/ - as mentioned there, `$response_time` doesn't take into account the time the upstream server takes to generate the request. It also provides a solution so that you can measure the true response time. – Martin Konecny Jun 01 '14 at 04:07
0

I would recommend using New Relic if you aren't already. It is possible that the server-side code you have could be the issue. If you think that might be the issue, there are quite a few free code profiling tools.

Ray
  • 2,713
  • 3
  • 29
  • 61
-1

You may want to consider an option to preload autocomplete options in the background while the page is rendered and then save a trie or whatever structure you use on the client in the local storage. When the user starts typing in the autocomplete field you would not need to send any requests to the server but instead query local storage.

Web SQL Database and IndexedDB introduce databases to the clientside. Instead of the common pattern of posting data to the server via XMLHttpRequest or form submission, you can leverage these clientside databases. Decreasing HTTP requests is a primary target of all performance engineers, so using these as a datastore can save many trips via XHR or form posts back to the server. localStorage and sessionStorage could be used in some cases, like capturing form submission progress, and have seen to be noticeably faster than the client-side database APIs.

For example, if you have a data grid component or an inbox with hundreds of messages, storing the data locally in a database will save you HTTP roundtrips when the user wishes to search, filter, or sort. A list of friends or a text input autocomplete could be filtered on each keystroke, making for a much more responsive user experience.

http://www.html5rocks.com/en/tutorials/speed/quick/#toc-databases

bancer
  • 7,475
  • 7
  • 39
  • 58