How to delay google geocoding?

Question

I am trying to geocode around 100 postcodes from a json file by looping through the postcode, however I am restricted by the OVERY_QUERY_LIMIT error after 11 postcodes. It seems I am using the geocoder too fast per second. I have tried incorporating setTimeout in to it but can;t get it to work. How can i go about slowing down the process?

code

<head>
    <meta charset="UTF-8">
    <title>heatmap test</title>

    <style>
        html {
            height: 100%
        }

        body {
            height: 100%;
            margin: 0;
            padding: 0;
            font-family: sans-serif;
        }

        #map-canvas {
            height: 100%
        }
    </style>
</head>

<body>
    <div id="map-canvas"></div>

    <script src="https://maps.googleapis.com/maps/api/js"></script>
    <script src="assets/gmaps.js"></script>
    <script>
        document.addEventListener("DOMContentLoaded", function (event) {

                var getPostcodes = function () {

                    //json link
                    var json = "http://localhost/addresses.json/";

                    var request = new XMLHttpRequest();
                    request.open('GET', json, true);

                    request.onload = function () {
                        if (request.status >= 200 && request.status < 400) {
                            // Success!
                            var data = JSON.parse(request.responseText);

                            //GET POSTCODES IN ARRAY
                            var postcodesArray = [];
                            for (var i = 0; i < data.length; i++) {
                                postcodesArray.push(data[i].from_postcode);
                            }

                            //filter all falsey values from array
                            var postcodes = postcodesArray.filter(Boolean);

                            //creating object for inserting data
                            var testData = {
                                data: []
                            };

                            //GET LAT/LONG POSITIONS
                            //for loop to create geocode object for each postcode
                            for (var n = 0; n < postcodes.length; n++) {

                                setTimeout(function(){

                                GMaps.geocode({
                                    address: postcodes[n],
                                    callback: function (results, status) {
                                        if (status == google.maps.GeocoderStatus.OK) {
                                            var location = results[0].geometry.location;

                                            var latitude = parseFloat(location.lat().toFixed(8)),
                                                longitude = parseFloat(location.lng().toFixed(8));

//                                            pushing lat and long data to object
                                            testData.data.push({
                                                lat: latitude,
                                                lng: longitude
                                            });
                                        } else if (status === google.maps.GeocoderStatus.OVER_QUERY_LIMIT) {
                                            console.log("error");
                                        }

                                    }

                                });

                                }, 500);
                            }

                            console.log(testData);
                        }
                        else {
                            console.log("failed");
                        }
                    };
                    request.onerror = function () {
                        // There was a connection error of some sort
                    };
                    request.send();
                }

                getPostcodes();

            });
    </script>
</body>

The limis is 50 requests per second, but your issue is that if this is something intended for the web, multiple users can be running that code simultaneously from their browser, and even if you do rate limit it on the clientside, there's no way to rate limit it across different users, hence why most would do this serverside whith a queue in place instead. — adeneo, Oct 04 '16 at 14:43
ummm...geocode them once and store data?? Makes no sense to do them each time page loads — charlietfl, Oct 04 '16 at 14:43
I am doing this to use the lat/lng in a heatmap, is it recommended to do it serverside? also where would you store the data? — Sai, Oct 04 '16 at 14:51
Very simple to update that array and then output it as json and put it back in the file if that is your only storage. Or use a database on your server or third party service — charlietfl, Oct 04 '16 at 14:53
so would this work in your opinion, convert my testdata object to json and store in a exterior file, do a character match so next time i open it skips existing postcodes and keep building up the file? — Sai, Oct 04 '16 at 15:15

score 1 · Answer 1 · 2016-10-04T14:49:12.493

1

SetTimeout simply delays code, and you've given the timeout a set delay.

What happens is all of your functions wait for 500ms and then all get run instantly.

What you can either do is have a dynamic delay like so:

...

for (var n = postcodes.length; n--;) {
    setTimeout(function(){
        GMaps.geocode({
            ...
        })
    }, 500 * n);
}

....

or use a setInterval, but the above code would work best. The setInterval would make less sense.

edited Oct 04 '16 at 14:49

answered Oct 04 '16 at 14:44

Actually, as geocoding is async, a recursive function would make the most sense. – adeneo Oct 04 '16 at 14:55
1

His problem is simply initiating too many requests a second, asynchrinosity plays no part in it. – Oct 04 '16 at 14:55
He hasn't stated he wan'ts to only send the request once he recieves his response – Oct 04 '16 at 14:56
Hey I tried this and my testdata object returns empty, however there a fewer errors the more increase the time. I am trying to use the lat/lng for a heatmap so the data being processed can't take too long – Sai Oct 04 '16 at 15:00
To limit the time change `500 * n` to `20 * n`. according to adeneo ` The limis is 50 requests per second` – Oct 04 '16 at 15:05
It's still returning error the majority of the time, and the object remains empty – Sai Oct 04 '16 at 15:09

How to delay google geocoding?

1 Answers1