How can I prevent my javascript from polling websites too fast?

Question

I made this website quite a while ago for the purpose of grabbing random puu.sh images and displaying them. Also a warning, the content this site shows is user generated, and I can not guarantee it is SFW.

https://dl.dropboxusercontent.com/s/dlb1uke5udz8kwy/index.html

I just started looking at it again, and it appears it has some serious bugs with the systems it uses to gather content. How can I change my javascript so that it doesn't spam out puu.sh, and make it refuse connections?

Here is the code:

var currentThumb = 0;

function getImgUrl()
{
    var text = (Math.round(Math.random() * 9)).toString();
    //var text = '3T';
    var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";

    for(var i=0; i < 4; i++ )
        text += possible.charAt(Math.floor(Math.random() * possible.length));

    return 'http://puu.sh/' + text;
}

function open_in_new_tab(url)
{
  var win=window.open(url, '_blank');
  win.focus();
}

function getImages()
{
    //var width = window.innerWidth;
    var images = 10;

    for(var i = 0;i < images;i++)
    {
        var url = getImgUrl();

        document.getElementById('thumbs').innerHTML += '<img class="thumb" id="thumb' + i + '" src=' + url + '>';

        if(i == 0)
        {
            document.getElementById('fullimage').innerHTML = '<img id="big" src=' + url + '>';
            $('#thumb' + currentThumb).css('border','2px solid white');
        }
    }
}

function refreshImages()
{
    var images = 10;

    for(var i = 0;i < images;i++)
    {
        var url = getImgUrl();

        $('#thumb' + i).attr('src',url);
    }

    $('img').css('border','');
    $('#thumb' + currentThumb).css('border','2px solid white');
}

function resize()
{
    var width = $(window).width();
    var height = $(window).height();
    $('#fullimage img').css('max-width',width);
    $('#fullimage img').css('max-height',height - 87);
}

function setBig()
{
    $('#big').attr('src',($('#thumb' + currentThumb).attr('src')));

    $('img').css('border','');
    $('#thumb' + currentThumb).css('border','2px solid white');

    resize();
}

getImages();

$('img').error(function() {
    $(this).attr('src',getImgUrl());
    setBig();
});

$('#thumbs img').click(function() {
    $('#fullimage').html('<img id="big" src=' + $(this).attr('src') + '>');

    currentThumb = parseInt($(this).attr("id").match(/\d+/));

    $('img').css('border','');
    $(this).css('border','2px solid white');

    resize();
});

$('#fullimage').click(function() {
    open_in_new_tab($('#fullimage img').attr('src'));
});

$(window).resize(function() {
    resize();
});

$(document).ready(function() {
    resize();
});

The problem most likely lies in

$('img').error(function() {
    $(this).attr('src',getImgUrl());
    setBig();
});

@TheShellfishMeme The problem with that solution is then the images will constantly change. In the current setup, images that "successfully" load stay. Also, the current randomization system in place isn't perfect, and it may pick a 404 image. — NAME__, Apr 22 '14 at 05:57
What do you mean, they will constantly change? That you insert one at a time so once they load they will take up more space and then move the other images around? That you can fix for example by inserting placeholders first. — Tim, Apr 22 '14 at 05:59
@TheShellfishMeme Sorry, I mis-interpreted you. I thought you were saying to insert a constantly executing setInterval that changed all the images. I already do insert placeholders to stop them from moving around. The problem is, if one of the images is 404, it spams puu.sh over and over, and with multiple images spamming puu.sh, the site eventually refuses connection. — NAME__, Apr 22 '14 at 06:02

score 0 · Answer 1 · answered Apr 22 '14 at 06:08

0

Untested, but try this new getImages() function (replace the old one)

function getImages(images)
{
    if (!images) {
        document.getElementById('fullimage').innerHTML = '<img id="big" src=' + url + '>';
        $('#thumb' + currentThumb).css('border','2px solid white');
        images = 1;
    }

    document.getElementById('thumbs').innerHTML += '<img class="thumb" id="thumb' + i + '" src=' + url + '>';

    images++;

    if (images < 10) {
        setTimeout(function() { getImages(images); }, 25);
    }
}

It will be called 9 times, but the first time it should do what you were doing at i = 0 and 1; It will be fired every 25ms regardless of whether images have loaded yet or not. Increase that number (25) to slow it down.

answered Apr 22 '14 at 06:08

Hamza Kubba

2,259
14
12

I just fixed a bug with it, FYI (was adding 2 to images). – Hamza Kubba Apr 22 '14 at 06:10
Note that I'm assuming that `getImages()` is in the global/window scope, which from the code given, it seems to be. – Hamza Kubba Apr 22 '14 at 06:11
This is a duct tape solution btw. A better solution would involve using the answer on the following link, but involves changing more of your code. http://stackoverflow.com/questions/20613984/jquery-or-javascript-check-if-image-loaded – Hamza Kubba Apr 22 '14 at 06:14
I tested out your code after fixing up a few errors and it had some strange effects on my error handling, as in, it no longer functioned. – NAME__ Apr 22 '14 at 06:14
What do you mean by strange effects? Also, try setting the number lower, to 50ms or 75ms. If you have the stamina for it, you should restructure the code a bit, so that you only load 3-5 images, then wait a while, then load more etc. You can use a variable, say `imagesLoading` to keep track of how many images are loading, then decrease `imagesLoading` when an image completes loading or errors out. When `imagesLoading` hits 0 again, wait some time then load another small batch of images. – Hamza Kubba Apr 22 '14 at 06:21
I'm going to attempt to restructure my code. I will load a single image at a time perhaps, and work with that. – NAME__ Apr 22 '14 at 06:22

score 0 · Accepted Answer · answered Apr 22 '14 at 06:13

0

Your main problem seems to be that you are spamming the servers when you hit a 404 because of the error handler. Try something like

$('img').error(function() {
    var _this = this;
    setTimeout(function () {
        $(_this).attr('src',getImgUrl());
        setBig();
    }, 500); // Whatever timeout is good not to spam
});

answered Apr 22 '14 at 06:13

Tim

2,430
19
20

This seemed to simply slow down the rate at which they all refreshed at the same time, after which they all began being granted 503 service errors. – NAME__ Apr 22 '14 at 06:20
I think you have to create one error handler per image, then have a queue of new images to load using `setTimeout` or `setInterval` so that several 404s at the same time don't just cause you to start loading all the images again at the same time after the same time offset. Another thing you could do is randomize the timeout so that the requests don't all happen at the same time. – Tim Apr 22 '14 at 06:26

How can I prevent my javascript from polling websites too fast?

2 Answers2