AJAX continous response of PHP script output

Question

I have problem with AJAX response of my PHP script...

I've created "Status" div, where I want to output response of PHP script. It works good, but the response shows up only, when whole script finishes, and I would like to output each echo "live"..

Here are my files:

form.php

<!-- left column -->
<div class="col-md-6">
    <!-- general form elements -->
    <div class="card card-primary">
        <div class="card-header">
            <h3 class="card-title"><?php echo $xmlData->name ?></h3>
        </div>
        <!-- /.card-header -->
        <!-- form start -->
        <form id="upload_form" enctype="multipart/form-data" method="post">
            <div class="card-body">
                <div class="form-group">
                    <label for="debFile">Deb file</label>
                    <span id="debMSG"></span>
                    <div class="input-group">
                        <div class="custom-file">
                            <input name="debFile" type="file" class="custom-file-input" id="debFile">
                            <label class="custom-file-label" for="debFile">Choose .deb file</label>
                        </div>
                    </div>
                </div>
                <div class="form-group">
                    <label for="version">Version</label>
                    <input type="text" name="version" id="version" class="form-control" value="<?php echo ver_up($xmlData->changelogs->change->version); ?>">
                </div>
                <div class="form-group">
                    <label for="log">Log</label>
                    <input type="text" name="log" id="log" class="form-control" value="Bug fix">
                </div>
                <div class="form-check">
                    <input name="tweet" type="checkbox" class="form-check-input" id="tweet" value="1">
                    <label class="form-check-label" for="tweet">Auto-post on Twitter</label>
                </div>
            </div>
            <!-- /.card-body -->

            <div class="card-footer">
                <button id="submit" name="upload" type="submit" class="btn btn-primary">Update</button>
            </div>
        </form>
    </div>
    <!-- /.card -->
</div>
<!-- Status -->
<div id="status" class="col-md-6" style="display:none;">
    <div class="card card-default">
        <div class="card-header">
            <h3 class="card-title">Status:</h3>
        </div>
        <div class="alert alert-default">
            <ul id="output" class="list-group"></ul>
        </div>
    </div>
</div>
<!--/.col (right) -->

ajax in form.php

<script type="text/javascript">
$(document).ready(function () {
    bsCustomFileInput.init();

    const Toast = Swal.mixin({
        toast: true,
        position: 'top-end',
        showConfirmButton: false,
        timer: 5000
    });

    $(document).on('submit', '#upload_form', function(event){
        event.preventDefault();

        if($('#debFile').val() == '')
        {
            $('#debMSG').html('<div class="alert alert-danger">Enter .deb file!</div>');
            $('#debMSG').show();
            setTimeout(function() {
                $('#debMSG').fadeOut('fast');
            }, 5000);
            return false;
        }
        else
        {

            $.ajax({
                type: 'POST',
                url: 'scripts/test.php',
                data: new FormData(this),
                // async: true,
                cache: false,
                processData: false,
                contentType: false,
                beforeSend:function()
                {
                    $('#debMSG').hide();
                    $('#submit').attr('disabled', 'disabled');
                    // $('#status').fadeIn('slow');
                }
            })
            .done(function(data) {
                // console.log(data);
                $('#output').append(data);
                $('#status').fadeIn('slow');
                // if(!data.success){
                //  $('#submit').removeClass('btn-primary').addClass('btn-danger');
                //  $('#submit').attr('disabled', false);
                //  $('#submit').removeAttr('name type');
                //  $('#submit').attr('onClick', 'location.reload();');
                //  $('#submit').html('Reload');
                //  Toast.fire({
                //      icon: 'error',
                //      title: 'There was an error while updating tweak!'
                //  })
                // } else {
                    $('#submit').removeClass('btn-primary').addClass('btn-success');
                    $('#submit').attr('disabled', false);
                    $('#submit').removeAttr('name type');
                    $('#submit').attr('onClick', 'window.location.href = "manage-packages.php";');
                    $('#submit').html('Finish');
                    Toast.fire({
                        icon: 'success',
                        title: 'Tweak has been successfully updated!'
                    })
                // }
            })
            .fail(function(data) {
                console.log(data);
                $('#output').append("There was an error while updating tweak!");
                // $('#output').removeClass('alert-success').addClass('alert-danger');
                $('#status').fadeIn('slow');
                $('#submit').removeClass('btn-primary').addClass('btn-danger');
                $('#submit').attr('disabled', false);
                $('#submit').removeAttr('name type');
                $('#submit').attr('onClick', 'location.reload();');
                $('#submit').html('Reload');
                Toast.fire({
                    icon: 'error',
                    title: 'There was an error while updating tweak!'
                })
            });
        }
    });
});
</script>

test.php

<?php

header('Content-type: text/html; charset=utf-8');

require_once("../../src/config-ssh.php");
require_once("../../src/functions.php");

$deb_dir = "/var/www/html/debs/";
$repo_dir = "/var/www/html/";

function output($val) {
    echo nl2br ($val);
    flush();
    ob_flush();
    sleep(1);
}

try {
    if (isset($_FILES["debFile"]["name"])) {
        $debFile = $_FILES["debFile"]["tmp_name"];
        $deb_file = $deb_dir . basename($_FILES["debFile"]["name"]);

        if (!file_exists($deb_file)) {
            output("<li class=\"list-group-item list-group-item-success\">DEB file \"". basename( $_FILES["debFile"]["name"]). "\" has been uploaded.</li>");
        } else {
            throw new Exception("<li class=\"list-group-item list-group-item-danger\">DEB file \"". basename( $_FILES["debFile"]["name"]). "\" already exists!</li>");
        }
    } else {
        throw new Exception("<li class=\"list-group-item list-group-item-danger\">You didn't choose any DEB file!</li>");
    }

    preg_match('/(.*)_(.*)_(.*)\.deb/', basename($_FILES["debFile"]["name"]), $deb_reg);
    $dirname = $deb_reg[1];
    $prev_ver = ver_down($deb_reg[2]);
    $exten = $deb_reg[3] . ".deb";
    $prev_deb_name = $dirname."_".$prev_ver."_".$exten;
    $prev_deb = $deb_dir . $prev_deb_name;

    if (isset($_POST['version']) && isset($_POST['log'])) {
        output("<li class=\"list-group-item list-group-item-success\">Values were added to XML!</li>");
    }

    if (file_exists($prev_deb)) {
        output("<li class=\"list-group-item list-group-item-success\">Previous version of DEB file (\"". $prev_deb_name . "\") was deleted!</li>");
    } else {
        throw new Exception("<li class=\"list-group-item list-group-item-danger\">DEB file \"". $prev_deb_name . "\" wasn't deleted!</li>");
    }


    // Run a command
    output("<li class=\"list-group-item list-group-item-info\">Running Repo update script...</li>");
        output("<li class=\"list-group-item list-group-item-success\">Script finished successfully.</li>");

    // Auto-post Tweet
    if (isset($_POST['tweet'])) {
        output("$msg");
        output("<li class=\"list-group-item list-group-item-success\">Tweet has been successfully posted.</li>");
    }

    exit();
}

catch( Exception $e ) {
    $message = $e->getMessage();

    die( "ERROR: " . $message );
}

?>

Finally, it prints this status output:

I've already tried several tutorials, but none of them are working for this...

Thanks for your advices!

Your code is not complete so it can't be tested by others (you have some 'required' files that are not in the question) and it gets rather long anyway (best questions have troubleshooting down to a minimal bit of code). That said, it is also quite standard that PHP outputs when the file is complete. If you want 'live' updates, then you have to use JS (which it looks like you are doing, but again the problem is not clear - just what does "each echo" mean??) — Apps-n-Add-Ons, Nov 04 '20 at 13:44
What using server sent events, this simplify everything, the server will continue to push fesh data to the client, see example php js on this page; https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events — NVRM, Nov 04 '20 at 15:14
"Each echo" means that my script (test.php) generates echo responses (defined in "function output") and I want each that response to be shown in real-time (when it's successfully done) one by one using AJAX (as described in provided code)... Now it works good, but it shows all outputs at once, when script successully finished.. That's the reason, why I provided just this pieces of code, because it has to be either AJAX settings, or something in test.php... — Strejda603, Nov 04 '20 at 15:22
And since it depends on many files (bootstrap, my custom page templates and config files - mysql and ssh - with my credentials), I didn't post it whole, because I don't want to share my credentials, and it will be like 800 lines of code... — Strejda603, Nov 04 '20 at 15:26

Ivan Shatsky · Accepted Answer · 2020-11-05T17:09:20.320

You need to bind a handler to the progress event of XmlHttpRequest object. jQuery doesn't provide a native interface to do this, but if you still want to do your AJAX call with jQuery instead of using XmlHttpRequest object directly, you can alter the underlying XmlHttpRequest object with the xhr callback function adding your handler this way:

$.ajax({
    ...
    xhr: function() {
        // get the native XmlHttpRequest object
        const xhr = $.ajaxSettings.xhr();
        // set the onprogress event handler
        xhr.onprogress = function() {
            // replace the '#output' element inner HTML with the received part of the response
            $('#output').html(xhr.responseText);
        }
        return xhr;
    }
})
.done(function(data) {
    ...
})
.fail(function(data) {
    ...
});

It is quite hard to figure out from the current version of XMLHttpRequest standard, but the archived one says that this event is fired about every 50ms or for every byte received, whichever is least frequent.

The other changes you should do to your javascript code are

uncomment the $('#status').fadeIn('slow'); line in the beforeSend function and remove it from the other functions;
remove the $('#output').append(data); line from the done event handler function, all the received data should be already injected via the onprogress handler function.

The last but not least part is about buffering. I assume you are using nginx with the PHP-FPM since you add the nginx tag to your question. You already use flush() and ob_flush() PHP functions, and you are right, you need both of them in order to flush the PHP buffers. However there is one more buffering mechanism - an nginx itself. You can turn off nginx FastCGI buffering completely with the fastcgi_buffering off; directive, but it doesn't look like a good idea. Fortunately, nginx allows to turn off response buffering via the special X-Accel-Buffering header, so your AJAX PHP handler should start with

<?php

header('Content-type: text/html; charset=utf-8');
header('X-Accel-Buffering: no');

...

Thank you! Progress works now, but now I encoutered new error: Even if script ends with Exception (die), AJAX output it as success (with XHR status 200)... Is there a way, where I could tell AJAX to handle exceptions? I've tried to add `header($_SERVER["SERVER_PROTOCOL"] . ' 500 Internal Server Error', true, 500);` to my Exception, but AJAX somehow ignores this header... — Strejda603, Nov 05 '20 at 23:45
@Strejda603 It isn't an AJAX ignoring the header. I think the reason is that you are trying to change the response status **after** some output already happened. This means the `HTTP/ 200 OK` header already has been sent to the client among all the other response headers, now server is sending the response body data so you can't change or add any other header after this happens. Most likely there is a message `Warning: Cannot modify header information - headers already sent by (output started at...` somewhere in your logs. (to be continued in the next comment) — Ivan Shatsky, Nov 06 '20 at 00:15
@Strejda603 Read [this](https://stackoverflow.com/questions/8028957/how-to-fix-headers-already-sent-error-in-php) thread for more details. So the answer is no, you can't combine these two approaches, you should either buffer all the output before setting the HTTP response status or always receive the response with HTTP 200 OK status and check for the errors analyzing the response body. — Ivan Shatsky, Nov 06 '20 at 00:19
I've solved it by creating condition `if(xhr.responseText.match(/ERROR:/))`... Once again, thanks for help! — Strejda603, Nov 06 '20 at 13:39
@Strejda603 Yes, that would work inside the `onprogress` handler function, and if you'd want to move this check to the `done` handler function, change it to `if(data.match(/ERROR:/)){...}` — Ivan Shatsky, Nov 06 '20 at 13:53
@Strejda603 do you mind to share the working code, please? I have the same problem with you. — frozenade, Jun 13 '22 at 02:46

AJAX continous response of PHP script output

1 Answers1

Linked