Flask: Display Flash Messages Constantly

Question

I developed a flask app which runs a mathematical optimization script (PuLP+ Python) for user provided data input. Since the computation takes quite a while, I want to display the optimizer's print statements constantly (without refreshing) on my webpage.

Based on this solution I managed to run a subprocesswith my .py script. Within this script, the print() statement does what it should, however I can only see the output in the command line. By using flash() I managed to display the output, yet it gets rendered only once the computation has finished and the page is reloaded. I try to output the print() statements in realtime within my HTML. Is there a way to achieve this?

Thanks!

Excerpt from routes.py:

@app.route('/optimize', methods=['GET', 'POST'])
    def solve():
    path = os.path.join(app.root_path, 'optimization', 'assets')
    file = "test.py"
    execute(file, path)

return redirect(url_for('home'))

The execute(file,path) function:

import os
import subprocess as sub

def execute(command, directory):

    # change directory to execute the command
    os.chdir(directory)

    proc = sub.Popen(['python','-u', command], stdout=sub.PIPE, universal_newlines=True) # Python 2: bufsize=1

    for stdout_line in proc.stdout:
        print(stdout_line.rstrip() + '\n')
        flash(stdout_line.rstrip(), 'python_print')
    proc.stdout.close()
    return_code = proc.wait()
    if return_code:
        raise sub.CalledProcessError(return_code, command)

And finally my HTML which is rendered in the home route:

{% with prints = get_flashed_messages(category_filter=["python_print"]) %}
{% if prints %}
<div class="content-section">
    <h3>Solver Status</h3>
    <ul>
        {%- for message in prints %}
        <li>{{ message }}</li>
        {% endfor -%}
    </ul>
</div>
{% endif %}
{% endwith %}

Answer 1

As far as I know this is how message flashing is supposed to work, i.e. it will display all the messages for a particular end point all at once.

I am not sure about the exact implementation since I have not looked at the source code for flash() and get_flash_messages() but this is my understanding of what might be happening in the background, every message you flash with flash function in your python code it gets appended to an iterable, and when you call the get_flashed_messages() it returns that iterable back, which you can loop over to get the messages.

Though your execute() function is adding messages to that iterable before the process has been completed, but notice the execute function is being called inside the view solve and the actual redirect can happen only after the execute function has completed its execution, and that's when the template home will gets the iterable having all the flashed messages.

Hope that makes sense.

Answer 2

Based on the suggestions provided I changed my approach and got it running. Instead of using subprocess, the solver is imported as module. I'm generating a .txt using the python logging module rather than trying to catch the print statements. This file is streamed constantly using flask response + yield and parsed via AJAX. The getter is called in jQuery via setInterval (here: 1Hz) and added to my HTML. I will post some snippets below for anyone dealing with the same issue.

Thanks for your support!

Flask optimize route:

import supplierAllocation.optimization.optimizer as optimizer

@app.route('/optimize')
def optimize():

    if session.get('input_file') is not None:

        # Get input_file
        input_file = session.get('input_file')

        # Path to logfile
        log_file = os.path.join(
            app.config['LOG_FOLDER'], session.get('log_file'))

        # Assign random unique hex file name from session to output file
        out_file_fn = session.get('session_ID') + '.xlsx'
        output_file = os.path.join(app.config['DOWNLOAD_FOLDER'], out_file_fn)

        # Search for outdated output; if present: delete
        if session.get('output_file') is None:
            session['output_file'] = out_file_fn
        else:
            session_output = os.path.join(
                app.config['DOWNLOAD_FOLDER'], session.get('output_file'))
            silent_remove(session_output)

        # Pass input and output parameter to solver
        optimizer.launch_solver(input_file, output_file, log_file)

    else:
        flash("Please upload your data before launching the solver!", 'warning')

    return redirect(url_for('home'))

Flask streaming route:

@app.route('/streamLog')
def stream_logfile():
    if session.get('log_file') is None:
        return Response(status=204)
    else:
        session_log = os.path.join(
            app.config['LOG_FOLDER'], session.get('log_file'))

        def stream():
            with open(session_log, "rb") as file:
                yield file.read()

        return Response(stream(), mimetype='text/plain')

Call of AJAX request on button click:

// Print solver output with 1Hz
var session_log = setInterval(get_Log, 1000);

AJAX request in jQuery:

// AJAX Request to download solutions
function get_Log() {
  console.log('Sending AJAX request...');
  $.ajax({
    url: "streamLog",
    dataType: "text",
    success: function(result) {
      $("#code-wrapper").html(result);
    },
    error: function() {
      console.log("ERROR! Logfile could not be retrieved.");
    }
  });
}