I have a simple chat application written using FastAPI in Python and jQuery. The user enters a question into form, and a response is returned from the server. However, I also need to send the user message to a separate process that takes a long time (say, querying a database). I don't want the user to have to wait for that separate process to complete, but I have been unable to get that to work. I've tried all sorts of variations of Promise and await, but nothing works. Here is a toy example that demonstrates the problem:
<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<script type="text/javascript">
$(document).ready(function () {
$('#message-form').submit(async function (event) {
event.preventDefault();
const input_message = $('#message-form input[name=message]').val()
$('#message-list').append('<li><strong>' + input_message + '</strong></li>');
side_track(input_message);
const response = await fetch('/submit_message', {
method: 'POST',
body: JSON.stringify({ message: input_message }),
headers: { 'Content-Type': 'application/json' },
});
// Reset the message input field
$('#message-form')[0].reset();
const newMessage = document.createElement('li');
$('#message-list').append(newMessage);
const stream = response.body;
const reader = stream.getReader();
const decoder = new TextDecoder();
const { value, done } = await reader.read();
const message = JSON.parse(decoder.decode(value)).message;
newMessage.innerHTML += message
});
});
async function side_track(question) {
const response = fetch('/side_track', {
method: 'POST',
body: JSON.stringify({ message: question }),
headers: { 'Content-Type': 'application/json' },
});
alert('Getting questions')
}
</script>
</head>
<body>
<ul id="message-list">
<li>Message list.</li>
<!-- Existing messages will be inserted here -->
</ul>
<form id="message-form" method="POST">
<input type="text" name="message" placeholder="Enter your message">
<button type="submit">Submit</button>
</form>
</body>
</html>
And the corresponding Python:
# -*- coding: utf-8 -*-
from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from pydantic import BaseModel
from fastapi.staticfiles import StaticFiles
app = FastAPI()
templates = Jinja2Templates(directory="templates")
app.mount("/static", StaticFiles(directory="static"), name="static")
class MessageInput(BaseModel):
message: str
@app.post("/side_track")
async def side_track(message_data: MessageInput):
import time
time.sleep(10)
return {"status": "ok"}
@app.get("/", response_class=HTMLResponse)
async def index(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
async def random_numbers():
import random
import asyncio
while True:
await asyncio.sleep(.1) # Wait for 1 second
yield random.randint(1, 10)
@app.post("/submit_message")
async def submit_message(message_data: MessageInput):
from fastapi.encoders import jsonable_encoder
async for number in random_numbers():
return jsonable_encoder({'message': number})
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="127.0.0.1", port=8000)
In the example above, the user has to wait the full 10 seconds for side_track to complete before displaying the message returned from submit_message. I want the message from submit_message (which doesn't take any time to process) to display immediately, and the response from side_track to be handled separately whenever it completes, without tying up the program.
EDIT: I modified the toy program to more accurately demonstrate the asynchronous generator that responds to submit_message and make it easier to replicate the problem.
