High Scale REST API

Question

One of our REST APIs will cause a long-running process to execute. Rather than have the client wait for a long time, we would prefer to return an immediate response.

So, let's consider this use case: An applicant submits an application, for which there will be an eventual result. Since this is a very high-scale platform, we cannot persist the application to storage, but must place it onto a queue for processing.

In this situation, is it acceptable practice to return the URI where the application will eventually live, such as http://example.com/application/abc123?

Similarly, would it be acceptable practice to return the URI of the result document, which represents the decision regarding the application, as part of the representation of the application resource? The result document will not be created for some minutes, and an HTTP GET to its URI (or the URI of the application for that matter) will result in a 404 until they are persisted.

What is the best practice in this kind of situation? Is it acceptable to hand out "future" URIs for resources?

Answer 1

I don't see anything wrong with such design, but have a closer look at the list of HTTP status codes for better responses. IMHO the first request should return 202 Accepted:

The request has been accepted for processing, but the processing has not been completed.

while requests to the URL where the result will eventually be should in the meantime return 204 No Content (?):

The server successfully processed the request, but is not returning any content

And of course it should eventually return 200 OK when processing finishes.

Answer 2

From "RESTful Web Services Cookbook"

Problem

You want to know how to provide resource abstractions for tasks such as performing computations or validating data.

Solution

Treat the processing function as a resource, and use HTTP GET to fetch a representation containing the output of the processing function. Use query parameters to supply inputs to the processing function.

This entails just GET requests on a URI that represents the processing function. Your example 'http://example.com/application/abc123' URI. When returning a response you would include what information you have by now and use HTTP codes to indicate the status of the processing as already suggested by Tomasz.

However..., you should not use this approach, if the subsequent application processing stores or modifies data in any way.

GET requests should never have side effects. If the submittal of the application leads in anyway (even if only after being processed in from queue) to new information / data being stored, you should use a PUT or a POST request with the application's data in the request's body. See "Why shouldn't data be modified on an HTTP GET request?" form more information.

If they application's submittal stores or modifies data, use the pattern for asynchronous processing: a POST or PUT request with the application's details.

For example

POST http://example.com/applications

which returns "201 Created" with the URI of the new application resource.

or

PUT http://example.com/applications/abc123

which returns "201 Created" and

Both would also return any resource information that is already known at that time.

You can then safely perform GET requests on the URI of the new resource as they now only retrieve data - the results of the application processing so far - and no data is stored or modified as a result of the GET.

To indicate the application's processing progress, the GET request can either return some specific status code in the response (queued, processing, accepted, rejected), and/or use the HTTP response codes. In either case a "200 OK" should only be returned when the application's processing is complete.