Deferred or streaming response bodies

The example shows a one-shot request, wait, response cycle. But it is possible to have multiple wait, response segments to allow the headers to be sent to the client immediately, or the body to trickle in slowly without blocking the worker process.

Async Rack servers, when env['async.callback'] is called, send the status and headers to the client and then begin iterating through each part of the body with #each.

After the last body part the server must decide if the connection to the client should be closed (entire body has been provided) or if it should remain open (body parts will be provided later). The details of this decision are implementation-specific. For now, assume the connection is not closed. To send additional body parts, env['async.callback'] may not be called a second time since the status code and headers have already been sent to the client and can not be changed. The app takes advantage of the server's iteration through the body with #each:

the server calls body.each(&block), and the trick is to save &block for later use. This turns the iteration inside-out: rather than the sever iterating through a body, the app takes control to send each part of the body itself.

class DeferredBody
  def each(&block)
    # normally we'd yield each part of the body, but since
    # it isn't available yet, we save &block for later
    @server_block = block
  end

  def send(data)
    # calling the saved &block has the same effect as
    # if we had yielded to it
    @server_block.call data
  end
end

class AsyncApp
  def call(env)
    Thread.new do
      sleep 5  # simulate waiting for some event
      body = DeferredBody.new
      response = [200, {'Content-Type' => 'text/plain'}, body]
      env['async.callback'].call response

      # at this point, the server may send the status and headers,
      # but the body was empty
      
      body.send 'Hello, '
      sleep 5
      body.send 'World'
    end
    
    [-1, {}, []]  # or throw :async
  end
end

Note that the above won't quite work because we haven't signaled to the server that the body will be deferred and streamed in part by part.