Progressive Response 🧪

A common use of the JSON Lines format is to sequentially send multiple data chunks (JSON lines) in response to a single request. This is useful for long‑running operations, such as LLM completions, where you want to deliver partial results as they become available.

But what if you don’t know which chunk will arrive first, second, and so on? In this case, you can use an experimental feature called “progressive response,” inspired by Dan Abramov’s proposal Progressive JSON , from which the “progressive” name originates.

The progressive response feature can also be described as “partial response streaming,” similar to Partial Prerendering , but instead of rendering UI, it “renders” response data in an unknown order. When all data has been sent, the response is closed.

Let’s say you have two functions that return data after some random delay: getUsers and getTasks, implemented as static methods of a service module. In a real application, these could be API calls or queries to different databases.

With the help of the JSONLinesResponse class, we can create a simple service method that looks like this:


// ...
void Promise.all([
  this.getUsers().then((users) => resp.send({ users })),
  this.getTasks().then((tasks) => resp.send({ tasks })),
])
  .then(resp.close)
  .catch(resp.throw);
// ...

Once getUsers() or getTasks() resolves, resp.send sends a JSON line to the client.
When all promises resolve, resp.close closes the response stream.
If any promise rejects, resp.throw sends an error response to the client.

The full implementation of the service module looks like this:


import { JSONLinesResponse, type VovkIteration } from 'vovk';
import type ProgressiveController from './ProgressiveController';
 
export default class ProgressiveService {
  static async getUsers() {
    await new Promise((resolve) => setTimeout(resolve, Math.random() * 10_000));
    return [
      { id: 1, name: 'John Doe' },
      { id: 2, name: 'Jane Smith' },
      { id: 3, name: 'Alice Johnson' },
      { id: 4, name: 'Bob Brown' },
      { id: 5, name: 'Charlie White' },
    ];
  }
 
  static async getTasks() {
    await new Promise((resolve) => setTimeout(resolve, Math.random() * 10_000));
    return [
      { id: 1, title: 'Task One', completed: false },
      { id: 2, title: 'Task Two', completed: true },
      { id: 3, title: 'Task Three', completed: false },
      { id: 4, title: 'Task Four', completed: true },
      { id: 5, title: 'Task Five', completed: false },
    ];
  }
 
  static streamProgressiveResponse(
    resp: JSONLinesResponse<VovkIteration<typeof ProgressiveController.streamProgressiveResponse>>
  ) {
    void Promise.all([
      this.getUsers().then((users) => resp.send({ users })),
      this.getTasks().then((tasks) => resp.send({ tasks })),
    ])
      .then(resp.close)
      .catch(resp.throw);
  }
}

On the controller side, instantiate JSONLinesResponse, pass it to the service method, and return it as the response.


// ...
const response = new JSONLinesResponse(req);
void ProgressiveService.streamProgressiveResponse(response);
return response;
// ...

The full controller implementation with typing and validation looks like this:


import { get, JSONLinesResponse, prefix, type VovkIteration } from 'vovk';
import { withZod } from 'vovk-zod';
import { z } from 'zod';
import ProgressiveService from './ProgressiveService';
 
@prefix('progressive')
export default class ProgressiveController {
  @get()
  static streamProgressiveResponse = withZod({
    validateEachIteration: true,
    iteration: z.union([
      z.strictObject({
        users: z.array(
          z.strictObject({
            id: z.number(),
            name: z.string(),
          })
        ),
      }),
      z.strictObject({
        tasks: z.array(
          z.strictObject({
            id: z.number(),
            title: z.string(),
            completed: z.boolean(),
          })
        ),
      }),
    ]),
    async handle(req) {
      const response = new JSONLinesResponse<VovkIteration<typeof ProgressiveController.streamProgressiveResponse>>(
        req
      );
 
      void ProgressiveService.streamProgressiveResponse(response);
 
      return response;
    },
  });
}

The server-side code is ready. Now we can implement the client-side code to consume this progressive response. For this, we will use the progressive function from the vovk package, which creates a promise for each property of the resulting object. It accepts the RPC method to call (e.g., ProgressiveRPC.streamProgressiveResponse) and optional input parameters. The function returns an object with promises per property, which can be awaited separately.


const { users: usersPromise, tasks: tasksPromise } = progressive(ProgressiveRPC.streamProgressiveResponse);

If the RPC method requires input parameters, you can pass them as the second argument:


const { users: usersPromise, tasks: tasksPromise } = progressive(ProgressiveRPC.streamProgressiveResponse, {
  params: { id: '123' },
  body: { hello: 'world' },
});

After that, the promises can be awaited separately, and the data will be available as soon as the corresponding JSON line is received from the server:


usersPromise.then(console.log).catch(console.error);
tasksPromise.then(console.log).catch(console.error);

Behind the scenes, progressive returns a Proxy that implements a get trap to return a promise for each accessed property.

When a new JSON line arrives, the corresponding promise resolves with that data.
If a JSON line arrives for a property without an existing promise, the promise is created and resolved (so it can be retrieved later).
When the response closes, all unsettled promises are rejected with an error indicating that the connection closed before sending a value for that property.
If the response errors, all unsettled promises are rejected with that error.