Progressive Responses with `progressive` Function

View Progressive example on examples.vovk.dev »

JSON Lines is often used to send several chunks of data, one after another, in answer to one request. Long-running operations, such as LLM completions, use it to deliver partial results as they become ready.

When you don’t know which chunk comes first, use the experimental “progressive response”. It is inspired by Dan Abramov’s proposal Progressive JSON , which gave it its name.

Take two functions that return data after a random delay: getUsers and getTasks, static methods of a service class. In a real app, these could be API calls or queries to different databases.

With the JSONLinesResponder class, a service method sends each result when it is ready:


// ...
void Promise.all([
  this.getUsers().then((users) => resp.send({ users })),
  this.getTasks().then((tasks) => resp.send({ tasks })),
])
  .then(resp.close)
  .catch(resp.throw);
// ...

When getUsers() or getTasks() resolves, resp.send sends a JSON line to the client.
When all promises resolve, resp.close closes the response stream.
If a promise rejects, resp.throw sends an error to the client.

The full service module:

src/modules/progressive/progressive-service.ts


import type { JSONLinesResponder, VovkIteration } from 'vovk';
import type ProgressiveController from './progressive-controller.ts';
 
export default class ProgressiveService {
  static async getUsers() {
    await new Promise((resolve) => setTimeout(resolve, Math.random() * 10_000));
    return [
      { id: 1, name: 'John Doe' },
      { id: 2, name: 'Jane Smith' },
      { id: 3, name: 'Alice Johnson' },
      { id: 4, name: 'Bob Brown' },
      { id: 5, name: 'Charlie White' },
    ];
  }
 
  static async getTasks() {
    await new Promise((resolve) => setTimeout(resolve, Math.random() * 10_000));
    return [
      { id: 1, title: 'Task One', completed: false },
      { id: 2, title: 'Task Two', completed: true },
      { id: 3, title: 'Task Three', completed: false },
      { id: 4, title: 'Task Four', completed: true },
      { id: 5, title: 'Task Five', completed: false },
    ];
  }
 
  static streamProgressiveResponse(
    responder: JSONLinesResponder<VovkIteration<typeof ProgressiveController.streamProgressiveResponse>>
  ) {
    return Promise.all([
      ProgressiveService.getUsers().then((users) => responder.send({ users })),
      ProgressiveService.getTasks().then((tasks) => responder.send({ tasks })),
    ])
      .then(responder.close)
      .catch(responder.throw);
  }
}

The code above is fetched from GitHub repository.

In the controller, create a JSONLinesResponder, pass it to the service method and return it as the response.


// ...
const responder = new JSONLinesResponder<IterationType>(req);
void ProgressiveService.streamProgressiveResponse(responder);
return responder;
// ...

The full controller, with types and validation:

src/modules/progressive/progressive-controller.ts


import { procedure, get, JSONLinesResponder, prefix, type VovkIteration } from 'vovk';
import { z } from 'zod';
import ProgressiveService from './progressive-service.js';
 
@prefix('progressive')
export default class ProgressiveController {
  @get('', { cors: true })
  static streamProgressiveResponse = procedure({
    validateEachIteration: true,
    iteration: z.union([
      z.strictObject({
        users: z.array(
          z.strictObject({
            id: z.number(),
            name: z.string(),
          })
        ),
      }),
      z.strictObject({
        tasks: z.array(
          z.strictObject({
            id: z.number(),
            title: z.string(),
            completed: z.boolean(),
          })
        ),
      }),
    ]),
  }).handle(async (req) => {
    const responder = new JSONLinesResponder<VovkIteration<typeof ProgressiveController.streamProgressiveResponse>>(
      req
    );
 
    void ProgressiveService.streamProgressiveResponse(responder);
 
    return responder;
  });
}

The code above is fetched from GitHub repository.

On the client, use the progressive function from the vovk package. It takes the RPC method to call (such as ProgressiveRPC.streamProgressiveResponse) and optional input parameters. It returns an object with a promise for each property, and you await each one on its own.


const { users: usersPromise, tasks: tasksPromise } = progressive(ProgressiveRPC.streamProgressiveResponse);

If the RPC method takes input parameters, pass them as the second argument:


const { users: usersPromise, tasks: tasksPromise } = progressive(ProgressiveRPC.streamProgressiveResponse, {
  params: { id: '123' },
  body: { hello: 'world' },
});

Each promise resolves as soon as its JSON line arrives from the server:


usersPromise.then(console.log).catch(console.error);
tasksPromise.then(console.log).catch(console.error);

progressive returns a Proxy whose get trap returns a promise for each property you read.

When a JSON line arrives, the matching promise resolves with its data.
A JSON line for a property that has no promise yet creates the promise, already resolved, for you to read later.
When the response closes, every pending promise is rejected with an error saying that the connection closed before a value for that property came.
If the response fails, every pending promise is rejected with its error.

Progressive Responses with progressive Function

Progressive Responses with `progressive` Function