Choreographing multi-agent interactions

Large language models are non-deterministic constructs that respond to ambiguous English prose. Agent harnesses make those models behave themselves in non-trivial automation use cases. However, the built-in non-determinism, when combined with non-local agent harnesses, makes the overall management of any agent topology suffer from many of the same problems as a concurrent & distributed system.

I have been experimenting with tackling the ambiguity of agentic responses by enforcing types and programming multi-agent topologies as distributed systems using the Choreographic Programming paradigm.

My experience suggests that the discipline of a strong type system, combined with an explicit description of the system’s global behaviour via Choreographic Programming, yields much better determinism than plumbing English prose and markdown documents for multi-agent autonomous workflows.

Choreographic programming (CP)

My advent into CP began with the paper on the Haskell library HasChor , which led me to working on its port in Unison - UniChorn . CP is a declarative way of writing distributed system protocols, where the author thinks globally rather than what needs to go inside each node. The system of endpoint-projection would then generate the correct node-specific code based on that high-level program. This eliminates many distributed-systems-specific issues, such as deadlocks, and helps us reason about our code much better. For proper and better explanations, check out the HasChor paper, the seminal book by Fabrizio Montesi called Introduction to Choreographies and/or this lovely zine by the group of Lindsey Kuper named Communicating Correctly with Choreography .

Recently, Lindsey Kuper et al. worked on the sequel to HasChor, MultiChor , and on libraries in Rust and TypeScript . The novel aspects in the sequel are the concepts of Census Polymorphism (or the lack of strict location fixation when writing choreographies), Conclaves (or efficient broadcast to a subset of locations), and dependency-injection-based library design for languages lacking a Haskell-like efficient type system. Details are present in this fantastic paper - Efficient, Portable, Census-Polymorphic Choreographic Programming .

I plan to upgrade UniChorn with these new features, and in the meantime, I want to experiment with CP’s application for managing multi-agent workflows in which agents may not be on the local system.

Recent frontier model enhancements and harness engineering approaches are making long-running autonomous workflows approachable for non-trivial use cases, especially in enterprise settings. The workflows deal with agents either running in managed cloud environments or with developers performing management, following the same principles of distributed system design.

I want to use CP to manage systems running these agents declaratively via CP constructs.

Constraining agents with types

Schema-bound agentic interaction is not new; check out Pydantic , Vercel’s AI-SDK , BAML , and Embabel . But I wanted to build something on my own to get into the guts of how things work from first principles. In a production setting, we could use some of the above, with one caveat: except for BAML, the others are quite complex for my taste.

I plan to base my experiments on an agent SDK that is simple yet featureful. The Pi coding-agent CLI comes with a suite of packages for working with agents programmatically. Pi is human-scale-ready as it’s what powers the venerable & disreputable – OpenClaw . And Pi’s libraries are pretty simple to work with thanks to Mario’s opinions .

I chose to leverage the dynamic tool calling conventions of modern agents to constrain the agent’s output to adhere to my custom schema. The library that I built on top of Pi SDK deals with a single function, agentTurn . It exists to make one constrained-output turn predictable: caller input is validated before the model runs, the model can use Pi read-only tools while reasoning, and the final answer is accepted only if it arrives through the named output tool and satisfies the declared TypeBox schema.

Modern harness engineering works well when it has access to more than just the agent’s output. I want my library to generate proper telemetry data provided by Pi for agentic turns.

There is more to this library with respect to system prompts, how I am handling output validation, compaction, and the use of tools & skills. I will write about them in future posts.

function agentTurn<TInput extends TSchema, TOutput extends TSchema>(options: AgentTurnOptions<TInput, TOutput>): AgentTurnHandle<Static<TOutput>>;

interface AgentTurnOptions<TInput extends TSchema, TOutput extends TSchema> {

cwd: string;

provider: string;

model: string;

reasoningLevel: ReasoningLevel;

instructions: string;

inputSchema: TInput;

inputValue: Static<TInput>;

outputSchema: TOutput;

outputToolName: string;

onTraceEvent?: AgentTurnTraceListener;

}

interface AgentTurnHandle<TOutput> {

/**

* Resolves with the final structured result or with `null` after cancellation.

*/

readonly result: Promise<AgentTurnResult<TOutput>>;

/**

* Subscribes to ordered telemetry snapshots and replays the current one immediately.

*/

subscribe(listener: (snapshot: TelemetrySnapshot) => void): () => void;

/**

* Returns the most recent telemetry snapshot without waiting for another event.

*/

getSnapshot(): TelemetrySnapshot;

/**

* Requests cancellation and waits until the run reaches a terminal state.

*/

cancel(): Promise<void>;

}

interface AgentTurnResult<TOutput> {

result: TOutput | null;

telemetry: FinalTelemetry;

}

Use Case: Ralph Loop

Ralph is an autonomous looping pattern created by Geoff Huntley . Each loop starts with a fresh context. In the actual pattern, the filesystem and git serve as the agent’s memory across iterations. However, in the following experiment, I am maintaining a dummy virtual file system that will serve as memory across iterations. Also, this only orchestrates the outer loop and is not concerned with checkpointing and HITL approvals.

Types

clancy hands over the task to ralph which leverages the agent to carry out the task, over and over again.

Note here that both clancy and ralph could be any location - local system threads, remote HTTP servers or Cloud-based serverless functions.

const locations = ["clancy", "ralph"] as const;

type Locations = (typeof locations)[number];

WorkState models the virtual filesystem where each key is a file path, and each value is ordered lines that belong in that file. Typebox-based types create in-memory JSON Schema objects which Pi directly pass to models, and hence the descriptions work as contextual prompts.

const WorkState = Type.Object(

{

__value: Type.Record(Type.String(), Type.Array(Type.String()), {

description:

"virtual filesystem and preserve entries that should remain in the updated state. maps file paths to arrays of file lines. Choose descriptive file paths and keep each array entry as one line of file content.",

}),

__brand: Type.Literal("WorkState", { description: "keep this always equal to `WorkState`" }),

},

{

description: 'Example: {"__brand":"WorkState","__value":{"answer.md":["1 + 1 = 2"]}}',

},

);

type WorkState = Type.Static<typeof WorkState>;

Task contains the todo that the user would want the agent to do + the current WorkState that Ralph would leverage during iterations.

const Task = Type.Object(

{

todo: Type.String({

description: "incoming task, read it carefully",

}),

priorWork: WorkState,

__brand: Type.Literal("Task", { description: "keep this always equal to `Task`" }),

},

);

type Task = Type.Static<typeof Task>;

Choreographies

RalphLoop is a Choreography which happens in one location ralph where the argument is the Task for the agent and the output is a located-value of type WorkState at ralph. For more information on these types, check out the MultiChor paper mentioned above. ralphAct executes the agentic turn locally , i.e. this operation happens in the ralph location.

type RalphLoop = Choreography<

"ralph",

Task,

MultiplyLocated<WorkState, "ralph">

>;

// Ralph's choreography

const ralphAct: RalphLoop = async ({ comm, locally }, task) => {

// run the agent with the task at ralph and return the located result

return locally("ralph", async (un) => {

return runRalphAgentTurn(task);

});

};

runRalphAgentTurn resolves the model selection from the environment configuration. The ralphInstructions is a use-case-specific system prompt.

/**

* Runs one schema-pi turn that turns a task into the next WorkState.

*/

async function runRalphAgentTurn(task: Task): Promise<WorkState> {

const modelSelection = resolveRalphModelSelection();

const run = agentTurn({

...modelSelection,

instructions: ralphInstructions,

inputSchema: Task,

inputValue: task,

outputSchema: WorkState,

outputToolName: "deliver_work_state",

onTraceEvent: emitRalphTrace,

});

const finalResult = await run.result;

if (finalResult.result === null) {

throw new Error("Ralph agent turn finished without a WorkState result.");

}

return finalResult.result;

}

const ralphInstructions = [

"Read the incoming task and return the complete updated WorkState after doing the requested todo.",

].join("\n");

Finally, we arrive at the important ClancyAct which is the choreography of clancy where it receives the user’s input (the todo and number of iterations) and then manages ralph’s loop to finally deliver the final WorkState located at clancy.

type ClancyAct = Choreography<

Locations,

MultiplyLocated<[string, number], "clancy">,

MultiplyLocated<WorkState, "clancy">

>;

clancyAct is a higher-order choreography, i.e. it takes another choreography as input. In this case, it’s the RalphLoop choreography.

const clancyAct = (loop: RalphLoop): ClancyAct => {

return async ({ broadcast, locally, call, comm, conclave }, input) => {

const runLoop = async (

run: MultiplyLocated<number, "clancy">,

ws: MultiplyLocated<WorkState, "clancy">,

): Promise<MultiplyLocated<WorkState, "clancy">> => {

// check if need to run the loop by comparing current iteration with numIteration

const doRunLoopAtClancy = await locally("clancy", (un) => {

const [, numIteration] = un(input);

const currentIteration = un(run);

return currentIteration < numIteration;

});

const doRunLoop = await broadcast("clancy", doRunLoopAtClancy);

// return the current work state if we don't need to loop anymore

if (!doRunLoop) {

return ws;

}

// create the task at clancy from the todo

const taskAtClancy = await locally("clancy", (un) => {

const [todo] = un(input);

return createTask(todo, un(ws));

});

// call ralph to run the loop choreography with the task at clancy

const task = await broadcast("clancy", taskAtClancy);

const resultAtRalph = await call(loop, task);

// once done, get the result from ralph at clancy

const resultAtClancy = await comm("ralph", "clancy", resultAtRalph);

// update the loop counter for the next loop

const iterationAtClancy = await locally("clancy", (un) => un(run) + 1);

// check the results at clancy

const wsAtClancy = await locally("clancy", (un) => {

const turnResult = un(resultAtClancy);

const iteration = un(run);

console.log(

`Turn Result::${iteration}:: ${JSON.stringify(turnResult.__value, null, 2)}\n\n`,

);

return turnResult;

});

// continue the loop

return runLoop(iterationAtClancy, wsAtClancy);

};

const initIterationAtClancy = await locally("clancy", () => 0);

const initWorkStateAtClancy = await locally("clancy", () => emptyWorkState);

return runLoop(initIterationAtClancy, initWorkStateAtClancy);

};

};

Here’s a sequence diagram to showcase the protocol embedded in clancyAct

Running

Finally, we come to how to run compile and run the choreographic program.

Choreographic programs are projected to respective endpoints and then communicate via custom transport backends. choreography-ts - the library I am using for my experiments - takes care epp and comes with two transport backends - transport-express and transport-socketio.

However, I was not happy with those transport backends for the real-time communication primitives required for agentic workflows. Hence, I worked on a new transport backend based on NATS protocol . The transport uses JetStream , NATS’ persistent-streaming feature that also enables decoupled flow control. I will write about the new transport-nats in future posts.

We configure the NatsTransportConfig by passing the address of the NATS broker and a unique prefix for the transport backend to use when sending messages to relevant subjects. And finally, the locations which would be the subjects in the NATS ecosystem.

async function main() {

const ts = Bun.randomUUIDv7();

const config: NatsTransportConfig<Locations> = {

servers: "nats://127.0.0.1:4222",

prefix: `ralph-loop-${ts}`,

locations,

};

const [clancyTransport, ralphTransport] = await Promise.all([

NatsTransport.create(config, "clancy"),

NatsTransport.create(config, "ralph"),

]);

To compile the choreographies, we create relevant Projector s and then epp the choreographies passing the relevant arguments.

// argumements for our use case

const todo = "create a cli tool to sum 2 numbers in python";

const numIterations = 2

// create the projectors

const clancyProjector = new Projector(clancyTransport, "clancy");

const ralphProjector = new Projector(ralphTransport, "ralph");

// `epp` and run the choreographies

try {

const [result] = await Promise.all([

clancyProjector.epp(clancyAct(ralphAct))(

clancyProjector.local([todo, numIterations] as [string, number]), // send the argument as located values in `clancy`

),

ralphProjector.epp(clancyAct(ralphAct))(clancyProjector.remote("clancy")), // a placeholder value for ralph as ralph's choreography is not concerned about the arguments

]);

// log the final output

console.log(JSON.stringify(result, null, 2));

} finally {

// cleaning up

await Promise.all([

clancyProjector.transport.teardown(),

ralphProjector.transport.teardown()

]);

}

} // end of main

Showcase

Ralph Loop Use Case Demo

Recorded by kaychaks

Next Up

I will be open-sourcing transport-nats and schema-pi (the Pi-based library for constraining agents’ output), as well as a dedicated repo to create higher-level abstractions and use-case examples of multi-agent choreographies. I plan to have everything in a monorepo (which I have locally at present).

I have many interesting use cases to try out, and I can’t wait to write about them. Until then, keep vibing 👋

If you have read this far, thanks! I am available for contract, temporary, freelance, and advisory roles where I can work closely with stakeholders or as an IC to solve hard product and engineering problems quickly. I am open to working in a variety of languages - Haskell, Rust, TypeScript/JavaScript, and, of course, Unison. I am fun to work with and have almost 20 years of experience, with expertise in functional programming, distributed systems, cloud-native architectures, and agentic workflows.