Skip to main content

Step Mode

Step mode is the serverless face of the agent system: nothing runs between messages, nothing is held in memory, and each delivery of messages for a thread is processed by one call. It exists because on a platform like AWS Lambda the platform is the scheduler. SQS batches messages per thread, invokes the function, and the function's whole job is one slice.

Build a step-mode system with AgentSystemBuilder::new, passing your platform's InputSender:

let system = AgentSystemBuilder::new(conversation_store, state_store, model, sqs_sender)
    .tools(tool_impls)
    .callback_url(callback_url)
    .rap_notifier(rap_notifier)
    .build();

The sender is the loopback path: everything the runtime wants to happen later (a child thread's seed message, a report to a parent, a timer wake-up) is sent through it instead of being called directly, which is exactly what makes the slice free to end.

Anatomy of a step​

AgentSystem::step(inputs, observer, defer) is the whole per-slice job:

let collector = EventCollector::new();
let outcomes = system
    .step(inputs, &collector, &mut NoDeferral)
    .await?;

inputs is a Vec<(InputMessage, String)>: each message paired with a stable dedup ID (on SQS, the message ID), so redeliveries are absorbed idempotently. The batch may span multiple threads — an SQS FIFO delivery with a batch size above 1 can interleave several message groups, guaranteeing order only within each group. step partitions the batch by thread and runs the per-thread steps concurrently: each loads its thread's history and dedup state from the stores, applies the deferral policy, prepares its inputs into history, runs at most one completion (with synchronous-tool loopback), syncs history durably, and dispatches at most one asynchronous tool call. Nothing is cached between calls, so the process is free to exit afterwards.

Steps must still be serialized per thread across calls, since two concurrent steps for the same thread would race each other's history writes. In-process, step takes &mut self, so one system instance runs one call at a time; across processes, serialization is the transport's job, which SQS FIFO message groups provide (Lambda never processes the same group in two invocations concurrently).

The observer receives every event as it happens, tagged with the emitting thread; EventCollector buffers (thread_id, event) pairs for inspection after the slice, which is the natural shape for a handler that turns each thread's output into a response message (see Observers).

Each thread's StepOutcome tells you whether anything happened: Skipped means every input was absorbed during preparation (duplicates, events routed to subscribing threads, messages for closed threads); Completed carries the token usage and context window, which a scheduler can use to trigger compaction.

Deferral​

While a thread is waiting on a non-passive tool call, subscription events and child reports should not barge in and interrupt the pending call. The step's deferral phase implements that policy against the DeferQueue you pass:

  • NoDeferral processes everything immediately. This is appropriate when there is nowhere durable to park events, as in a Lambda.
  • InMemoryDeferQueue holds deferred events in memory. This is what the local driver uses, flushing when the tool call settles.
  • Your own DeferQueue implementation can park events durably (a database table, a delay queue) for a step-mode platform that wants driver-grade semantics.

The Lambda embedding​

infinity-agent-lambda (src/event_handler.rs) is the production step-mode embedding and the reference to copy from. An SQS FIFO input queue keyed by thread ID provides everything the step API assumes about its transport: per-thread ordering (FIFO within a message group), automatic batching of messages that arrive together, and stable message IDs to use as dedup IDs.

Each invocation builds the system fresh, since a Lambda holds no state worth caching: DSQL and DynamoDB stores, Bedrock behind StaticModel, and the same SQS queue as the loopback sender. Tools come from a ThreadConfigSource that loads each thread's RAP toolsets (through a DynamoDB manifest cache) and adds the platform sleep tools, so a batch spanning several sessions resolves each session's own tools. The handler runs one step with NoDeferral and an EventCollector, and finally drains each thread's collected events into an output-queue message (accumulated text, tool call notices, or an OAuth challenge). Deploying on AWS Lambda covers the surrounding infrastructure.