To resume an agent safely, persist its execution state and enough operation identity to reconcile effects that may already have happened. A saved conversation is useful context, but it does not prove which writes completed. Recovery must distinguish work that never started, confirmed effects, and attempts whose outcomes remain unknown.
The dangerous gap is small: a tool completes a write, then the process exits before saving the next checkpoint. Replaying the previous step may repeat the write even though the checkpoint looks unfinished.
Separate three kinds of state
Conversation history explains the task and prior observations. Execution state identifies the current step, pending work, counters, and dependencies. External state records what the target system actually changed. Keeping the first two does not atomically preserve the third.
LangGraph's persistence documentation distinguishes thread-scoped checkpoints from stores used across threads. Its graph guide explains that checkpoints occur at super-step boundaries, so a resumed node can run again from its beginning. These are documented framework behaviors, not a guarantee that arbitrary side effects become safe to replay.
A recovery record for a synthetic labeling operation might include:
{
"run_id": "run-7",
"step": "apply-label",
"operation_key": "run-7:label:1",
"target_id": "record-17",
"payload": {"label": "reviewed"},
"state": "pending",
"receipt": null
}Here pending means the checkpoint has no confirmed completion record. It does not mean the target is unchanged. Keep the payload or its verifiable identity so recovery cannot reuse the key for a different operation.
Make the effect identifiable before attempting it
Generate an operation key before issuing the write and persist it with the planned effect. If the service supports idempotency, reuse the same key and same payload according to that service's contract. Check retention, scope, conflict behavior, and whether retries return the original result.
If no idempotent operation exists, look for a supported way to inspect the target or operation status. A uniquely named artifact may be discoverable; a delivered message may have an authoritative receipt. If the system cannot distinguish an earlier success from an absent write, recovery may need human reconciliation instead of an automatic retry.
A read followed by a write is not universally safe. Another worker can act between them. The target needs an atomic uniqueness or idempotency mechanism, or an equivalent concurrency control, when duplicate effects matter. A coordinator's local memory cannot enforce uniqueness in an unrelated remote service.
Timeouts, retries, and idempotency examines the request-level problem. A checkpoint adds another place where acknowledgment can be lost, including after the caller already received it.
An executed interruption between effect and checkpoint
We ran a small Python program using a temporary SQLite database as a surrogate for durable state. A child process committed a pending checkpoint, committed one effect with key run-7:label:1, and deliberately exited with code 23 before marking the checkpoint complete.
A new connection then inspected the database:
| Observation | Before resume | After resume |
|---|---|---|
| Checkpoint | pending |
completed |
| Stored effects | 1 | 1 |
| Effect receipt | Stored with the effect | Recovered as receipt-1 |
The resume path found the existing key and matching payload, reused its receipt, and completed the checkpoint. An additional attempt to reuse the key with a different payload was rejected. All assertions passed.
This is a controlled process exit with committed SQLite state and one writer. It is not a power-loss test, distributed transaction, network failure, concurrent deduplication test, or LangGraph execution. In particular, the fixture's read-before-write path is not a production concurrency design; its primary-key constraint is a further boundary, but conflict handling would need to be completed and tested for concurrent callers.
The example establishes the failure window and the value of reconciliation: the checkpoint was behind reality, and recovery succeeded without adding a second effect.
Choose checkpoint durability consciously
The LangGraph checkpointer guide documents exit, async, and sync durability modes. They differ in when state is persisted relative to execution and therefore in the exposure to intermediate process failure. Confirm the behavior in the version and storage backend you deploy.
Even synchronous checkpoint persistence does not combine an arbitrary remote write and a local checkpoint into one transaction. The two systems still need an idempotent protocol, a transactional boundary they share, or reconciliation. Faster checkpointing can reduce lost computation without solving that semantic gap.
Persist only the data recovery needs. Keep credentials in the approved secret mechanism, and protect checkpoint contents because conversations and tool results may contain private data. Apply retention and deletion policies to checkpoints as well as the main application store.
Test recovery at each boundary
Inject failures before the effect, after the effect but before acknowledgment, after acknowledgment but before checkpoint persistence, and after the checkpoint. Check the final target state, not just the resumed transcript. Add a second worker if the production design permits concurrent recovery.
Then test changed inputs: a revoked permission, an edited target, a new application version, or an expired idempotency record. A checkpoint should not revive authority that no longer applies. Preserve a workflow version and explicitly handle incompatible state migrations rather than assuming today's code understands every older checkpoint.
Durable agent memory concerns knowledge worth keeping between sessions. Recovery concerns whether this particular execution can continue from a known state. Treat those as separate lifecycles, even when the same database stores both.
Documentation and local exercise: October 28, 2026. No external operation was performed, and no exactly-once guarantee is claimed for a production system.
