Keeping an investigation going when a model provider fails

A design for saving an investigation’s progress so it can move to a different AI model and carry on.

TOKEN/401 · Four minutes

Design proposal · Not built yet

An attack that lasts four hours is likely to run into a model API that is slow, rate-limited or down. If the agent investigating it depends on one provider, the investigation stops at the worst moment, or starts again on another model that knows nothing of what was found.

One answer is to keep the investigation’s state separate from the model doing the reasoning. A checkpoint would record confirmed facts, open hypotheses, proposed actions and the current plan. This is a proposal for the future agent layer. Our current work is on Tacit.

“If a provider fails, the investigation moves to another model and continues from its last checkpoint.”

A routing layer would watch whether each provider is available and hand the checkpoint to another model when needed. Saving the state is only half the problem: the new model also has to interpret it the same way.

Running two models in parallel could help with some steps, but it costs more and the models can disagree. An implementation would need clear rules for choosing between answers, and tests showing that the second model actually improves the result.

The goal is to keep the working context through an outage and resume the investigation when a model is available again. None of this is built yet. Offline containment, automatic recovery and switching providers mid-investigation are all proposals.

End of the essay