LLM 0.32a0: primitives for reproducible reasoning pipelines

The best engineering moves are the ones that make hard work easy later. LLM 0.32a0, Simon Willison’s alpha release of his Python/CLI LLM toolkit, is one of those moves: rather than chasing model-by-model feature maps, it surfaces a compact set of explicit API primitives that map directly to the things you need when building repeatable reasoning systems — message lists that represent conversation state, typed streamed response parts that distinguish “thinking” from final output and tool calls, and a clear serialization format so responses can be persisted and replayed. Those primitives remove a lot of accidental complexity you otherwise have to wrestle out of vendor black boxes. (simonwillison.net) Before 0.32, llm’s mental model was the classic single-prompt/single-response: you call model.prompt(...) and get text back. That’s fine for one-off generation, but brittle as soon as you want replay, reproducibility, or to emulate other providers’ chat-style APIs. 0.32a0 flips the abstraction to two minimal, orthogonal concepts: a prompt can be a list of typed messages (user, assistant, etc.), and a response can be streamed as a sequence of typed parts (text, tool_call_name, tool_call_args, images, audio snippets, reasoning tokens). Those are the right primitives because they match what modern frontier models actually produce: interleaved narrative, internal-chain-of-thought tokens, structured tool call requests, and binary attachments — all in one stream. (simonwillison.net) The practical consequences are immediate. If you treat a conversation as an explicit list of messages, you can import/export conversations, replay them deterministically, and compose them with programmatic tooling. Want to emulate OpenAI-style chat completions? Pass a messages=[user(...), assistant(...), user(...)] list and you get exactly that shape back. Want to persist a conversation for later audit or A/B testing? The release adds Response.to_dict() and Response.from_dict(...) plus a TypedDict schema in llm/serialization.py so the whole run — not just the final text — can be stored as JSON and rehydrated later. That’s the plumbing you need for reproducible experiments and for building your own agent orchestration layer on top of a library that’s not opinionated about storage. (simonwillison.net) The other primitive — typed streamed parts — is the one that changes the surface area between models and user code. Instead of an opaque stream of tokens, you get events with types: 'text' for ordinary output, 'tool_call_name' and 'tool_call_args' for structured tool requests, plus other types for images or audio. In Simon’s examples the client can iterate over response.stream_events() (or the async variant) and dispatch when the model requests a tool, buffer structured arguments, or color “thinking” tokens differently in a CLI. Crucially, the library exposes a final primitive to actually run requested tools: response.execute_tool_calls(). That call turns the model’s structured request into deterministic function invocations in your runtime, and then you can reply with those results back into the model. That is the exact mechanism you need for turning speculative model reasoning into accountable, auditable side-effects. (simonwillison.net) Two design choices make this practical, not just possible. First, the primitives are explicit and small: messages, typed parts, and a serializable schema. Small APIs are composable; engineers can implement complex behavior (tool orchestration, multimodal pipelines, replayable chains) in user space without special vendor hooks. Second, backwards compatibility: existing prompt= calls are auto-wrapped into single-item messages arrays, so projects can migrate incrementally rather than rewire everything overnight. That’s how real ecosystems evolve — you add the minimal primitives that cover the new cases and leave the old ones working. (simonwillison.net) Why does this matter beyond tidy APIs? Because real reasoning stacks need reproducibility and determinism. If a model returns an internal chain-of-thought token stream, or requests a tool, or emits an image in the middle of a reply, you want to be able to capture those signals, store them, reproduce them in tests, and replay them against different tools or model versions. The typed stream and serializable response give you those capabilities. They let you implement unit tests for reasoning chains, regression tests for tool integrations, and offline audits that show exactly what tokens and tool requests led to an action. That’s how you move from “it worked once in prod” to “we can prove how and why it worked.” (simonwillison.net) There’s also a pragmatic UX win: Simon added a CLI flag to suppress reasoning tokens (-R/--no-reasoning) so the CLI can hide internal chain-of-thought from piped consumers while still recording it in the serialized response. That pattern—separating human-facing output from machine-facing traces—should be standard in any engineering workflow that mixes human review and automated tooling. It reduces accidents (people piping thinking tokens into parsers) without throwing away the trace data you need for debugging and audits. (simonwillison.net) This release is alpha for a reason: Simon will be exercising plugins and upgrading adapters over the next few days, and he’s explicit about the remaining work (a redesigned SQLite logging layer modeled as a graph to avoid duplicating conversation history). But the core move — expose message lists, typed streamed parts, and a stable serialization — is the practical plumbing everyone building reasoning systems has been waiting for. You can now implement determinism, replay, and clean tool execution in your application code rather than chasing vendor-specific streaming formats or fragile heuristics. (simonwillison.net) If you’re building agents, data pipelines, or multimodal apps, start thinking like an engineer: capture the structure, not just the text. Serialize the events, run the requested tools deterministically, and keep a copy of the raw token streams for auditing. Vendor APIs will keep evolving, but when your stack is built on explicit, composable primitives you control, those upstream changes become manageable upgrades instead of catastrophic rewrites. LLM 0.32a0 is not a flashy model release; it’s plumbing that makes the interesting engineering work — reproducible reasoning, auditable tool execution, and multimodal orchestration — actually easy to build and maintain.