Ollama's Responses API accepts previous_response_id, returns 200, and forgets the whole conversation
Ollama’s Responses API, meant to support stateful dialogue, silently drops the previous_response_id field, making each turn appear fresh to the model. This bug leads to identical token usage for context‑aware and context‑free requests, confusing clients and breaking tool‑calling workflows. The fix…
When developers build chat‑based applications on top of Ollama, they often rely on the OpenAI‑compatible /v1/responses endpoint to preserve conversation context. The endpoint is supposed to accept a previous_response_id that lets the model remember earlier turns. In practice, that field is silently discarded, so every request starts from a clean slate. The result is that a conversation that should carry context behaves exactly like a single‑turn query, both in token count and in the answer it produces.
What the API is supposed to do
The Ollama /v1/responses API mirrors the OpenAI ChatCompletion interface. A client sends a JSON body that includes a model name, an input (the user’s message), and optionally a previous_response_id. The server should look up the conversation state associated with that ID, prepend the stored messages to the new input, and generate a reply that continues the dialogue. The response contains a previous_response_id field so that the client can chain turns.
How the bug shows up
In a test with Ollama 0.34.0 on Debian 13, the Qwen‑2.5‑1.5B model, a simple two‑turn conversation was sent:
- Turn 1:
"My secret word is PINEAPPLE. Remember it. Reply with just OK." - Turn 2:
"What is my secret word? Reply with just the word."withprevious_response_idset to the ID returned from Turn 1.
The server responded with a 200 status, status: "completed", and the answer “password”. The token usage was 41 for the second request, the same as if the client had sent the question alone without any history. When the full conversation history was included in the input array, the server returned the correct word “PINEAPPLE” and the token count jumped to 68. This discrepancy shows that the server never saw the prior turn.
Why the field is dropped
The problem lies in the Go struct that decodes the request. The struct defines a field for the model and the input, but it omits a previous_response_id field. Go’s encoding/json package ignores unknown keys unless DisallowUnknownFields is set, which it isn’t. Consequently, the previous_response_id is thrown away before the request reaches the logic that would normally restore conversation state. The response struct, however, does include a previous_response_id field, but it is always set to null with a comment “// Not supported”. Thus, the client receives a response that looks complete, but the model has never seen the earlier turn.
Impact on tool‑calling workflows
Tool‑calling introduces another layer of complexity. In a typical loop, the model emits a function_call, the client runs the tool, and then sends back a function_call_output along with the previous_response_id. Because the server discards the ID, the model receives the tool output as if it had arrived from nowhere. The reply it generates can be off‑topic or incorrect, as seen when the model responded “The test passed successfully! Is there anything else you need help with?” instead of the expected “DONE”.
What to do instead
The simplest workaround is to send the entire conversation history with every request. Instead of relying on previous_response_id, the client builds an input array that contains every user and assistant message, including any function calls and outputs. This approach costs more tokens (68 for the example above) but guarantees that the model sees the full context. Most OpenAI‑compatible clients already do this, so the issue mainly affects those that depend on the stateful API mode, such as Codex Desktop.
How to verify the bug
Token counts are the most reliable diagnostic. A request that should carry more context will have a higher input_tokens value if the server is truly stateful. If the counts are identical, the server is ignoring the history. The following script demonstrates the anomaly:
```python
import json, secrets, urllib.request
URL = "http://127.0.0.1:11434/v1/responses"
MODEL = "qwen2.5-1.5b"
word = "".join(secrets.choice("BCDFGHJKLMNPQRSTVWXZ") for _ in range(7))
plant = f"My secret word is {word}. Reply with just OK."
ask = "What is my secret word? Reply with just the word."
def post(body):
req = urllib.request.Request(URL, data=json.dumps(body).encode(), headers={"content-type": "application/json"})
return json.load(urllib.request.urlopen(req))
r1 = post({"model": MODEL, "input": plant, "stream": False})
r2 = post({"model": MODEL, "previous_response_id": r1["id"], "input": ask, "stream": False})
r4 = post({"model": MODEL, "input": ask, "stream": False})
print(word, '| via id:', r2["usage"]["input_tokens"], repr(r2["text"]), '| no history:', r4["usage"]["input_tokens"], repr(r4["text"]))
```
The output shows the same token count for r2 and r4, confirming that the server treats them identically.
Conclusion
Ollama’s /v1/responses endpoint currently does not support conversation state via previous_response_id. Clients expecting a stateful API should instead send the full conversation history with each request. The issue is tracked in the Ollama repository (issue #15954) and remains unresolved as of the latest release. Until a fix is released, developers must adopt the full‑history approach to avoid unexpected behavior and token‑count anomalies.
Why it matters
The bug undermines the reliability of chat applications built on Ollama, leading to inconsistent responses and wasted tokens. Understanding and working around it is essential for developers who rely on stateful dialogue or tool‑calling features.
Key points
- Ollama’s /v1/responses silently drops previous_response_id
- Token usage is identical for context‑aware and context‑free requests
- Full conversation history must be sent to preserve context
- The issue affects tool‑calling workflows and client libraries like Codex Desktop
- A script demonstrates the anomaly by comparing token counts
Frequently asked questions
Why does Ollama ignore previous_response_id?
The request struct in the Ollama codebase lacks a field for previous_response_id, so the JSON decoder discards it before the server can use it.
Can I still use tool‑calling with Ollama?
Yes, but you must include the full conversation history in each request, or the model will treat tool outputs as unrelated.
Is there a fix coming soon?
The issue is tracked in the Ollama repository (issue #15954) but no release date has been announced.
