From: Anderson A. Giacomolli <redacted>
Date: Fri, 6 Nov 2026 23:27:35 +0000 (-0300)
Subject: Lay out turn requests for the prompt cache
X-Git-Url: https://git.giacomolli.uk/onechat/commitdiff_plain/e0ab6220b6fd35f7e92a04e4bf149940185713ab?ds=sidebyside;hp=e0ab6220b6fd35f7e92a04e4bf149940185713ab

Lay out turn requests for the prompt cache

The ChatGPT backend writes a cache entry only at the end of the latest
message of a request, and a later request reads it only at one of its
own message ends. The first request of a turn now puts the stable parts
first and the per-turn state last: the prompt and the tools, then the
view as user messages, then a message with the end of the view and the
date, then each new message.

The view messages only grow between batches. layout.json in the agent's
directory keeps the line count and a hash of each view message sent. The
next request sends the same messages while their lines give the same
text, and the new lines follow as one new message. At the first message
that differs, such as after a batch merge, the rest of the view becomes
one new message.

After the turns, when the builds are done and no message waits, a
warm-up request sends only the prompt, the tools, and the view messages.
It ends at the view, where the next turn has a message end, so that turn
can read the view from the cache. The warm-up stops reading at its first
output item, after the input is read, to save output tokens, and its
output is never logged or used. Each usage record names the request of a
turn: first, next, or warmup.

Issue: 5daf23e
---