The first request of a turn reads the view of the turn before from the
prompt cache without a warm-up. In two live runs of 6 turns on a view of
about 31,000 tokens, turns 2 to 6 read 99.1% to 99.4% of their input
from the cache without the warm-up, and 99.3% to 99.6% with it. One call
per turn for about 0.1 points is not worth it, so the loop sends no
warm-up and the client does not stop a stream early.
The view messages that only grow stay. With the whole view in one user
message, turn 2 of a third run read nothing from the cache, and turns 3
to 6 read 99.2% to 99.5%. With the grow-only messages, each of the 10
later turns hit.
Issue:
5daf23e