Measure the compactor with a probe run
onechat probe LOG COUNT appends the first COUNT messages of a log to a
new chat in probe/ in the data directory and builds its tree with real
model calls, as the live check of the compactor. After each message it
waits for the builds, as a turn does. It reports the calls per message,
the cached share of the input tokens, the retries, the sizes of the
model lines, and the quota at the first and the last call.
The run guards the quota of the account: it stops at the first failed
call, so a wrong setting cannot repeat a failing call at each message,
and it stops when the used percent of any quota window grows by more
than 15 points since the first call.
A test generates the probe log: 1,000 readable made-up messages about a
small bakery order system, with the mix of kinds and sizes of the
synthetic log of the core. It writes the log to a file when
ONECHAT_PROBE_LOG names one, and needs no network.
Issue:
b14d537