budget: count tool usage without rebuilding every call and reply #136
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "land/santi-tui-direct-relationship"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
budget: count tool usage without rebuilding every call and reply
Reading the context budget took twenty seconds on a strand with five hundred
tool calls. Nearly all of it was one line: usage asked the store for every
call and every reply, decoded each into a domain object, resolved each call's
turn, and deserialised each reply's payload, only to keep two numbers — how
many calls there were, and how many bytes their output came to. Around two and
a half thousand queries were issued and their results thrown away.
The store now answers that question directly. Calls are counted from the entry
ledger that already orders them, and outputs are read in batches keyed by
result, so the work is four queries and a fold rather than a query per row.
Batching is bounded because a strand may hold more results than a statement
may hold variables.
The numbers are unchanged: the same strand reports the same 524 calls and the
same 2346765 bytes before and after, checked against the previous
implementation rather than against a rewritten estimate of it.
Twenty seconds becomes under half a second.