Carry new images through model request chains

[onechat] / prompt.go

diff --git a/prompt.go b/prompt.go
index 6bc737da08e1c18c0d027d4b72df3bd1666bdd62..df6f17d4a0c898d6ee2a0a448a916d85c0caf3c6 100644 (file)
--- a/prompt.go
+++ b/prompt.go
@@ -41,10 +41,10 @@ var promptEfforts = []string{"low", "medium", "high", "xhigh"}
 // one added paragraph on the user's instructions file. The reference names its
 // own agent, so promptText has the name of the agent and the kind reply in its
 // place. It describes only what OneChat can do, so it leaves out the spec text
 // one added paragraph on the user's instructions file. The reference names its
 // own agent, so promptText has the name of the agent and the kind reply in its
 // place. It describes only what OneChat can do, so it leaves out the spec text

-// on images, computers, and computer tasks. Its paragraph on persistent mode
-// tells the model that a final answer ends the turn. prompt adds the clause on
-// the instructions of the user only when there are instructions. It holds no
-// date and no state, so turns and compactions share its prompt cache.
+// on computers and computer tasks. Its paragraph on persistent mode tells the
+// model that a final answer ends the turn. prompt adds the clause on the
+// instructions of the user only when there are instructions. It holds no date
+// and no state, so turns and compactions share its prompt cache.

 const promptText = `You are {name}, an AI agent that works for one user in a single chat that never
 ends. Each call to you is a turn or a compaction: the view below is followed by
 the user's new message, or by a task starting "Compaction:".
 const promptText = `You are {name}, an AI agent that works for one user in a single chat that never
 ends. Each call to you is a turn or a compaction: the view below is followed by
 the user's new message, or by a task starting "Compaction:".


@@ -76,6 +76,9 @@ Tools:
 - zoom("Name") gives an agent's whole chat
 - date(id) gives the date and time of message id
 
 - zoom("Name") gives an agent's whole chat
 - date(id) gives the date and time of message id
 

+An [image FILE] line names a stored image. New images include pixels for this
+turn only. Use zoom(id, 1) to see an older image again.
+

 # Turns
 
 Do the user's tasks yourself, with your tools.
 # Turns
 
 Do the user's tasks yourself, with your tools.


@@ -149,14 +152,16 @@ Use agent_create for permanent children, whose areas are in the state.
 Use tell for children and subagents, and zoom("Name") for their memory.
 Pass area messages from mixed chats to the right child with sender, date, and
 words. Never send a WhatsApp reply drafted by a child without the user's
 Use tell for children and subagents, and zoom("Name") for their memory.
 Pass area messages from mixed chats to the right child with sender, date, and
 words. Never send a WhatsApp reply drafted by a child without the user's

-agreement.`
+agreement.
+Propose routing a source that talks only about one area to its child.
+Set or remove a route only after the user agrees.`

 
 // promptSubagentText is the system prompt of a subagent, from the subagent
 // prompt of the reference design: its head, then the doc of the view, without
 
 // promptSubagentText is the system prompt of a subagent, from the subagent
 // prompt of the reference design: its head, then the doc of the view, without

-// the parts on devices and images. A subagent gets the view of the agent, then
-// its whole log, so it goes on from its log after a tell or a restart.
-// promptSubagent adds the clause on the instructions of the user only when
-// there are instructions.
+// the parts on devices. A subagent gets the view of the agent, then its whole
+// log, so it goes on from its log after a tell or a restart. promptSubagent
+// adds the clause on the instructions of the user only when there are
+// instructions.

 const promptSubagentText = `You are a subagent of {name}, an AI agent that works for one user in a single
 chat that never ends. {name} gave you a task. Do it yourself, with your tools.
 
 const promptSubagentText = `You are a subagent of {name}, an AI agent that works for one user in a single
 chat that never ends. {name} gave you a task. Do it yourself, with your tools.
 


@@ -187,7 +192,8 @@ was made from; zoom(id, 1) gives message id in full. Zoom whenever a summary
 only mentions something you need, such as what your last reply said, a
 decision, a past attempt or where a file is, before you act, guess or ask.
 date(id) gives the date and time of message id. zoom("Name") gives an agent's
 only mentions something you need, such as what your last reply said, a
 decision, a past attempt or where a file is, before you act, guess or ask.
 date(id) gives the date and time of message id. zoom("Name") gives an agent's

-whole chat.`
+whole chat. An [image FILE] line names a stored image. Use zoom(id, 1)
+to see its pixels.`

 
 // prompt returns the system prompt of the reference with one added paragraph on
 // the user's instructions file at instructionsPath. It uses the agent name and
 
 // prompt returns the system prompt of the reference with one added paragraph on
 // the user's instructions file at instructionsPath. It uses the agent name and


@@ -230,8 +236,8 @@ func promptSubagent(name, instructions string) string {
 
 // promptTools returns the function tools of every call of the agent: the tools
 // of promptSubagentTools, then spawn, agent_create, tell, the WhatsApp tools,
 
 // promptTools returns the function tools of every call of the agent: the tools
 // of promptSubagentTools, then spawn, agent_create, tell, the WhatsApp tools,

-// and the schedule tools. A compaction sends them too and never calls them,
-// because a change of the tools ends the shared prompt cache.
+// the route tools, and the schedule tools. A compaction sends them too and
+// never calls them, because a change of the tools ends the shared prompt cache.

 func promptTools() []ModelTool {
        return append(promptSubagentTools(), ModelTool{
                Type:        "function",
 func promptTools() []ModelTool {
        return append(promptSubagentTools(), ModelTool{
                Type:        "function",


@@ -286,6 +292,29 @@ func promptTools() []ModelTool {
                        "text": promptStringSchema("The text of the message."),
                }),
                Strict: true,
                        "text": promptStringSchema("The text of the message."),
                }),
                Strict: true,

+       }, ModelTool{
+               Type:        "function",
+               Name:        "route",
+               Description: "Route a source to your child, only after the user agrees. Use whatsapp:CHAT-ID with the ID from whatsapp_find or a [WhatsApp] message.",
+               Parameters: promptObjectSchema(map[string]any{
+                       "source": promptStringSchema("The source key, whatsapp:CHAT-ID."),
+                       "agent":  promptStringSchema("The name of your child."),
+               }),
+               Strict: true,
+       }, ModelTool{
+               Type:        "function",
+               Name:        "unroute",
+               Description: "Remove a source route, only after the user agrees. New inputs go to you.",
+               Parameters: promptObjectSchema(map[string]any{
+                       "source": promptStringSchema("The source key, whatsapp:CHAT-ID."),
+               }),
+               Strict: true,
+       }, ModelTool{
+               Type:        "function",
+               Name:        "routes",
+               Description: "List source routes to your children.",
+               Parameters:  promptObjectSchema(map[string]any{}),
+               Strict:      true,

        }, ModelTool{
                Type:        "function",
                Name:        "schedule_set",
        }, ModelTool{
                Type:        "function",
                Name:        "schedule_set",