Show stored images to the user

[onechat] / prompt.go

diff --git a/prompt.go b/prompt.go
index 36fda3288de8778c156fc51f35e79122b1015725..5847ee82742d07b97862db2ac07ca661ed7236c2 100644 (file)
--- a/prompt.go
+++ b/prompt.go
@@ -41,10 +41,10 @@ var promptEfforts = []string{"low", "medium", "high", "xhigh"}
 // one added paragraph on the user's instructions file. The reference names its
 // own agent, so promptText has the name of the agent and the kind reply in its
 // place. It describes only what OneChat can do, so it leaves out the spec text
-// on images, computers, and computer tasks. Its paragraph on persistent mode
-// tells the model that a final answer ends the turn. prompt adds the clause on
-// the instructions of the user only when there are instructions. It holds no
-// date and no state, so turns and compactions share its prompt cache.
+// on computers and computer tasks. Its paragraph on persistent mode tells the
+// model that a final answer ends the turn. prompt adds the clause on the
+// instructions of the user only when there are instructions. It holds no date
+// and no state, so turns and compactions share its prompt cache.
 const promptText = `You are {name}, an AI agent that works for one user in a single chat that never
 ends. Each call to you is a turn or a compaction: the view below is followed by
 the user's new message, or by a task starting "Compaction:".
@@ -76,6 +76,9 @@ Tools:
 - zoom("Name") gives an agent's whole chat
 - date(id) gives the date and time of message id
 
+An [image FILE] line names a stored image. New images include pixels for this
+turn only. Use zoom(id, 1) to see an older image again.
+
 # Turns
 
 Do the user's tasks yourself, with your tools.
@@ -155,10 +158,10 @@ Set or remove a route only after the user agrees.`
 
 // promptSubagentText is the system prompt of a subagent, from the subagent
 // prompt of the reference design: its head, then the doc of the view, without
-// the parts on devices and images. A subagent gets the view of the agent, then
-// its whole log, so it goes on from its log after a tell or a restart.
-// promptSubagent adds the clause on the instructions of the user only when
-// there are instructions.
+// the parts on devices. A subagent gets the view of the agent, then its whole
+// log, so it goes on from its log after a tell or a restart. promptSubagent
+// adds the clause on the instructions of the user only when there are
+// instructions.
 const promptSubagentText = `You are a subagent of {name}, an AI agent that works for one user in a single
 chat that never ends. {name} gave you a task. Do it yourself, with your tools.
 
@@ -189,7 +192,8 @@ was made from; zoom(id, 1) gives message id in full. Zoom whenever a summary
 only mentions something you need, such as what your last reply said, a
 decision, a past attempt or where a file is, before you act, guess or ask.
 date(id) gives the date and time of message id. zoom("Name") gives an agent's
-whole chat.`
+whole chat. An [image FILE] line names a stored image. Use zoom(id, 1)
+to see its pixels.`
 
 // prompt returns the system prompt of the reference with one added paragraph on
 // the user's instructions file at instructionsPath. It uses the agent name and
@@ -270,6 +274,15 @@ func promptTools() []ModelTool {
                        "message": promptStringSchema("The message."),
                }),
                Strict: true,
+       }, ModelTool{
+               Type:        "function",
+               Name:        "image_show",
+               Description: "Show a stored image to the user, only when the user asks to see it. Give its stored filename, not a path.",
+               Parameters: promptObjectSchema(map[string]any{
+                       "image":   promptStringSchema("The stored image filename from an [image FILE] line."),
+                       "caption": promptStringSchema("The caption, or an empty string."),
+               }),
+               Strict: true,
        }, ModelTool{
                Type:        "function",
                Name:        "whatsapp_find",