Manage source routes with root tools

[onechat] / README.rst

diff --git a/README.rst b/README.rst
index 0cc4b1e194d9a7deaf3a32d8281b1c0bbc4b67de..36db09c371132db8ddb1455b901533b5935521dc 100644 (file)
--- a/README.rst
+++ b/README.rst
@@ -82,15 +82,19 @@ The daemon owns the root agent in ``agent/`` in the data directory and
 listens on the Unix socket ``onechat.sock`` there. In the client, each
 line is a message to the agent, the line ``/stop`` interrupts the
 running turn, the line ``/stop NAME`` stops the subagent NAME, and
 listens on the Unix socket ``onechat.sock`` there. In the client, each
 line is a message to the agent, the line ``/stop`` interrupts the
 running turn, the line ``/stop NAME`` stops the subagent NAME, and

-Ctrl-D ends the client. The client shows the replies, the tool calls,
-and the first lines of each tool result as they happen. It also shows
-each message that the user sends from another client, with the client as
-a mark, such as ``[telegram] text``. When a subagent starts, the client
-shows ``[NAME] started``, and it shows the whole report of the subagent,
-which starts with ``[NAME]``, but not the tool calls of the subagent.
-The daemon is the only process that uses the token file while it runs,
-so ``onechat login`` and ``onechat probe`` refuse to start until it
-stops.
+Ctrl-D ends the client. On terminal output, the client shows ``> ``
+after each event while idle and clears it before the next event. An
+event can interrupt a line while the user types. The client shows the
+replies, the tool calls, and the first lines of each tool result as they
+happen. Replies and subagent reports render Markdown with styles and
+wrapped tables only when the output is a terminal, and stay unchanged in
+a pipe or file. It also shows each message that the user sends from
+another client, with the client as a mark, such as ``[telegram] text``.
+When a subagent starts, the client shows ``[NAME] started``, and it
+shows the whole report of the subagent, which starts with ``[NAME]``,
+but not the tool calls of the subagent. The daemon is the only process
+that uses the token file while it runs, so ``onechat login`` and
+``onechat probe`` refuse to start until it stops.

 
 The agent runs commands with bash as the user in its working directory,
 with no sandbox. It has the rights of the user, also on the token file.
 
 The agent runs commands with bash as the user in its working directory,
 with no sandbox. It has the rights of the user, also on the token file.


@@ -130,7 +134,43 @@ prompt cache key comes from it.
 ``workdir`` is an absolute path, and its default is the home directory.
 ``agent/instructions.md`` holds the user's instructions: who they are,
 how their files are organized, and how they want work done. The prompt
 ``workdir`` is an absolute path, and its default is the home directory.
 ``agent/instructions.md`` holds the user's instructions: who they are,
 how their files are organized, and how they want work done. The prompt

-ends with them, and a change applies at the next call.
+ends with them, and a change applies at the next call. The agent can
+propose a change to the file, and it makes the change only after the
+user agrees.
+
+A permanent child agent lives in ``agent/children/NAME/`` with the same
+files as the root. The root's ``agent_create(name, responsibility)``
+tool creates and starts it with the root's model, effort, and working
+directory, and a separate random ID. A name describes the area, such as
+``House`` or ``Calendar``. It is one word of 2 to 32 ASCII letters,
+starts with a capital letter, and cannot name the root, an existing
+child, or a subagent. The daemon opens all children at startup. A
+missing or invalid child config stops startup. Only one level of
+children is supported. Remove a child by hand while the daemon is
+stopped. The root's turn state lists each child and its responsibility.
+The root passes area messages from mixed chats with sender, date, and
+words, and asks the user before it sends a child's draft to WhatsApp.
+
+Each child has its own memory, model client, usage file, subagents,
+schedules, and turn loop. Its prompt limits work to its area, asks for
+only reports that need the parent or the user, and permits an empty
+reply. Children have no WhatsApp tools and draft replies instead of
+sending them. Clients address only the root. A child's saved schedules
+and subagents can resume at startup. The ready log line lists the child
+names.
+
+A message from a parent to its own child enters the child's log as
+``user`` and wakes it. A child's final reply reaches the parent as
+``work`` with the prefix ``[NAME] `` and wakes it. An empty reply sends
+nothing. A busy recipient takes the message between tool calls or in its
+next turn. ``agent/messages.json``, with mode 0600, keeps messages until
+their recipient finishes its turn. A daemon stop preserves queued
+messages and reports for the next start without a second log entry. An
+interrupted turn can repeat its model calls and tools after restart.
+Backups lock this queue with the agent files, including children made
+during the daemon run. The root uses ``tell`` for both children and
+subagents. ``zoom("Name")`` gives a child's view or a subagent's whole
+log. ``spawn`` starts short-lived subagents, not permanent children.

 
 **************
  User service
 
 **************
  User service


@@ -217,6 +257,128 @@ login. To stop it, or also remove its start at login:
    systemctl --user stop onechat.service
    systemctl --user disable onechat.service
 
    systemctl --user stop onechat.service
    systemctl --user disable onechat.service
 

+********************
+ Backup and restore
+********************
+
+Back up the running daemon to one mode 0600 ``.tar.gz`` archive. The
+command prints its path and size in bytes. With no daemon, it fails.
+
+.. code:: console
+
+   onechat backup BACKUP_FILE.tar.gz
+   ONECHAT_DIR=EMPTY_DATA_DIR onechat restore BACKUP_FILE.tar.gz
+
+The daemon flushes memory and holds its write locks during the agent
+copy. It also flushes and locks the child agents and all subagent logs.
+It copies the WhatsApp session with SQLite ``VACUUM INTO``, not as raw
+database files. It then compresses the copy and installs the archive
+through a temporary file and a rename. Use a backup path outside the
+data directory.
+
+The archive includes configuration, instructions, the log, tree,
+indexes, views, layout, schedules, subagents, children, Telegram offset,
+tokens, and WhatsApp state and session. It excludes sockets, lock files,
+``probe/``, and the ``agent-backup-*`` import copies. Links and special
+files fail, rather than enter the archive. **The archive holds
+secrets**, including login tokens, the Telegram bot token, and the
+WhatsApp link. Keep it private. There is no encryption or remote copy.
+
+Restore needs a stopped daemon and an empty destination. It checks the
+token lock and validates the archive before it installs files, with
+their modes. It rejects unsafe paths, links, and special files. A failed
+restore can leave ``auth.lock`` in the destination. Use another empty
+directory for a retry. Do not run the source and restored WhatsApp links
+at the same time. Check the restored log and view before you use it in
+place of the source.
+
+For a daily local archive with the last 14 kept, save this script as
+``BACKUP_SCRIPT``. Replace ``BACKUP_DIR`` with a private archive
+directory outside the data directory. Replace ``ONECHAT_BINARY`` with
+the absolute path through the stable ``result-service/bin/onechat``
+package link. Make the archive directory with mode 0700 before you
+enable the timer.
+
+.. code:: sh
+
+   #!/bin/sh
+   set -eu
+   umask 077
+   cd BACKUP_DIR
+   ONECHAT_BINARY backup "onechat-$(date +%F).tar.gz"
+   find . -maxdepth 1 -type f -name 'onechat-????-??-??.tar.gz' -printf '%f\n' |
+     sort -r | tail -n +15 |
+     while IFS= read -r file; do
+       rm -- "$file"
+     done
+
+Save these units as ``onechat-backup.service`` and
+``onechat-backup.timer`` in the systemd user unit directory. Replace
+``DATA_DIR`` and ``BACKUP_SCRIPT`` with absolute paths. The service uses
+the daemon unit from User service above.
+
+.. code:: ini
+
+   [Unit]
+   Description=Back up OneChat data
+   Requires=onechat.service
+   After=onechat.service
+
+   [Service]
+   Type=oneshot
+   Environment="ONECHAT_DIR=DATA_DIR"
+   UMask=0077
+   ExecStart=/bin/sh BACKUP_SCRIPT
+
+.. code:: ini
+
+   [Unit]
+   Description=Back up OneChat each day
+
+   [Timer]
+   OnCalendar=daily
+   Persistent=true
+
+   [Install]
+   WantedBy=timers.target
+
+Enable the timer with the user manager. A failed backup leaves the
+existing archives in place and skips removal of old archives.
+
+.. code:: console
+
+   systemctl --user daemon-reload
+   systemctl --user enable --now onechat-backup.timer
+   systemctl --user list-timers onechat-backup.timer
+
+************
+ Daemon log
+************
+
+The daemon writes short events with the part prefix and ordered
+``key=value`` attributes. An error cause comes last. Normal events use
+info, failures that can be retried or lose nothing use warning, and
+failures that lose data or stop work use error.
+
+With ``JOURNAL_STREAM`` set, each line carries a syslog priority without
+a time stamp or level word. The user journal can show only warnings and
+errors:
+
+.. code:: console
+
+   journalctl --user -u onechat -p warning
+
+The summary warning names its node. A schedule run has ``late`` only
+when it ran late:
+
+.. code:: text
+
+   <4>onechat: summary failed level=0 index=0: fake failure
+   <6>onechat: schedule ran id=1
+   <6>onechat: schedule ran id=2 late=1m0s
+
+Outside the journal, each line starts with the time stamp and level.
+

 **********
  Telegram
 **********
 **********
  Telegram
 **********


@@ -281,7 +443,8 @@ device. The terminal client and Telegram take three commands:
 
 ``/whatsapp status``
    Shows the link: not linked, linked and connected, or linked and
 
 ``/whatsapp status``
    Shows the link: not linked, linked and connected, or linked and

-   disconnected, with the times of the connection.
+   disconnected, with the times of the connection and the WhatsApp
+   routes.

 
 ``/whatsapp unlink``
    Ends the link and deletes the session. When WhatsApp is disconnected,
 
 ``/whatsapp unlink``
    Ends the link and deletes the session. When WhatsApp is disconnected,


@@ -356,17 +519,64 @@ agent has the WhatsApp tools, not its subagents.
 Each batch of a group goes to the chat, also with no turn, and the
 memory tree summarizes it with model calls, which use the quota.
 
 Each batch of a group goes to the chat, also with no turn, and the
 memory tree summarizes it with model calls, which use the quota.
 

+The root manages source routes with ``route(source, agent)``,
+``unroute(source)``, and ``routes()``, only after the user agrees. A
+source is ``whatsapp:CHAT-ID``, with the ID from ``whatsapp_find`` or a
+``[WhatsApp]`` message, and the agent must be its child. The root keeps
+the map in ``agent/routes.json`` with mode 0600. Children and subagents
+have no route tools. There is no command to change routes.
+

 When the session holds a linked device, the daemon connects at its
 start. If the first connection fails, it tries again after 2 seconds,
 then doubles the wait up to 5 minutes. The inbox and commands stay
 When the session holds a linked device, the daemon connects at its
 start. If the first connection fails, it tries again after 2 seconds,
 then doubles the wait up to 5 minutes. The inbox and commands stay

-active during each wait. An unlink, logout, or pair command stops these
-retries. After a network failure of an open connection, the library
-connects again by itself. When a disconnection lasts 10 minutes, all
-clients get one notice, and one more when the connection comes back.
-When WhatsApp ends the connection for good, for example because another
-program uses the same session, all clients get a notice at once. When
-the user removes the device in the phone, the daemon deletes the session
-and tells all clients one time.
+active during each wait. An unlink or a logout stops these retries.
+After a network failure of an open connection, the library connects
+again by itself. When a disconnection lasts 10 minutes, all clients get
+one notice, and one more when the connection comes back. When WhatsApp
+ends the connection for good, for example because another program uses
+the same session, all clients get a notice at once. When the user
+removes the device in the phone, the daemon deletes the session and
+tells all clients one time.
+
+*************************
+ Import WhatsApp history
+*************************
+
+Stop the daemon and finish all subagents before an import. Export each
+chat from WhatsApp as a ZIP with ``_chat.txt``. Keep the chat name in
+the ZIP name, such as ``WhatsApp Chat - Example.zip``. Read the plan
+first:
+
+.. code:: console
+
+   onechat import -n CHAT.zip OTHER.zip
+   onechat import CHAT.zip OTHER.zip
+
+``-n`` makes no model call and prints the chats, chat-day batches, local
+dates, cutoff, text bytes, merged message count, and estimated calls.
+The import keeps multiline text, system lines, and omitted-media
+markers. It maps the session's profile name to ``You``. Without that
+name, it keeps the senders as written and says so in the plan. An exact,
+unambiguous local contact or batch name supplies the chat ID when known.
+No network lookup is made.
+
+Each chat and local day becomes one work message marked ``(imported)``.
+Only messages before the first message of the earliest live WhatsApp
+batch enter the import. Live times have minute precision, so the cutoff
+excludes that whole minute. A chat-day already imported is skipped.
+Overlapping chat-days in the supplied ZIPs are refused. Review the
+exports before import: their text goes to the model as chat history.
+
+The import merges the full log in date order and builds its tree with
+real compactor calls, the agent prompt, and the agent cache key. It uses
+a limit of 1,500 calls and the probe's 15-quota-point limit, and prints
+the same usage report. Its estimate is about 1.3 calls per merged
+message, not a cost guarantee. A stopped or failed run leaves the agent
+unchanged. A new run starts over. Success keeps the other agent files
+and renames the old directory to ``agent-backup-TIMESTAMP`` before it
+installs the new one. Keep the backup until the new memory is checked,
+then start the daemon. Message IDs change to their positions in the
+merged log.

 
 ***********
  Schedules
 
 ***********
  Schedules


@@ -377,7 +587,8 @@ such as "remind me tomorrow at 9" or "every weekday at 8, tell me what
 my WhatsApp groups talked about". A schedule has a text, the task for
 the agent, and either one time or a cron expression. The agent also
 lists the schedules with their next times, and removes one when the user
 my WhatsApp groups talked about". A schedule has a text, the task for
 the agent, and either one time or a cron expression. The agent also
 lists the schedules with their next times, and removes one when the user

-asks. Only the agent has the schedule tools, not its subagents.
+asks. The root and its children have the schedule tools, not their
+subagents. Each schedule records its owner and wakes only that agent.

 
 A cron expression has five fields: minute, hour, day of month, month,
 and day of week, where 0 and 7 are Sunday. A field is a comma list of
 
 A cron expression has five fields: minute, hour, day of month, month,
 and day of week, where 0 and 7 are Sunday. A field is a comma list of


@@ -457,6 +668,9 @@ Build and run:
    nix build
    ./result/bin/onechat
 
    nix build
    ./result/bin/onechat
 

+``onechat version`` prints ``yyyy-mm-dd-gitshort`` for a Nix build, adds
+``-dirty`` for a dirty tree, and prints ``dev`` for a direct Go build.
+

 Format and check all files:
 
 .. code:: console
 Format and check all files:
 
 .. code:: console