Send stored images to WhatsApp chats

[onechat] / README.rst

diff --git a/README.rst b/README.rst
index b17cd64349b05077ef6a0c4d6e7db8976c90e154..bbb4475d3cc89d416bc62dbf898ca00b7bc3e8bc 100644 (file)
--- a/README.rst
+++ b/README.rst
@@ -3,31 +3,23 @@
 #########
 
 OneChat is one AI agent in a single chat that never ends. A daemon owns
 #########
 
 OneChat is one AI agent in a single chat that never ends. A daemon owns

-the agent and its files, and a terminal client and a Telegram bot talk
-to it. It is in early development. Linux is supported.
+the agent and its memory, and you talk to it from a terminal client,
+Telegram, and through your WhatsApp chats. It is in early development.
+Linux is supported.

 
 

-****************
- Data directory
-****************
-
-OneChat keeps its data and its token file in one directory. It uses the
-first of these:
-
-#. ``ONECHAT_DIR``, when it is set.
-#. ``$XDG_DATA_HOME/onechat``, when ``XDG_DATA_HOME`` is an absolute
-   path. A relative value is ignored.
-#. ``~/.local/share/onechat``. With no home directory, OneChat exits
-   with an error.
+*******
+ Setup
+*******

 
 

-OneChat creates the directory with mode 0700.
+Data directory
+==============

 
 

-***************
- Configuration
-***************
+OneChat keeps all its data in one directory, with mode 0700. It uses
+``ONECHAT_DIR`` when it is set, else ``$XDG_DATA_HOME/onechat``, else
+``~/.local/share/onechat``. The development shell sets ``ONECHAT_DIR``
+to ``.devenv/onechat`` in the checkout.

 
 

-``config.json`` in the data directory holds the settings. OneChat does
-not create it. A missing file or field takes the default, and an unknown
-field is an error.
+Optional settings go in ``config.json`` in the data directory:

 
 .. code:: json
 
 
 .. code:: json
 


@@ -36,274 +28,221 @@ field is an error.
      "compactor": {"model": "gpt-6.1-sol", "effort": "high"}
    }
 
      "compactor": {"model": "gpt-6.1-sol", "effort": "high"}
    }
 

-``endpoint``
-   ``chatgpt`` (the default) sends model calls to the ChatGPT backend
-   with the subscription login. ``api`` sends them to the OpenAI API
-   with the key in ``OPENAI_API_KEY``, which must then be set. With
-   ``chatgpt``, OneChat ignores ``OPENAI_API_KEY``.
-
-``compactor``
-   The model and the reasoning effort of the summaries. The defaults are
-   ``gpt-6.1-sol`` and ``high``.
-
-*******
- Login
-*******
+``endpoint`` is ``chatgpt`` (the ChatGPT subscription, the default) or
+``api`` (the OpenAI API, with ``OPENAI_API_KEY`` set). ``compactor`` is
+the model and effort of the memory summaries.

 
 

-The ``chatgpt`` endpoint needs a login with a ChatGPT account:
+Login
+=====

 
 .. code:: console
 
    onechat login
 
 
 .. code:: console
 
    onechat login
 

-The command shows a link and a code. Open the link on any device, enter
-the code, and approve the login. Device code login must be allowed in
-the ChatGPT security settings. OneChat writes the tokens to
-``auth.json`` in the data directory, with mode 0600, and refreshes them
-before they expire. The file belongs to OneChat only. Do not share it
-with the Codex CLI, because each program rotates the refresh token and
-logs the other out.
+Open the link on any device and enter the code. Device code login must
+be allowed in the ChatGPT security settings. Do not share the login with
+the Codex CLI: each program logs the other out.

 
 

-*******
- Agent
-*******
+Telegram
+========

 
 

-Start the daemon, and then the client in another terminal:
+#. Make a bot with BotFather and keep its token.
+#. Find your numeric Telegram user ID.
+#. Write both to ``telegram.json`` in the data directory, with mode
+   0600:

 
 

-.. code:: console
+.. code:: json

 
 

-   onechat daemon
-   onechat chat
-
-The daemon owns the root agent in ``agent/`` in the data directory and
-listens on the Unix socket ``onechat.sock`` there. In the client, each
-line is a message to the agent, the line ``/stop`` interrupts the
-running turn, the line ``/stop NAME`` stops the subagent NAME, and
-Ctrl-D ends the client. The client shows the replies, the tool calls,
-and the first lines of each tool result as they happen. It also shows
-each message that the user sends from another client, with the client as
-a mark, such as ``[telegram] text``. When a subagent starts, the client
-shows ``[NAME] started``, and it shows the whole report of the subagent,
-which starts with ``[NAME]``, but not the tool calls of the subagent.
-The daemon is the only process that uses the token file while it runs,
-so ``onechat login`` and ``onechat probe`` refuse to start until it
-stops.
-
-The agent runs commands with bash as the user in its working directory,
-with no sandbox. It has the rights of the user, also on the token file.
-
-When the user asks for it, the agent starts subagents for long tasks.
-Each subagent has a name, such as ``GreenOtter``, and runs in the
-background while the agent answers other messages. It gets the view of
-the agent and its task, and it runs its tools as the agent does, in
-``agent/subagents/NAME/work``. A task in a repository gives the path of
-the repository. The steps of a subagent go to its own log in
-``agent/subagents/NAME``, not to the chat, and its final reply comes
-back to the agent as one message that starts with ``[NAME]``. The agent
-can send a subagent more messages: a running subagent reads them between
-its tool calls, and a finished one goes on from its log, and its reply
-comes back in the same way. When the user stops a subagent, the chat
-gets a note, and the report of the subagent is logged, but it starts no
-turn. Each model call of a subagent has the name in the ``agent`` field
-of ``agent/usage.jsonl``. ``agent/work.json`` holds the names of the
-running subagents. A subagent that runs when the daemon stops goes on at
-the next start: the chat gets the note that the daemon restarted, and
-the subagent gets the same note and goes on from its log.
-
-``agent/config.json`` holds the settings of the agent. The first start
-writes it with the defaults and a random ID. Do not change the ID: the
-prompt cache key comes from it.
+   {"token": "123456:ABC-token-from-BotFather", "owner": 123456789}

 
 

-.. code:: json
+Start the daemon and write to the bot in a private chat. It answers only
+you.

 
 

-   {
-     "name": "OneChat",
-     "id": "0123456789abcdef0123456789abcdef",
-     "model": "gpt-6.1-sol",
-     "effort": "medium",
-     "workdir": "/home/user"
-   }
+WhatsApp
+========

 
 

-``workdir`` is an absolute path, and its default is the home directory.
-``agent/instructions.md`` holds the user's instructions: who they are,
-how their files are organized, and how they want work done. The prompt
-ends with them, and a change applies at the next call.
+Link your WhatsApp account as a linked device from the terminal client
+or Telegram:

 
 

-**********
- Telegram
-**********
+``/whatsapp pair PHONE``
+   Gives a pairing code. On the phone, open Linked devices, Link a
+   device, Link with phone number instead, and enter the code.

 
 

-The daemon also connects the agent to a Telegram bot when
-``telegram.json`` exists in the data directory. It polls the Bot API for
-new messages, so it needs no open port.
+``/whatsapp status``
+   Shows the link and the routes of chats to child agents.

 
 

-#. Make a bot with BotFather and keep its token.
+``/whatsapp unlink``
+   Ends the link.

 
 

-#. Find your numeric Telegram user ID.
+The session file gives full access to the account. If it leaks, remove
+the device in Linked devices on the phone.

 
 

-#. Write the token and the ID to ``telegram.json``, and give the file
-   mode 0600, because the token is a secret. The daemon refuses the file
-   with any other mode.
+User service
+============

 
 

-#. Start the daemon, open a private chat with the bot, and write.
+Run the daemon as a systemd user service. Keep the ``result-service``
+link, so that Nix keeps the package:

 
 

-.. code:: json
+.. code:: console

 
 

-   {"token": "123456:ABC-token-from-BotFather", "owner": 123456789}
+   nix build --out-link result-service
+   systemctl --user link --force "$PWD/result-service/share/systemd/user/onechat.service"
+   systemctl --user edit onechat.service

 
 

-The bot reads only the private chat of the owner and ignores all other
-updates. Each text message goes to the agent as a user message, the line
-``/stop`` interrupts the running turn, and the line ``/stop NAME`` stops
-the subagent NAME. The bot sends the replies in parts of at most 4096
-characters, with their Markdown as Telegram formats: bold, italic,
-strikethrough, code, code blocks, links, and headings as bold text. When
-Telegram refuses the formats of a part, the bot sends the part again as
-plain text. Each error goes as a short message in plain text. It shows
-the typing action while a turn runs. It does not send the tool calls.
-For a subagent it sends only ``[NAME] started`` and ``[NAME] finished``,
-``[NAME] stopped``, or ``[NAME] failed``, because the reply of the agent
-carries the content of the report. It sends each message that the user
-types in the terminal client, with the mark ``[terminal]``.
-
-``agent/telegram-offset.json`` holds the ID of the next update. The bot
-moves it past an update only after the message is durable in the log, so
-a restart of the daemon does not lose a message. A crash just after the
-write of a message can make the message repeat once. A network failure
-or a rate limit makes the bot wait and try again, and does not stop the
-daemon.
-
-**********
- WhatsApp
-**********
-
-The daemon connects to the WhatsApp account of the user as a linked
-device. The terminal client and Telegram take three commands:
+In the editor, set the data directory, and the tools that the agent may
+use in its shell:

 
 

-``/whatsapp pair PHONE``
-   Links the account of PHONE, the number with its country code, such as
-   ``+15550001111``. The daemon answers with the steps and then with a
-   pairing code in its own message: in WhatsApp on the phone, open
-   Linked devices, tap Link a device, then tap Link with phone number
-   instead, and enter the code. The code expires in about 2 minutes, and
-   the same command gives a new code. When the link succeeds, the daemon
-   answers with the linked number.
+.. code:: ini

 
 

-``/whatsapp status``
-   Shows the link: not linked, linked and connected, or linked and
-   disconnected, with the times of the connection.
+   [Service]
+   Environment="ONECHAT_DIR=DATA_DIR"
+   Environment="PATH=TOOLS_PATH"

 
 

-``/whatsapp unlink``
-   Ends the link and deletes the session. When WhatsApp is disconnected,
-   the phone does not get the logout, so remove the device in Linked
-   devices on the phone.
-
-The answers go only to the client that asked, and the commands do not go
-to the agent, so the agent never gets the phone number or the code. The
-chat gets only the notes that WhatsApp is linked or unlinked.
-
-The session is ``whatsapp/session.db`` in the data directory, with mode
-0600 in a directory with mode 0700. It gives full access to the WhatsApp
-account, so the daemon refuses the file when its mode is not 0600. The
-shell of the agent runs as the user, so it can read the session. If the
-session leaks, remove the linked device in WhatsApp on the phone, and
-the session stops working. Without the file, WhatsApp is not linked.
-
-The daemon reads the text messages of the private chats and the groups,
-from other people and from the user. It does not read media, reactions,
-edits, deletions, view-once and disappearing messages, or the history
-from before the link. The messages of one chat form one batch until the
-chat is quiet for 20 seconds, for 2 minutes or 50 messages at the most.
-Each batch goes to the chat of the agent as one work message that starts
-with ``[WhatsApp]``, with the name and the ID of the chat, and the time,
-the sender, and the text of each message. The messages of the user show
-as ``You``. ``whatsapp/state.json``, with mode 0600, keeps the batches
-that are not in the chat yet, so they survive a restart. A crash just
-after a batch goes to the chat can make the batch repeat once.
-
-A batch of a private chat with a message of another person starts a
-turn, and so does a batch of a group with a message that mentions the
-user or replies to a message of the user. The other batches go to the
-chat with no turn, and the agent sees them at its next turn. A batch
-that starts a turn ends with a line that tells the agent to end its turn
-with an empty reply when nothing in the batch needs the user. No client
-shows an empty reply, so such a batch gives no message in Telegram. The
-terminal client shows each batch, and Telegram shows none.
-
-The phone number of the user never goes to the chat: a mention of the
-user shows as ``@you``, the number shows as ``OWN``, and the chat of the
-user with itself is not read. But the IDs of the other chats go to the
-chat and to the model, because the agent needs them to send. The ID of a
-private chat holds the phone number of the contact, such as
-``15550002222@s.whatsapp.net``, and the ID of a group can hold the
-number of the person who made it.
-
-Each batch of a group goes to the chat, also with no turn, and the
-memory tree summarizes it with model calls, which use the quota.
-
-When the session holds a linked device, the daemon connects at its
-start, and after a network failure it connects again by itself. When a
-disconnection lasts 10 minutes, all clients get one notice, and one more
-when the connection comes back. When WhatsApp ends the connection for
-good, for example because another program uses the same session, all
-clients get a notice at once. When the user removes the device in the
-phone, the daemon deletes the session and tells all clients one time.
+Then start it, and let it run while you are logged out:

 
 

-*******
- Probe
-*******
+.. code:: console
+
+   systemctl --user daemon-reload
+   systemctl --user enable --now onechat.service
+   loginctl enable-linger "$USER"

 
 

-``onechat probe LOG COUNT [BUILDS]`` measures the compactor on a log. It
-appends the first COUNT messages of LOG, one JSON object ``{"kind",
-"text"}`` on each line, to a new chat in ``probe/`` in the data
-directory, and builds its tree with real model calls. The calls use the
-quota of the account. The run stops at the first failed call, after 660
-calls, or when the used percent of a quota window grows by more than 15
-points. At the end it reports the calls per message, the cached share of
-the input tokens, the retries, the line sizes, and the quota at the
-first and the last call. BUILDS, from 1 to 8, limits the model calls at
-once in the run.
+After a ``git pull``, ``bin/service-update`` builds and restarts it.
+Never run two daemons on the same data.

 
 

-A test writes a generated log of readable made-up messages:
+Backups
+=======

 
 .. code:: console
 
 
 .. code:: console
 

-   ONECHAT_PROBE_LOG=probe.jsonl go test -run TestProbeLogHasTheSyntheticMix
+   onechat backup BACKUP_DIR/onechat.tar.gz
+   ONECHAT_DIR=EMPTY_DIR onechat restore BACKUP_DIR/onechat.tar.gz

 
 

-*************
- Development
-*************
+A backup runs while the daemon runs. Restore needs a stopped daemon and
+an empty directory. **An archive holds your logins, the bot token, and
+the WhatsApp link**, so keep it private.

 
 

-Nix supplies the Go toolchain and all tools. Enter the development shell
-with direnv:
+For a daily backup with the last 14 kept, save this script, with
+``BACKUP_DIR`` replaced by a private directory outside the data
+directory:
+
+.. code:: sh
+
+   #!/bin/sh
+   set -eu
+   umask 077
+   cd BACKUP_DIR
+   onechat backup "onechat-$(date +%F).tar.gz"
+   ls -1 onechat-????-??-??.tar.gz | sort -r | tail -n +15 | xargs -r rm --
+
+and run it from a ``Type=oneshot`` user service with the same
+``ONECHAT_DIR`` and an ``OnCalendar=daily`` timer.
+
+*********
+ Talking
+*********

 
 .. code:: console
 
 
 .. code:: console
 

-   direnv allow
+   onechat
+
+``onechat`` connects to the running daemon. Each line is a message, and
+replies show with Markdown styles and tables. These lines are commands:
+
+``/stop``
+   Interrupts the running turn.
+
+``/stop NAME``
+   Stops the subagent NAME.
+
+``/image PATH [caption]``
+   Sends a JPEG, PNG, or WebP image. Quote a path with spaces.
+
+``/whatsapp ...``
+   The WhatsApp commands above.
+
+In Telegram, write as in the client, also with photos and image files.
+``onechat help`` lists all commands.

 
 

-Or enter it directly:
+*********************
+ What the agent does
+*********************
+
+Memory
+   The agent remembers the whole chat as summaries that it can open
+   again, so it never starts over. ``agent/instructions.md`` holds your
+   lasting preferences. The agent can propose a change to it and makes
+   the change only after you agree.
+
+Tools
+   The agent runs shell commands as you, in its working directory, with
+   no sandbox.
+
+Subagents
+   For long tasks it starts subagents in the background, such as
+   ``GreenOtter``, and gets their reports as messages.
+
+Child agents
+   A child is a permanent agent for one area of your life, such as
+   ``House``, with its own memory. The root creates it, talks to it, and
+   hears from it only what needs you. Children never send WhatsApp
+   messages; they draft replies for you to approve.
+
+WhatsApp
+   The agent reads your chats in batches and wakes for a private message
+   or a mention of you. It finds chats and sends messages only when you
+   ask. It proposes to route a chat that is only about one area to the
+   child of that area, and sets the route after you agree.
+
+Images
+   Send a photo in Telegram, WhatsApp, or with ``/image`` to the agent.
+   Its memory keeps what the image shows, and it can look at an older
+   image again. Ask it to show an image or send one to a WhatsApp chat.
+
+Schedules
+   Ask for reminders and timed tasks, such as "every weekday at 8, tell
+   me what my groups talked about". Each run uses the quota, so runs are
+   at least 5 minutes apart.
+
+Import
+   Old WhatsApp chats can join the memory in date order. Stop the
+   daemon, export the chat as a ZIP, and run the plan before the import.
+   The import rebuilds the target's memory with model calls:

 
 .. code:: console
 
 
 .. code:: console
 

-   nix develop
+   onechat import -n [--agent NAME] CHAT.zip
+   onechat import [--agent NAME] CHAT.zip

 
 

-The shell sets ``ONECHAT_DIR`` to ``.devenv/onechat`` in the checkout,
-unless it is already set. Development data goes there, and Git ignores
-``.devenv``.
+*****
+ Log
+*****

 
 

-Build and run:
+The daemon writes short events with ``key=value`` attributes to the user
+journal, at three levels: info, warning for a failure that is retried or
+loses nothing, and error for a failure that loses data or stops work.

 
 .. code:: console
 
 
 .. code:: console
 

-   nix build
-   ./result/bin/onechat
+   journalctl --user -u onechat -f
+   journalctl --user -u onechat -p warning

 
 

-Format and check all files:
+*************
+ Development
+*************
+
+Nix supplies the Go toolchain and all tools. Enter the development shell
+with ``direnv allow`` or ``nix develop``. Then:

 
 .. code:: console
 
 
 .. code:: console
 

+   nix build
+   ./result/bin/onechat version

    bin/format
    bin/check
    nix flake check
 
    bin/format
    bin/check
    nix flake check
 

-``bin/check`` and ``nix build`` run a short replay of the synthetic log.
-``nix flake check`` also runs the full replay of 100,000 messages.
+``onechat version`` prints ``yyyy-mm-dd-gitshort`` for a Nix build and
+``dev`` for a direct Go build. ``bin/check`` runs the linters and a
+short replay of a synthetic log, and ``nix flake check`` also runs the
+full replay of 100,000 messages. ``onechat probe LOG COUNT`` measures
+the memory summaries on a log with real model calls.