# Desktop harness — first slice

Manual SSH and X11 controls. There is **no model loop, persistent conversation, scheduler, or Jev integration yet**. The package name is a placeholder.

## On berlin (or your development computer)

Requires Node.js 22.13+ (for built-in SQLite) and an SSH client. Configure an SSH alias `home` for the VM's desktop user, with key authentication. Connect manually first to verify and save its host key:

```sh
ssh home
```

The CLI requires known host keys and noninteractive authentication. It does not bypass SSH verification.

```sh
npm install
npm run build
npm run home -- install home
```

## Inside home

Install dependencies:

```sh
sudo pacman -S --needed python python-pillow xdotool xclip
```

From a terminal **inside the logged-in XFCE desktop**, run:

```sh
python3 ~/.local/lib/desktop-harness/desktop.py --register-session
```

This records the current graphical session environment for SSH requests. Installation also creates `~/.config/autostart/desktop-harness-session.desktop`, so subsequent XFCE logins refresh registration automatically. Re-running `install` updates both the helper and its autostart entry. No periodic refresh is needed within the same session. Keep the desktop logged in and unlocked for these initial checks.

To test automatic registration, log out and back into XFCE, then capture from berlin without running the registration command manually. The helper assumes one graphical session for this account; it does not log in, unlock the screen, or choose between concurrent sessions. To disable automatic registration, remove the autostart file.

## Try capturing

From berlin:

```sh
npm run home -- capture home artifacts/first.png
```

Open the PNG. Check dimensions and that it shows the actual desktop rather than a blank or different display.

## Try input

Open Mousepad or another ordinary GUI text editor manually, focus a blank document, and create `actions.json` on berlin:

```json
{
    "actions": [
        { "type": "text", "text": "Hello from berlin!\nUnicode: café — こんにちは" },
        { "type": "wait", "milliseconds": 300 }
    ]
}
```

```sh
npm run home -- act home actions.json artifacts/typed.png
```

Verify the text and screenshot. Text insertion **replaces the clipboard** and pastes with Ctrl+V; this is for ordinary GUI text fields, not terminals (which often need Ctrl+Shift+V). For shell work use the shell command below.

Other action examples:

```json
{
    "expectedSize": [1280, 800],
    "actions": [
        { "type": "click", "x": 300, "y": 200, "button": "left" },
        { "type": "keys", "keys": ["ctrl", "a"] },
        { "type": "scroll", "direction": "down", "steps": 2 }
    ]
}
```

Use coordinates from your own screenshot, not these example coordinates. `expectedSize` is optional and rejects input if the screen size changed. Supported actions are click, scroll, keys, text, and wait. Drag is deferred. Actions are serialized by a VM-side lock; avoid manual interaction while a request runs. The full request is validated before any actions execute, but runtime failures can still leave partial effects.

## Try shell execution

```sh
npm run home -- shell home 'printf "hello\n"; uname -a'
npm run home -- shell home 'sleep 10' 2
```

The second command should time out. GNU `timeout` runs inside the VM and sends TERM, then KILL after two seconds. This is not a sandbox: commands have the desktop user's permissions, and processes that deliberately detach can escape the timeout. Output is limited to 32 MiB across stdout/stderr; exceeding it disconnects SSH and reports an unknown outcome. Commands must be one quoted local argument. Default timeout is 30 seconds, maximum 300.

## Local checks

```sh
npm run check
npm test
python3 -m unittest discover -s test -p 'test_*.py'
```

Desktop dependencies are imported only when operating the display, so validation tests can run without X11 or Pillow.

## Configuration and SQLite

```sh
cp .env.example .env
npm run build
npm run setup:check
```

`.env.example` contains berlin's llama.cpp origin and model ID. Edit `.env` for your installation. The origin excludes `/v1`; future model requests will append the API path. `DATABASE_PATH` defaults to `./data/token.sqlite`, relative to the process working directory. `.env` and database files are ignored by Git. The check validates configuration and opens/checks SQLite; it does not contact the model or operate the desktop.

`src/database.ts` exports a general `openDatabase()` connection using Node's built-in SQLite, with foreign keys, WAL, and a five-second busy timeout. The caller owns the connection and must close it. Use one connection in the eventual supervisor and pass it to domain modules; it is not tied to conversations or logs. Append numbered SQL migrations to `migrations` as actual schemas are introduced (projects, schedules, messages, etc.). Applied migration SQL is recorded and checked against subsequent builds. Pending migrations run transactionally; incompatible history fails rather than silently changing existing data. There is intentionally no speculative domain schema or ORM yet. Back up live databases using a SQLite-aware backup mechanism, not by copying only the main file while WAL is active.

## Next milestone

Once capture, Unicode insertion, key combinations, and shell timeout work on the real VM, add the llama.cpp adapter, durable tool-call records, and a single model/tool loop. Wake/sleep and restart recovery follow. No SSH connection or real graphical session is available in the development sandbox, so those checks must be run on your setup.