Chat with your hardware

wendy chat is an interactive terminal assistant for developing apps and working with hardware running Wendy. It can inspect and edit a local project, run builds and tests, discover and connect to devices, deploy apps, and inspect device state using the same tools as Wendy's MCP server.

Start chatting

Open a terminal in your project and run:

wendy chat

The first time, Wendy helps you choose where your AI runs, connect to it, and pick a model from the available models. You don't need to know a model name or set environment variables first.

• On your computer: use Ollama, LM Studio, or another local model server. Wendy looks for available models. Local inference needs no cloud account or API key.
• With a hosted provider: choose OpenAI or Anthropic. If a key is needed, Wendy prompts for it with the input hidden, then lists your available models.

Wendy remembers your selected provider, server, and model for the next wendy chat. To choose a different setup, run:

wendy chat --setup

During a conversation, type /setup to change your AI or enter a replacement API key in the private setup screen. Choosing a new setup starts a fresh conversation; canceling setup resumes your current conversation.

Once chat opens, try:

Inspect this project and discover my Wendy devices.

You can also ask it to diagnose an app's logs, add a feature, run tests, or deploy to a named device. Reads run automatically; file changes, shell commands, and device changes ask for approval.

Use a local model

Start Ollama or enable LM Studio's local server, then run wendy chat and choose On this computer. Wendy finds the server and fetches its model list so you can select a model without typing its identifier. Choose one that supports tool calling so the assistant can use files, commands, and devices.

If Ollama has no models yet, Wendy offers to download a starter model and shows its download size. Choose the download to proceed, or load a model yourself and check again. In LM Studio, download and load a model in the app first.

For llama.cpp, vLLM, or another compatible server, choose a custom local server and enter its address when prompted. Its API address usually ends in /v1:

http://localhost:8080/v1

If no server is running, setup explains how to get one and lets you check again without leaving Wendy. See Ollama's compatibility documentation or LM Studio tool use for more about server setup and model support.

The model server can also run on another reachable machine, including a Wendy device. Enter that server's address during setup. The model server and the device you control with --device are selected independently.

Configure a context window large enough for the conversation and Wendy's tool definitions. Tool accuracy depends on the model and its server's tool-call parser/template. A model that only generates plain text can chat but cannot execute Wendy tools.

Use your voice

Start with voice enabled:

wendy chat --voice

Or type /voice during a conversation to turn it on. Type /voice again to turn it off and continue typing. /voice on and /voice off also work.

Voice uses your computer's default microphone and speakers with OpenAI GPT Live (gpt-live-1). Your selected chat model still handles reasoning, files, tests, deployment, and device tools. That model can be local. Voice itself requires OpenAI access and sends microphone audio and brief task context to OpenAI, even when the reasoning model runs locally.

Wendy keeps voice brief: it listens while you work, shows tool progress in the terminal, and speaks short results or questions when your input is needed. It does not narrate each action. File changes, commands, and device changes still show their arguments for approval in the terminal; press y or n. Speech does not approve tool calls.

If needed, Wendy opens a private OpenAI key prompt. It can reuse the key from an official OpenAI chat connection, or use OPENAI_API_KEY or WENDY_VOICE_API_KEY. A separate key entered for voice is saved privately in ~/.wendy/voice.json; environment keys are not copied there. Use /voice setup to replace the voice key. OpenAI API usage is billed separately from ChatGPT.

Use headphones to avoid microphone feedback. The terminal shows when voice is on. /voice off closes the microphone, playback, and Live connection while ongoing agent work continues in text. Escape cancels active work and interrupts speech; cancellation cannot undo completed tool actions. Escape also discards an answer waiting to be spoken after the agent finishes, while keeping the answer in the transcript and voice listening for your next request. If voice fails, the text conversation remains available.

Native audio requires a CLI built with cgo on macOS, Linux, or Windows and permission to use the microphone. Builds without cgo still support text chat and explain when native voice is unavailable. Current Windows release builds disable cgo.

The integration uses the GPT Live WebSocket API with client delegation, so voice and reasoning models are selected independently.

See through a device camera

Ask Wendy to inspect a camera on a connected device:

Connect to beecam and tell me what you see through its camera.

Wendy discovers the camera, asks for approval to capture a snapshot, and sends the image to your selected model for visual inspection. Use a model that accepts images: a vision-capable local model works through a compatible server, as do supported OpenAI and Anthropic models. A text-only model cannot inspect a frame; use /setup to choose a model with vision if the provider rejects image input.

camera_snapshot takes one frame using the active device connection, including Wendy Cloud connections. It returns a bounded JPEG with camera, capture-time, and dimension metadata. It does not open a desktop preview or leave a camera stream running. The CLI uses GStreamer to decode the device video stream; if it is unavailable, the tool explains how to install it.

Snapshots go to your configured model provider, so a local model keeps visual inference local. Image bytes are kept out of terminal tool JSON and are retained only in memory, with older image attachments expiring as more are captured. Camera status alone does not prove what a camera sees. Continuous video and audio playback for the user are available through separate background tools, described below.

Open live video and listen to a device

Ask Wendy to open a live camera window or play the connected device's microphone through your computer's speakers:

Show Woof's camera in a window and let me listen to its microphone.

Chat uses camera_view and audio_listen to launch the local Wendy media commands in the background. They follow the active direct or Wendy Cloud connection and return quickly, so you can keep chatting while video or audio continues. Starting a stream asks for normal tool approval; --yes also applies to these launches.

For a specific camera or microphone, ask Wendy to select it. camera_view accepts camera_id or stable_id and optional width, height, and fps. audio_listen accepts device_id, sample_rate, channels, and buffer_ms. The local CLI needs the playback dependencies required by its camera and audio commands; launch failures include diagnostics in the job's output.

Each launch returns a job_id. The transcript shows its state: running, exited, failed, or stopped. A running process does not yet prove that a window has rendered a frame or the speakers have played audio. Ask Wendy to check playback to inspect current state and recent output with background_process_list, or say “Stop the camera and audio” to use background_process_stop. Listing and stopping jobs need no additional tool approval. Jobs remain active between chat turns and close when stopped or when Chat exits.

The live window and speakers are for you. Camera playback does not send a live video feed to the model; use camera_snapshot when you want Wendy to inspect a frame. Audio playback does not send the device microphone stream to the model. Chat's /voice mode separately uses your computer's microphone for conversation.

Select a project and device

The current directory is the project workspace. Use -C to select another directory and --device to request a direct device connection:

wendy chat -C ./my-app --device my-pi.local

You can include the first message on the command line:

wendy chat "Inspect this project and discover my devices"

The assistant uses Wendy's discovery and connection tools to establish device context. Cloud devices use your existing Wendy Cloud login (wendy auth login). Local model inference also works with cloud-connected Wendy devices.

Working in the terminal

The transcript groups consecutive tool calls and subagent activity into a single summary line that updates as work progresses. Subagent prompts and reports stay collapsed; errors and denied actions remain visible. Press Ctrl+T or type /tools to expand the full activity history, including tool arguments, results, and subagent details. The agent receives the full results in either view. Read-only operations run automatically. File writes, shell commands, and device changes show their arguments for approval before running. Approve with y or deny with n; use the scroll keys to review longer calls.

Local file tools operate within the selected workspace. Approved shell commands run with your user permissions on the developer machine; the workspace is their working directory, not a shell sandbox. The model can also use offline Wendy documentation through the wendy_docs tool.

For commands inside a running device container, Wendy uses container_exec through the active direct or cloud connection. Commands use an argument array; shell syntax requires an explicit shell command. Execution is noninteractive, bounded by a timeout and output limit, and asks for approval. Use telemetry_logs for passive log inspection. container_start may restart and interrupt the app; the legacy container_attach tool is no longer advertised.

Wendy starts with 16 core MCP tools. The assistant can enable specialist tools with wendy_tools, including setup, hardware, robotics, observability, and cloud groups. Telemetry returns compact records by default; format="otlp" returns raw batches. See MCP tools for configuration, compatibility, and workflows that still require the CLI.

Cancellation does not undo changes that have already completed. Use /clear to start fresh if a long conversation exceeds your model's context window. While Wendy works, Enter adds your message to a visible queue. Queued messages run in order after the current turn finishes. To change direction immediately, cancel the current turn, then send the new instruction.

--yes approves every tool call for the session, including shell commands, file writes, and disruptive device operations. Chat saves selected memory notes, but does not save conversation transcripts. With hosted inference, the conversation and tool results are sent to the configured provider; with a local endpoint, they go to that local server. Existing Wendy Cloud/device connections and CLI analytics settings continue to apply.

Settings and API keys pasted during setup are saved in ~/.wendy/chat.json, with file permissions restricted to your user. API keys supplied through environment variables are not copied into that file.

The chat screen requires interactive stdin and stdout. Scripts use --prompt instead, described next.

Run one turn from a script

--prompt runs a single turn without the chat screen and exits. It uses the setup you saved with wendy chat --setup, or the connection flags below, and never opens a setup screen. Use it from shell scripts, CI, or another program that wants Wendy's reasoning and tools without its terminal.

wendy chat --prompt "Which of my devices are online?"

When stdout is a terminal, the assistant's final text is printed. When stdout is a pipe or a file, or with --json, each event is printed as it happens, one JSON object per line, following the CLI's usual rule that non-interactive output is JSON. Notices go to stderr, so stdout is only the stream:

wendy chat --prompt "Why does app A on device X keep restarting?" --json

{"type":"status","text":"Thinking…"}
{"type":"text","text":"Let me look at the logs. "}
{"type":"tool_start","id":"call_1","tool":"telemetry_logs","arguments":{"app_name":"A","min_severity":"error"}}
{"type":"tool_result","id":"call_1","tool":"telemetry_logs","text":"..."}
{"type":"text","text":"The app cannot reach port 8080..."}
{"type":"done","text":"Let me look at the logs. The app cannot reach port 8080..."}

Event types are status, text (streamed chunks), tool_start, approval, tool_result, and a final done or error. The done event carries the complete assistant text. A non-zero exit status accompanies error.

Tools that would ask a person, such as shell commands, file writes, and device changes, are denied in this mode: there is nobody to answer. The stream reports each denial as an approval event with "approved":false, and the model is told the tool did not run. Pass --yes to allow them:

wendy chat --prompt "Redeploy the relay to Unitree G1 Nx" --json --yes

--prompt cannot be combined with --setup or --voice. Each run is a fresh conversation. Profile memory is recalled and learned in both interactive and headless runs; use --no-memory to disable it.

Remember working methods and mistakes

Memory is enabled by default. Wendy recalls relevant notes before a task, and can search for additional notes as it works. After using tools, it reviews the results for useful lessons if it has not already saved a note during the turn. That review uses the selected model, makes at most one additional model request, and can save up to three notes. The terminal shows saved notes and memory errors. A failed review leaves the task result intact.

For example, after a direct connection attempt fails and a cloud command works, Wendy can remember the connection prerequisite, the working command, and the result that confirmed it. Procedures and failure lessons must cite tools that actually ran in the current turn. A procedure requires successful tool evidence; failed attempts are recorded as lessons. Notes remain fallible: a successful command exit alone does not establish that a physical action happened.

Notes have one of three scopes:

• Workspace: project-specific build, deployment, and debugging knowledge.
• Device: knowledge about an explicitly named device, reusable across projects.
• Global: general Wendy workflows and user preferences.

Memory is stored locally in ~/.wendy/memory/, as private JSON files. Saving a note is automatic and does not approve any future shell command or device action. Relevant notes go to your selected model alongside task context. Wendy is instructed to save concise lessons without credentials, raw transcripts, or temporary readings. The storage is shared across model changes and CLI sessions.

Use /memory to browse up to 50 recent notes with titles, brief previews, and short IDs. Use /memory <id> to read the full note and its evidence, or /forget <id> to delete it. /memory search <words> finds notes about a task or named device. These commands run locally, including while Wendy is working or memory is paused. Ask Wendy to correct an outdated note or remember an explicit preference. Reusing a title and scope updates the note. Deleted notes stop appearing in automatic recall and refreshed memory searches; ordinary user and assistant text already in the conversation remains until /clear.

Use /memory off to stop new recall and learning for the session, or start with:

wendy chat --no-memory

/memory on enables it again. Disabling memory keeps existing notes on disk; /clear also keeps notes while resetting the conversation. Memory changes what context Wendy can reuse; it does not retrain the selected model.

Advanced configuration

Run wendy chat --help-all to see all options. You can supply settings directly instead of using guided setup:

wendy chat --provider ollama --model my-model
wendy chat --provider local --base-url http://localhost:1234/v1 --model my-model

Replace my-model with a model served by your endpoint. For hosted models, OPENAI_API_KEY and ANTHROPIC_API_KEY are supported if you already manage credentials through your environment. Choose an accessible model ID with --model or use guided setup to browse available models.

For an authenticated custom API, set WENDY_CHAT_API_KEY along with --base-url. Provider-specific cloud keys are not automatically sent to custom endpoints.

Flags take precedence over environment variables, which override saved settings. The openai provider also accepts OPENAI_BASE_URL. Available providers are openai, anthropic, ollama, and local. The ollama preset uses http://localhost:11434/v1; the local preset defaults to http://localhost:8080/v1.

Profiles and parallel agents

general is the default. Select a specialization when starting a conversation:

wendy chat --list-profiles
wendy chat --profile developer
wendy chat --profile simulation -C ./sim
wendy chat --profile debugger --device my-robot
wendy chat --profile fleet
wendy chat --profile device-reasoning
wendy chat --profile device-sensors
wendy chat --profile device-control

Profiles select instructions, tools, bundled skills, and durable memory. Every session automatically loads its role's bundled SKILL.md, including delegated children and service tasks. These files live in go/internal/cli/chat/skills/ and ship inside the CLI. They cover task routing, tool availability, role-specific workflows, and concise result reporting. No separate skill installation is needed.

Use profile_skill through chat to reread the startup skill at profiles/<profile>/SKILL.md or read additional bundled skills and references on demand. The general profile can inspect all role skills; specialists can inspect their own. wendy chat --list-profiles --json includes each startup skill path.

Choose the specialist by the tools the task needs. Camera inventory belongs to device-sensors, even when checking many devices. fleet has connectivity and fleet-management tools but no camera, audio, or ROS 2 tools. Children inherit the intersection of their own and their parent's tool restrictions, so delegating from fleet cannot restore sensor access. Skill instructions do not grant tools or change approvals.

general retains existing notes in ~/.wendy/memory; other profiles store notes under ~/.wendy/memory/profiles/<profile>. Workspace, device, and global scopes apply within each profile. Switching a model does not change its profile memory. A profile's tool selection is not a shell sandbox. An approved shell or container command retains its normal OS and network access.

Wendy can call agent_delegate with up to four independent tasks in one batch. They run concurrently and return individual results in submission order. Each child gets a fresh conversation, its own MCP connection and model client, its profile memory, and only the tools allowed by both profiles. Children receive the explicit task text, not the parent's transcript. Children cannot delegate again. The runtime permits at most twelve children per parent turn and limits each batch to ten minutes. Mutating tool calls serialize within a batch; assign disjoint file edits and let local controllers arbitrate continuous motion.

Approvals identify the child and its profile. The parent's approval policy applies, including headless denials and --yes. Cancellation waits for local children to stop and closes their MCP and media processes. Child progress carries agent_id and profile in JSON events. The parent's final text excludes child streaming text. /memory off also pauses active children's recall and saves.

Children inherit the parent's model connection by default. Supply a trusted JSON file to choose another connection for each child profile:

{
  "developer": {
    "provider": "openai",
    "model": "YOUR_CLOUD_MODEL",
    "api_key_env": "OPENAI_API_KEY"
  },
  "debugger": {
    "provider": "local",
    "model": "YOUR_LOCAL_MODEL",
    "base_url": "http://localhost:8080/v1"
  }
}

wendy chat --agent-models ./agent-models.json

These overrides apply to delegated children. Use --provider and --model or --setup for the main conversation. Model connections do not inherit another endpoint's key. Model processes can be shared by multiple agent conversations; creating a child does not load another model into GPU memory.

For persistent services, sensor event triggers, and remote A2A peers, read the agent service guide.