OpenMausBot: Building a Local-First Multi-Agent Orchestrator with Permission Gates
Hook
Most AI agent frameworks choose between autonomous execution and human oversight. OpenMausBot's permission broker makes every shell command, file write, and API call wait for your approval—not as logging, but as a stateful runtime component that works identically whether the agent runs on your laptop, a Docker VM, or a cloud desktop.
Context
Grok Bot popularized the idea of conversational multi-agent teams with dedicated virtual machines per bot. But it locked you into xAI's cloud, made your transcripts and API keys live on their servers, and charged a subscription for orchestration that could run locally. The explosion of agentic AI tools—AutoGPT, babyagi, Claude Desktop—created a new problem: every vendor has its own protocol. Anthropic uses Model Context Protocol loops with server-declared tools. OpenAI does function calling with JSON schemas. xAI streams responses in yet another format. If you want to run multiple agents from different providers in a single workflow, you're building translation layers by hand.
OpenMausBot started as a self-hosted Grok alternative but evolved into something more architectural: a local-first orchestration layer that normalizes vendor protocols, enforces permission boundaries, and treats agents as portable packages. It's TypeScript throughout—Electron + React for the UI, Node.js for the harness server—and stores everything in ~/.openmausbot. No cloud telemetry. No proxied API requests. The harness spawns official CLI tools (claude, codex, grok) authenticated with your own credentials, transforms their outputs into a canonical NDJSON event stream, and surfaces every dangerous action as an approval card before execution. The result is a zero-trust multi-agent runtime where you pay model providers directly, keep your data local, and maintain human control over every consequential decision.
Technical Insight
The core innovation is the protocol normalization layer in the harness server. When you send a message to a bot, the harness spawns the appropriate provider CLI as a child process and pipes the conversation history as context. Anthropic's claude CLI returns tool requests wrapped in <tool_use> blocks; OpenAI's codex returns function_call objects; xAI's grok streams deltas with tool_invocations. The harness consumes all three formats and emits a unified event stream:
interface AgentEvent {
thread_id: string;
timestamp: number;
type: 'message' | 'tool_request' | 'tool_result' | 'approval_required';
provider: 'anthropic' | 'openai' | 'xai';
payload: {
content?: string;
tool?: string;
args?: Record<string, unknown>;
approval_id?: string;
};
}
This abstraction means the Electron frontend never knows or cares which model powers a bot. It subscribes to an SSE stream at http://127.0.0.1:8799/threads/{id}/events and renders cards for messages, tool requests, and approval prompts identically. Adding a new provider is mechanical: write a thin adapter that spawns the CLI, parses its output format, and emits AgentEvent objects. No changes to the UI, no changes to the permission broker, no changes to thread persistence.
The permission broker intercepts every AgentEvent with type: 'tool_request' where the tool is classified as risky—shell execution (bash_exec), file writes (fs_write), computer control (cua_click, cua_type), or OAuth-gated API calls (composio_*). Instead of immediately forwarding the request to the execution backend, the broker generates an approval_id, persists the pending action to ~/.openmausbot/approvals/{thread_id}.json, and emits an approval_required event. The frontend renders a card showing the exact command, target environment (host / local VM / cloud VM), and Allow / Deny buttons. Only after the user clicks Allow does the broker forward the original tool request to the execution layer. Here's the approval persistence schema:
{
"approval_id": "apr_1a2b3c4d",
"thread_id": "thr_xyz",
"timestamp": 1704931200000,
"tool": "bash_exec",
"args": {
"command": "rm -rf dist && npm run build",
"cwd": "/home/user/project"
},
"target": "host",
"status": "pending",
"decision": null,
"decided_at": null
}
This stateful approach means you can review the approval log later, audit what a bot attempted versus what you allowed, and even replay threads with different approval decisions. It's fundamentally different from frameworks that log actions after execution or require you to review code before running it. The broker enforces the boundary in the runtime, not as a pre-flight check.
The multi-target execution model is equally clever. When you create a bot, you assign it a 'computer'—either host (your actual machine), local-vm (a Docker container managed by the harness), or cloud-vm (a Boat.dev instance authenticated via API key). The harness maintains a session pool for each target type. For host execution, the Cua Driver translates cua_click(x, y) into macOS Accessibility API calls or X11 xdotool commands. For local-vm, the harness spins up a Debian container with VNC + noVNC, forwards tool requests over stdio to a guest agent, and streams the desktop back as MJPEG. For cloud-vm, it proxies tool requests to Boat.dev's HTTP API and embeds their web-based desktop viewer in the UI. The permission broker is target-agnostic—the same approval card appears whether the bot wants to run curl on your laptop or in a Boat.dev Ubuntu instance.
The portable team packages—BotMRR files—solve the agent composition problem beautifully. A .md file encodes an entire multi-agent setup as YAML frontmatter:
---
bots:
- name: Backend Engineer
provider: anthropic
model: claude-3-5-sonnet-20241022
instructions: |
You write Go services. Always write tests. Never commit without linting.
voice: elevenlabs:adam
- name: Frontend Engineer
provider: openai
model: gpt-4
instructions: |
You write React + TypeScript. Prefer Tailwind. Ask Backend Engineer for API contracts.
voice: fish:sara
channels:
- name: Feature Development
bots: [Backend Engineer, Frontend Engineer]
decision_model: anthropic:claude-3-haiku
auto_respond: true
routines:
- name: PR Review
trigger: github_webhook
channel: Feature Development
message: "New PR opened: {{pr_url}}. Backend Engineer, review the API changes. Frontend Engineer, check the UI."
---
# Feature Team Playbook
This team builds full-stack features. Backend Engineer owns services, Frontend Engineer owns UI, and they coordinate in the Feature Development channel.
You can share this file, commit it to Git, or let Claude read it to understand your team structure. Importing it in OpenMausBot creates the bots, sets up the channel, registers the routine, and configures the decision model—but doesn't carry over chat history, API keys, or model choices (those stay local per the security model). It's a distribution format for agent architecture, not agent state.
Gotcha
The CLI dependency model is fragile. OpenMausBot doesn't vendor or version-pin the claude, codex, or grok executables—it assumes they're in your $PATH and authenticated. When Anthropic updated the Claude CLI to change how it formats tool results (from XML attributes to JSON properties) in late 2024, the normalization layer broke for two weeks until a harness patch landed. There's no graceful degradation: if a CLI is missing or returns unexpected output, the thread hangs with a cryptic 'provider error' in the UI. You're also locked into whatever models the official CLIs expose. Want to use a self-hosted Llama model via Ollama? You'd need to write a custom CLI wrapper that mimics Anthropic's output format, which defeats the purpose of local-first architecture.
The approval fatigue problem is real for long-running tasks. If you ask a bot to 'refactor this codebase to use async/await,' it might request approval for 50+ file writes. There's no batch approval ('allow all writes to src/'), no learned trust policies ('auto-approve read-only git commands'), and no time-boxed delegation ('allow anything for the next 10 minutes'). Every action surfaces a card, and the thread blocks until you click. For exploratory workflows where you want the agent to iterate freely within safe boundaries, this becomes a usability wall. The single-user, single-machine limitation also bites: if you use OpenMausBot on a desktop and a laptop, you're running two completely independent instances with no transcript sync, no shared bot roster, and no way to continue a conversation across devices short of manually copying ~/.openmausbot.
Verdict
Use OpenMausBot if you already subscribe to Claude, ChatGPT, or Grok and want to orchestrate them locally without locking your data into a vendor's cloud. It's ideal for workflows where human oversight is a feature, not a bug—code reviews, infrastructure changes, customer data access—and where you need multiple specialized agents working together with clear permission boundaries. The protocol normalization and portable team packages are genuinely novel, and the local-first architecture means you're never beholden to a SaaS provider's rate limits or privacy policies. Skip it if you need headless automation where approval latency kills productivity, if you want multi-user collaboration (it's strictly single-desktop with no sync), if you run Wayland on Linux (host control is Xorg-only), or if you expect the polish of a hosted service. This is a power tool for developers who want full control and are comfortable debugging CLI integration issues. If 'open-source Grok Bot alternative' made you curious, stay for the permission broker architecture—it's the most thoughtful implementation of human-in-the-loop I've seen in an agent framework.