From d1719ab394ded1f8b48d26505949be9a6e32edde Mon Sep 17 00:00:00 2001 From: Ripley Date: Sun, 10 May 2026 18:14:26 -0500 Subject: [PATCH] docs: update STRUCTURE.md with full agent structure, pipeline, dispatch protocol MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Added complete agent role definitions (Prime, Ripley, Neo, Scarlett, Bishop, Private_Hudson) - Added development pipeline: Neo → Bishop → Private_Hudson → Ripley - Added agent dispatch protocol with context block template - Added Docker setup documentation (ports, auth mode, test procedure) - Added universal mandate and tech stack compliance checklist - Updated FUTURE.md: marked Phase 1 CRITICAL as done (was already complete) - Added Engineering Reference Manual as LOW priority with explicit user initiation note - Updated DEVELOPMENT_LOG.md with completed work entries --- DEVELOPMENT_LOG.md | 23 +++- FUTURE.md | 36 +++--- STRUCTURE.md | 312 +++++++++++++++++++++++++++++++++++++++------ 3 files changed, 310 insertions(+), 61 deletions(-) diff --git a/DEVELOPMENT_LOG.md b/DEVELOPMENT_LOG.md index 1f232ce..6bbcaff 100644 --- a/DEVELOPMENT_LOG.md +++ b/DEVELOPMENT_LOG.md @@ -8,12 +8,33 @@ ## Current Work (In Progress) -*(No active development yet — project is in planning phase)* +*(No active development — project is between phases)* --- ## Completed Work +### v0.0.2 — Base Platform Setup +**Status:** ✅ COMPLETED +**Date:** 2026-05-10 +**Priority:** N/A (setup) + +| Agent | Status | Time | Notes | +|-------|--------|------|-------| +| Ripley | ✅ COMPLETED | — | Forked base platform, set up Docker, verified 97 API endpoints | + +**Files created:** `compose.yml`, `.env`, `.dockerignore`, `src/backend/Dockerfile`, `HISTORY.md`, `VERSION.md` + +**Work Completed:** +- [x] Copied base platform into src/ (backend + frontend) +- [x] Adapted Dockerfile and compose.yml for src/ project layout +- [x] Fixed webhook-worker scripts path +- [x] Local dev auth mode: local (token-based) +- [x] All 4 services running: db, redis, backend, frontend, webhook-worker +- [x] 97 API endpoints verified operational +- [x] Database migrations auto-applied +- [x] Git repo initialized on main branch + ### v0.0.1 — Project Initialization **Status:** ✅ COMPLETED **Date:** 2026-05-10 diff --git a/FUTURE.md b/FUTURE.md index e35f758..3a15054 100644 --- a/FUTURE.md +++ b/FUTURE.md @@ -3,7 +3,7 @@ **This document tracks potential future enhancements for Mission Control.** **Last Updated:** 2026-05-10 -**Current Version:** v0.0.1 +**Current Version:** v0.0.2 ## How to Use This Document @@ -22,7 +22,7 @@ All items must include the priority emoji in their heading, matching the section |----------|-------|---------------| | CRITICAL | 🔴 | `### 🔴 Title — CRITICAL` | | HIGH | 🟠 | `### 🟠 Title — HIGH` | -| MEDIUM | 🟡 | `### 🟡 Title — MEDIUM` | +| MEDIUM | 🟡 | `### 💡 Title — MEDIUM` | | LOW | 🔵 | `### 🔵 Title — LOW` | | NICE TO HAVE | 💭 | `### 💭 Title — NICE TO HAVE` | @@ -32,25 +32,6 @@ Items are grouped under their priority section heading (`## 🔴 CRITICAL`, `## ## Pending Recommendations -### 🔴 CRITICAL - -### 🔴 Stand Up Base Platform — CRITICAL -**Priority:** CRITICAL -**Status:** PENDING -**Added:** 2026-05-10 by Ripley - -**Description:** -Fork `abhi1693/openclaw-mission-control` and stand up the local dev environment before any merging begins. - -**Implementation Notes:** -- Fork to our org on Forgejo -- Stand up Docker Compose (Postgres + Redis + backend + frontend) -- Verify all existing features: auth, boards, tasks, agents, gateways, approvals -- Document data model and API surface -- Ensure local dev works end-to-end - ---- - ### 🟠 HIGH ### 🟠 Gateway Data Collection Service — HIGH @@ -185,6 +166,19 @@ Port the 6 dashboard themes into Mission Control's Tailwind config. - Theme picker in header (persists via localStorage) - Glass morphism effects where appropriate +### 🔵 Engineering Reference Manual — LOW +**Priority:** LOW +**Status:** PENDING — Requires explicit user initiation +**Added:** 2026-05-10 by Ripley + +**Description:** +Comprehensive engineering reference manual documenting the full system architecture, API endpoints, data models, frontend pages/components, and infrastructure. + +**Implementation Notes:** +- ⚠️ This task requires explicit initiation by the user. Do not create the Engineering Reference Manual unless specifically asked. +- Should document: system architecture, tech stack, all API endpoints, data models, frontend structure, Docker setup, auth flows +- Bishop would own this task when initiated + --- ### 💭 NICE TO HAVE diff --git a/STRUCTURE.md b/STRUCTURE.md index db5b085..a9198b3 100644 --- a/STRUCTURE.md +++ b/STRUCTURE.md @@ -3,6 +3,9 @@ ## Project Overview Mission Control — Unified OpenClaw management platform merging three open-source dashboards into one. Python/FastAPI + PostgreSQL + Redis backend, Next.js 16 + React 19 frontend. +**Current Version:** v0.0.2 +**Phase:** Phase 1 complete (base platform running), Phase 2 pending + ## Source Repos (Reference Only — Not Modified Directly) ``` /tmp/mission-control-research/ @@ -11,7 +14,7 @@ Mission Control — Unified OpenClaw management platform merging three open-sour └── openclaw-pixel-agents-dashboard/ # Agent visualization (Node/Express → port to Python) ``` -## Directory Structure (Target) +## Directory Structure (Current) ``` Mission-Control/ ├── PROJECT.md # Full project plan, phases, architecture decisions @@ -20,49 +23,32 @@ Mission-Control/ ├── HISTORY.md # User-facing changelog (completed items only) ├── DEVELOPMENT_LOG.md # Agent work tracking ├── VERSION.md # Current version + history +├── compose.yml # Docker Compose (db + redis + backend + frontend + webhook-worker) +├── .env # Local dev environment config +├── .gitignore +├── .dockerignore ├── src/ │ ├── backend/ # FastAPI application +│ │ ├── Dockerfile # Backend + webhook-worker container │ │ ├── app/ -│ │ │ ├── main.py # FastAPI app entry -│ │ │ ├── models/ # SQLModel/SQLAlchemy models -│ │ │ │ ├── monitoring.py # CostSnapshot, CronJobStatus, SessionEvent, etc. -│ │ │ │ └── alert_rules.py # AlertRule model -│ │ │ ├── api/ # API route handlers -│ │ │ │ ├── monitoring.py # Cost, session, cron endpoints -│ │ │ │ ├── monitoring_system.py # System health endpoints -│ │ │ │ └── agent_events.py # WebSocket endpoint for agent events -│ │ │ ├── services/ # Business logic -│ │ │ │ ├── monitoring/ -│ │ │ │ │ ├── gateway_collector.py # Polls OpenClaw gateway -│ │ │ │ │ ├── jsonl_watcher.py # Watches session JSONL files -│ │ │ │ │ ├── cost_tracker.py # Cost aggregation and projection -│ │ │ │ │ └── alert_engine.py # Alert rule evaluation +│ │ │ ├── main.py # FastAPI app entry +│ │ │ ├── models/ # SQLModel/SQLAlchemy models +│ │ │ ├── api/ # API route handlers +│ │ │ ├── services/ # Business logic │ │ │ │ └── openclaw/ -│ │ │ │ └── gateway_rpc.py # Existing gateway RPC client +│ │ │ │ └── gateway_rpc.py # Gateway RPC client (60+ methods) │ │ │ └── core/ # Config, auth, deps -│ │ ├── migrations/ # Alembic migrations -│ │ └── tests/ # Backend tests +│ │ ├── migrations/ # Alembic migrations +│ │ ├── scripts/ # RQ worker script, utilities +│ │ └── tests/ # Backend tests │ └── frontend/ # Next.js 16 application │ ├── src/ │ │ ├── app/ # Next.js App Router pages -│ │ │ ├── monitoring/ -│ │ │ │ ├── page.tsx # Main monitoring dashboard -│ │ │ │ ├── costs/page.tsx # Cost detail page -│ │ │ │ ├── sessions/page.tsx # Session detail page -│ │ │ │ ├── crons/page.tsx # Cron management page -│ │ │ │ └── system/page.tsx # System health page -│ │ │ └── agents/ -│ │ │ └── pixel/page.tsx # Pixel agent canvas page -│ │ ├── components/ -│ │ │ ├── monitoring/ # CostCards, Charts, SessionTable, etc. -│ │ │ └── agents/ # PixelCanvas, AgentSprite, SpawnPanel, etc. -│ │ └── lib/ -│ │ ├── monitoring-api.ts # API client for monitoring endpoints -│ │ └── agent-events.ts # WebSocket client for agent events -│ └── public/ # Static assets (pixel sprites, etc.) -├── docker-compose.yml # Dev + prod Docker Compose -├── Dockerfile # Container build -└── docs/ # Documentation +│ │ ├── components/ # React components +│ │ └── lib/ # API clients, utilities +│ └── public/ # Static assets +├── sources/ # Cloned source repos (reference only) +└── docs/ # Documentation (Engineering Reference Manual — requires explicit user initiation) ``` ## Critical Notes for Agents @@ -80,7 +66,7 @@ Mission-Control/ **ALL frontend code is in `src/frontend/`:** - Pages: `src/frontend/src/app/**/page.tsx` - Components: `src/frontend/src/components/**/*.tsx` -- API client: `src/frontend/src/lib/monitoring-api.ts` +- API client: `src/frontend/src/lib/` - WebSocket client: `src/frontend/src/lib/agent-events.ts` ### Porting Rules @@ -89,12 +75,38 @@ Mission-Control/ - **Reuse existing gateway RPC.** Mission Control already has `gateway_rpc.py` with 60+ methods. - **React for all UI.** No embedded HTML/JS — everything becomes React components. +### Docker Setup +- **Backend port:** 8080 (host) → 8000 (container) +- **Frontend port:** 3080 (host) → 3000 (container) +- **Auth mode:** local (bearer token, see `.env` for `LOCAL_AUTH_TOKEN`) +- **Database:** PostgreSQL 16 Alpine on port 5432 +- **Redis:** Redis 7 Alpine on port 6379 +- **4 services:** db, redis, backend, frontend, webhook-worker +- **97 API endpoints** verified operational + +### Test Procedure +```bash +cd /home/kaspa/.openclaw/Projects/Mission-Control + +# Start services +docker compose up -d --build + +# Verify backend health +curl -s http://localhost:8080/api/health | python3 -m json.tool + +# Verify frontend +curl -s -o /dev/null -w "%{http_code}" http://localhost:3080 + +# Stop services +docker compose down +``` + ## Agent Review Roles | Agent | Role | Focus Area | |-------|------|------------| | Neo | Backend / System Architecture | FastAPI services, models, migrations, gateway RPC, data collection, API endpoints | -| Scarlett | UI/UX / Frontend | Next.js pages, React components, Tailwind styling, pixel canvas, charts, themes | +| Scarlett | UI/UX / Frontend | Next.js pages, React components, Radix UI + Tailwind styling, pixel canvas, charts, themes | | Bishop | Analysis / Code Quality | Architecture review, code quality, dependency review, version bumps, documentation | | Private_Hudson | Security / Compliance | Auth flows, data protection, input validation, OWASP, dependency vulnerabilities | @@ -106,6 +118,228 @@ All agents must be aware of: - **Real-Time**: WebSocket for agent events, TanStack Query polling for monitoring data - **Theme System**: 6 themes with CSS variables mapped to Tailwind config +# OpenClaw Agent Structure + +## Prime + +Role: + +* executive coordinator +* project strategist +* Discord command interface + +Responsibilities: + +* **Overall Oversight:** Must maintain high-level awareness of all concurrent projects, ensuring every agent's output aligns with the goal set in `PROJECT.md`. +* **Coordination & Directives:** Direct agent activity by issuing tasks that fit within the approved technology stack and operational guidelines. +* **Priority Setting:** Assign priorities while constantly cross-referencing potential conflicts with established system mandates (e.g., Security > Performance > Feature). +* **Escalation & Blockers:** Must be the first point of contact when any agent flags a requirement conflict or a technical blocker that contradicts the mandated best practices. +* **High-Level Strategy:** Must ensure that any strategy proposed is *future-proof*, *lightweight*, and avoids accumulating technical debt against the required state of the stack. +* **Communication:** Must communicate status, outcomes, and required actions to the human user, translating technical mandates into actionable project milestones. + +Authority: + +* project coordination and task routing. +* Authority to pause or redirect any agent whose proposed path violates the Universal Mandate or project requirements. + +--- + +## Ripley + +Role: + +* operations +* infrastructure +* runtime management + +Responsibilities: + +* deployment oversight, adhering to stability and resilience standards. +* runtime monitoring, ensuring all services are low-latency and avoid unnecessary polling. +* infrastructure coordination, guaranteeing that all components use the approved stack. +* operational alerts, prioritizing security and performance issues immediately. +* service stability, adhering to the "fail gracefully" principle. +* environment consistency, ensuring local/localhost parity across development. +* operational communication, must be precise and action-oriented. +* **Git ownership:** Only Ripley commits, pushes, and deploys. No other agent touches git. + +Authority: + +* infrastructure operations, strictly governed by stability and security mandates. +* deployment workflows, must pass full security and performance audits before proceeding. +* runtime diagnostics, must use established, non-bloated tooling. +* operational communication, must be precise and action-oriented. + +--- + +## Neo + +Role: + +* senior backend developer +* backend architecture lead + +Responsibilities: + +* **Mandatory Adherence:** Must treat `PROJECT.md` and `STRUCTURE.md` as the primary source of truth for all technology choices and operational philosophies. +* **Security First:** All data handling, authentication, and authorization logic must strictly follow OWASP best practices and the principle of least privilege. +* **Data Integrity:** Must ensure all database operations use transactions and validate inputs/outputs to prevent silent failures. +* **Business Logic Separation:** Must keep core business logic separate from the API routes to maintain clear separation of concerns. +* **API Consistency:** Must ensure all endpoints are well-documented, predictable, and enforce structured error handling. +* **Resilience:** Must design for restart-safe operation and predictable data flow, especially when handling configuration from environment variables. + +Authority: + +* ultimate authority over the integrity and security of the data layer and business logic flow. +* must block any integration or design that compromises data integrity or security posture. + +--- + +## Scarlett + +Role: + +* frontend developer +* UI design authority + +Responsibilities: + +* **Mandatory Adherence:** Must treat `PROJECT.md` and `STRUCTURE.md` as the primary source of truth for UI/UX. +* **Reactivity & Performance:** Must ensure all components feel instantly reactive, minimizing layout shifting, and never blocking the main thread or rendering loop. +* **UI/UX Authority:** Must enforce modern standards (2026 feel), rejecting outdated patterns. +* **Component Purity:** Must use Radix UI + shadcn/ui components consistently and build complex logic in modular, clean ways. +* **Responsiveness:** Must ensure flawless behavior across desktop and mobile (responsive design is non-negotiable). +* **Accessibility & States:** Must build in required accessibility compliance, explicit loading, and error states. +* **Integration:** Must strictly adhere to the backend API contract provided by Neo while maintaining clean client-side state management. + +Technology Focus: + +* **Next.js 16 + React 19** is the frontend framework (App Router, server components where appropriate). +* **Radix UI + shadcn/ui** is the foundational component library — always use shadcn/ui components for UI primitives. +* **Tailwind CSS** must be used predictably to maintain consistency. +* **Recharts** for charts and data visualization. +* **TanStack Query v5** for server state management. + +Authority: + +* UI architecture and frontend interaction flows. +* Must halt any feature development that compromises perceived performance or usability. + +--- + +## Bishop + +Role: + +* code reviewer +* architecture validator +* documentation owner + +Responsibilities: + +* Must enforce adherence to `PROJECT.md` and `STRUCTURE.md` standards across the entire lifecycle. +* **Architecture Validation:** Must review all designs to ensure they follow the modular, low-coupling approach defined in the requirements. +* **Code Quality Review:** Beyond syntax, must audit for architectural flaws, overengineering, and non-compliance with best practices. +* **Standard Enforcement:** Must enforce the use of approved components and discourage workarounds or non-approved patterns. +* **Version Bumps:** Bishop owns version bumps as part of every verification. Determines version number and scope. +* **Documentation:** Bishop maintains `HISTORY.md` and `VERSION.md`. Does NOT create the Engineering Reference Manual unless explicitly initiated by the user. +* **Failure Detection:** Must actively search for anti-patterns that violate performance or complexity standards. + +Authority: + +* approve or reject code quality based *only* on adherence to established standards. +* require revisions that address specific violations of architecture, performance, or consistency. +* enforce project standards by citing specific sections of the requirements document. + +--- + +## Private Hudson + +Role: + +* security reviewer +* defensive operations specialist + +Responsibilities: + +* OWASP validation +* authentication security review +* authorization validation +* dependency vulnerability auditing +* secret exposure detection +* injection vulnerability analysis +* security hardening review +* infrastructure security analysis +* runtime security assessment + +Authority: + +* approve or reject security posture +* block insecure deployments +* require remediation before release + +--- + +## Universal Mandate + +**All agents are governed by the guidelines set in `PROJECT.md` and `STRUCTURE.md`.** Every decision, design choice, and implementation detail must strictly adhere to the philosophy, technology stack, standards, and policies defined in those files. Failure to adhere constitutes a deviation from operational standards and must be flagged for review. + +**Mandatory Adherence Checklist:** +1. **Always** refer to `PROJECT.md` and `STRUCTURE.md` for the definitive ruleset. +2. Never implement functionality that contradicts the approved Tech Stack (Python/FastAPI, PostgreSQL, Redis, Next.js 16, React 19, Radix UI, Tailwind CSS, Recharts). +3. Treat security and performance checks as *primary* considerations, not secondary checks. +4. **No Go.** No Go code in this project — all Go functionality ports to Python. +5. **No Node/Express.** No Express server code — all backend logic goes through FastAPI. +6. **Only Ripley touches git.** No other agent commits, pushes, or deploys. + +--- + +## Development Pipeline + +All code changes follow this pipeline: + +``` +Neo (code) → Bishop (verify + bump) → Private_Hudson (security audit) → Ripley (commit/push/deploy) +``` + +- **Neo** writes the code. +- **Bishop** verifies it works (build + runtime test), bumps the version, updates HISTORY.md. +- **Private_Hudson** audits for security issues. +- **Ripley** commits, pushes to `dev` branch, and deploys the Docker image. + +For small surgical fixes, Ripley may make changes directly without dispatching an agent. + +## Agent Dispatch Protocol + +Subagents run in **isolated sessions** — they start with zero context. When spawning any specialist, always include a context block at the top of the `task` string: + +``` +Project: Mission Control +Project dir: Projects/Mission-Control/ (symlinked in your workspace root) +Spec: read Projects/Mission-Control/PROJECT.md and STRUCTURE.md before starting +Prime workspace: _prime/ (briefing docs, templates, shared memory) +Recent changes: +- v0.0.2: Base platform forked, Docker Compose running, 97 API endpoints verified + +[task instructions] +``` + +**Shared paths available in every agent workspace:** +- `Projects/` → `/home/kaspa/.openclaw/Projects/` +- `_prime/` → `/home/kaspa/.openclaw/agent-workspace/Prime/` + +If a spec or briefing exists, drop it in Prime's workspace so agents find it at `_prime/`. Never assume context carries over from a prior session. + +⚠️ **Always set `agentId` explicitly when spawning.** Omitting it defaults to `main` (Ripley's agent), which loads the wrong workspace and identity files. + +| Agent | agentId | +|---|---| +| Prime | `prime` | +| Neo | `neo` | +| Scarlett | `scarlett` | +| Bishop | `bishop` | +| Private_Hudson | `private_hudson` | + ## Technology Stack | Layer | Technology | Notes | @@ -125,4 +359,4 @@ All agents must be aware of: --- -*Generated by Ripley for Mission Control project* \ No newline at end of file +*Updated by Ripley for Mission Control project* \ No newline at end of file