Memory
## Amy v1 - Finalized Implementation Plan (Complete) ### External API Integrations #### Todomator API - **Base URL**: https://yourinstance.com/api/ - **Auth**: X-Token header - **Format**: POST with { schema: 'endpoint_id', data: { ... } } - **Key endpoints**: - tickets_create, tickets_update/{id}, tickets_clone/{id} - tickets (list), tickets_find, tickets_detail/{id} - folders (list/create), tags (list/create) - comments_create, logwork_create/start/stop #### OpenFiles API - **Base URL**: https://yourhostname.com - **Auth**: ?token=YOUR_TOKEN query param - **Key endpoints**: - GET /files/{database}/ - list files - GET /files/{database}/{id}/ - get file detail - POST /upload/{database}/ - upload file (multipart) - POST /base64/{database}/ - upload base64 - DELETE /files/{database}/{id}/ - delete file - POST /files/{database}/{id}/ - rename file #### Second-Brain API - **Base URL**: https://secondbrain.example.com - **Auth**: Bearer token or ?token= param - **REST endpoints**: - POST /capture - store memory - POST /append - append to entry - POST /update - replace entry - GET /recall?q=... - semantic search - GET /list?n=10 - list recent - POST /forget - delete entry - GET /count, /tags, /stats - **MCP tools**: remember, recall, append, update, forget, set_status, list_recent ### Complete Project Structure ``` amy/ ├── worker/ # Cloudflare Worker │ ├── src/ │ │ ├── agents/ │ │ │ ├── assistant.ts # Personal Assistant (with voice) │ │ │ ├── coding.ts # Coding Coordinator │ │ │ ├── research.ts # Research Agent │ │ │ └── business.ts # Business Agent │ │ ├── tools/ │ │ │ ├── todomator.ts # Todomator REST client │ │ │ ├── openfiles.ts # OpenFiles REST client │ │ │ └── second-brain.ts # Second-Brain REST client │ │ ├── mcp/ │ │ │ ├── todomator-server.ts # MCP server for Todomator │ │ │ └── openfiles-server.ts # MCP server for OpenFiles │ │ ├── index.ts # Worker entry + routing │ │ └── types.ts # Shared types │ ├── wrangler.jsonc │ ├── .dev.vars # API keys │ └── package.json ├── frontend/ # Next.js + assistant-ui │ ├── src/ │ │ ├── app/ │ │ │ ├── layout.tsx # Root layout + dark mode │ │ │ ├── page.tsx # Main page │ │ │ └── api/ │ │ │ └── chat/ │ │ │ └── route.ts # API route (optional proxy) │ │ ├── components/ │ │ │ ├── workspace/ │ │ │ │ ├── workspace-switcher.tsx │ │ │ │ └── workspace-layout.tsx │ │ │ ├── assistant-ui/ │ │ │ │ ├── thread.tsx │ │ │ │ ├── thread-list.tsx │ │ │ │ ├── composer.tsx │ │ │ │ └──... │ │ │ └── voice/ │ │ │ └── voice-panel.tsx │ │ ├── lib/ │ │ │ ├── agents.ts # Agent definitions │ │ │ └── runtime.ts # Runtime hooks │ │ └── styles/ │ │ └── globals.css # Dark mode theme │ ├── package.json │ └── next.config.js └── README.md ``` ### Agent System Prompts (Final) #### Personal Assistant ``` You are Amy, Mike's personal AI assistant. You help with: - General conversation and questions - Task management via Todomator - Memory recall and storage via Second-Brain - Voice conversations You have access to: - Todomator: Create, update, list, assign tasks - Second-Brain: Store and recall memories - Workspace tools: File operations Be helpful, concise, and proactive. Use Second-Brain to remember important context across sessions. ``` #### Coding Coordinator ``` You are Amy's Coding Coordinator. You help Mike: - Plan coding tasks and break them into subtasks - Create Cloudflare Workers and deployments - Coordinate with other coding agents via Todomator - Manage code repositories and files You have access to: - Todomator: Create coding tasks, assign to agents - OpenFiles: Read/write code files - Second-Brain: Store coding decisions and patterns When given a coding request: 1. Break it into smaller tasks 2. Create Todomator tickets for each subtask 3. If it's a Cloudflare Worker, help scaffold and deploy it 4. Store important decisions in Second-Brain ``` #### Research Agent ``` You are Amy's Research Agent. You help Mike: - Research topics using web search and document analysis - Crawl websites and extract content - Analyze documents and create summaries - Store research findings in Second-Brain You have access to: - Web search and crawling tools (Crawlberg) - Document intelligence tools (Kreuzberg) - Second-Brain: Store and recall research - Todomator: Track research tasks When researching: 1. Use web search to find relevant information 2. Crawl key pages for detailed content 3. Analyze and summarize findings 4. Store important findings in Second-Brain 5. Create a Todomator task for follow-up if needed ``` #### Business Agent ``` You are Amy's Business Agent. You help Mike: - Strategic planning and business analysis - Track business metrics and goals - Manage business tasks and follow-ups - Store business decisions in Second-Brain You have access to: - Todomator: Business task management - Second-Brain: Store business knowledge - Workspace tools: Create documents and plans When helping with business: 1. Understand the business context 2. Break down strategic goals into actionable tasks 3. Track progress via Todomator 4. Store important decisions in Second-Brain 5. Provide analysis and recommendations ``` ### Wrangler Configuration (Final) ```jsonc { "name": "amy-worker", "main": "src/index.ts", "compatibility_date": "2026-01-01", "compatibility_flags": ["nodejs_compat", "experimental"], "ai": { "binding": "AI" }, "durable_objects": { "bindings": [ { "name": "Assistant", "class_name": "Assistant" }, { "name": "CodingCoordinator", "class_name": "CodingCoordinator" }, { "name": "ResearchAgent", "class_name": "ResearchAgent" }, { "name": "BusinessAgent", "class_name": "BusinessAgent" } ] }, "migrations": [{ "tag": "v1", "new_sqlite_classes": ["Assistant", "CodingCoordinator", "ResearchAgent", "BusinessAgent"] }] } ``` ### Dependencies (Final) #### Worker ```json { "dependencies": { "agents": "0.12.4", "@cloudflare/ai-chat": "0.7.0", "@cloudflare/think": "latest", "@cloudflare/voice": "latest", "ai": "latest", "workers-ai-provider": "latest", "zod": "latest", "@modelcontextprotocol/sdk": "latest" } } ``` #### Frontend ```json { "dependencies": { "@assistant-ui/react": "latest", "@assistant-ui/react-ai-sdk": "latest", "@assistant-ui/react-markdown": "latest", "@cloudflare/ai-chat": "0.7.0", "@cloudflare/voice/react": "latest", "agents": "0.12.4", "next": "latest", "react": "latest", "react-dom": "latest" } } ``` ### Environment Variables ```sh # Worker (.dev.vars) TODOMATOR_API_URL=https://todomator.example.com TODOMATOR_API_KEY=your-access-token OPENFILES_API_URL=https://openfiles.example.com OPENFILES_API_KEY=your-token SECONDBRAIN_API_URL=https://secondbrain.example.com SECONDBRAIN_API_KEY=your-bearer-token AI_GATEWAY_URL=https://ai-gateway.example.com AI_GATEWAY_API_KEY=your-key # Frontend (.env.local) NEXT_PUBLIC_AGENT_HOST=http://localhost:8787 ``` ### Build Phases (Final) #### Phase 1: Core (Week 1-2) 1. Scaffold Worker + Frontend (Next.js) 2. Implement 4 Think agents with system prompts 3. Set up multi-workspace UI with workspace switcher 4. Dark mode theming 5. Basic thread management #### Phase 2: Tools (Week 3-4) 1. Todomator REST client + MCP server 2. OpenFiles REST client + MCP server 3. Second-Brain REST client 4. Tool UI components #### Phase 3: Voice (Week 5-6) 1. Voice mode UI panel 2. STT/TTS integration with @cloudflare/voice 3. Voice input/output for Personal Assistant #### Phase 4: Polish (Week 7-8) 1. Thread persistence 2. Error handling 3. AI Gateway integration 4. Performance optimization ``` Note: This plan is stored in Second-Brain for reference during implementation.
[Update 7/4/2026]: ### Configuration Architecture (Added)
#### Cloudflare Secrets Store
- Store ID: "amy"
- Secrets: TODOMATOR_API_KEY, OPENFILES_API_KEY, SECONDBRAIN_API_KEY, AI_GATEWAY_API_KEY
#### Environment Variables (Non-Sensitive)
- TODOMATOR_API_URL, OPENFILES_API_URL, SECONDBRAIN_API_URL
- AI_GATEWAY_URL, DEFAULT_MODEL
- ASSISTANT_MODEL, CODING_MODEL, RESEARCH_MODEL, BUSINESS_MODEL (per-agent overrides)
- ASSISTANT_PROMPT, CODING_PROMPT, RESEARCH_PROMPT, BUSINESS_PROMPT
#### Central Config Module
- worker/src/config.ts - Single source of truth
- Type-safe AmyConfig interface
- Validates all config at startup
- Resolves per-agent model overrides with fallback to default
#### Configuration Hierarchy
1. Secrets Store (API keys, tokens)
2. Environment Variables (URLs, models, prompts)
3. Central Config Module (validation, resolution)
4. Per-agent overrides (ASSISTANT_MODEL, etc.)
#### What's Configurable Without Redeploy
- API keys: Update in Secrets Store
- Models: Change env vars (ASSISTANT_MODEL, etc.)
- Prompts: Change env vars or point to remote URL
- URLs: Change env vars
[Update 7/4/2026]: ### Thread Data Model (Added)
Thread has a `type` field:
- `chat` (default) — standard conversation thread
- `copilot` — copilot workspace thread (v2 feature)
v1: type field exists but is ignored, all threads are `chat`
v2: copilot threads use different UI rendering and tools
---
### File Processing Flow (Finalized)
1. User attaches file in frontend
2. Frontend uploads directly to OpenFiles (100MB limit)
3. OpenFiles returns URL
4. URL sent to Think agent via WebSocket message
5. Agent downloads file into workspace using bash/curl
6. Agent decides processing:
- Extract text with Kreuzberg MCP tools
- Store as-is in workspace
- Process with bash/executable tools
7. Agent stores result URL in thread
---
### MCP Integration (Finalized)
- MCP Portal URL stored in env var (user configures portal separately)
- Agent connects to portal via `addMcpServer()` in Think
- Portal provides: Kreuzberg, Crawlberg, OpenFiles, Second-Brain, Todomator
- Agent can toggle servers on/off per session
- Cloudflare Access handles auth
---
### Voice Mode (Finalized)
- Conversation mode using `@cloudflare/voice` with `withVoice(Agent)` mixin
- Continuous STT (WorkersAIFluxSTT) + TTS (WorkersAITTS)
- Single button to start/end call — always listening when connected
- Manual reconnection on drop
- Interrupt handling built in
---
### Cost Strategy
- Workers AI for routine tasks (cheap: $0.011/1K tokens)
- Premium models for complex reasoning (via AI Gateway routing)
- Keep DOs idle when not in use ($0.01/mo each)
- OpenFiles on user's VPS (no CF storage cost)
- Kreuzberg on user's MCP server (no CF cost)
- AI Gateway for usage tracking and cost monitoring
[Update 7/4/2026]: ---
## Implementation Progress (Updated July 4, 2026)
### Phase 1: Core (Completed)
- Worker project scaffolded with 4 Durable Objects
- Central config module with per-agent model/prompt overrides
- 4 Think agents (Assistant, Coding, Research, Business)
- Next.js frontend with assistant-ui
- Multi-workspace UI with sidebar icons
- Custom thread list
### Phase 2: Tools (Completed)
- MCP Portal integration in all agents (configureSession with addMcpServer)
- OpenFiles attachment UI with upload API route
- Model selector component (registers with modelContext)
- Tool call UI (GenericToolUI with makeAssistantToolUI)
- Voice mode component (conversation mode with start/end call)
### Phase 3: Polish (Pending)
- Thread/message persistence verification
- Error handling and reconnection
- AI Gateway model routing
- Performance optimization
### Key Files Created
- amy/worker/src/config.ts - Central configuration
- amy/worker/src/agents/*.ts - 4 Think agents with MCP Portal
- amy/frontend/src/app/page.tsx - Main page with all components
- amy/frontend/src/components/assistant-ui/*.tsx - Thread, ThreadList, ModelSelector, ToolUI, Attachment
- amy/frontend/src/components/workspace/workspace-sidebar.tsx - Workspace icons
- amy/frontend/src/components/voice/voice-mode.tsx - Voice conversation mode
- amy/frontend/src/app/api/upload/route.ts - OpenFiles upload API
Tags: project-amy, implementation-plan, final, v1, project:amy, kind:semantic, status:canonical — Source: claude — 2026-07-04 12:22:40 UTC