Memory
# DOMContext Architecture — Stateful LLM Context via j-Components ## Project: DOMContext ## Core Idea The LLM's context IS a DOM page with j-components. Instead of replaying growing conversation history (Pipali's approach), the LLM has a persistent state tree — like a j-component path tree — that represents its "working memory" at each turn. ## Key Insight By using the j-component pattern (path-based addressing, observer reactivity, transport isolation), we can create a stateful context for the LLM. The LLM reads the DOM to see its context, writes to paths to set state for the next turn, and SurrealDB live queries keep dynamic sections fresh. ## Architecture: Three Layers of j-Component Interaction ### Layer 1: Context DOM (what the LLM reads/writes each turn) - `aicontext` — static context (system prompt, goals) - `datagrid` — interactive data views (NoSQL-backed) - `objecteditor` — record editing - LLM-writable paths — scratchpad, todo, state ### Layer 2: Flow Graph (how the context is assembled) - j-Flow nodes = context assembly steps - Connections = data flow between steps - The FlowStream pipeline IS a j-Flow graph ### Layer 3: Agent Workflows (what the LLM can compose) - j-Flow nodes = tools/actions - Connections = workflow steps - The LLM can read and modify the flow ## How a Turn Works ``` 1. OBSERVE: Serialize context DOM to text (like page-agent's flatTreeToString) 2. ASSEMBLE: Combine with system prompt + tools 3. THINK: LLM processes, decides what to do 4. ACT: LLM writes to context paths (jC.set()) - context.memory.scratchpad = "Working on auth.ts..." - context.memory.todo = ["Add tests"] - context.state.current_file = "src/middleware.ts" 5. PERSIST: Save state tree to NoSQL (not full history) 6. NEXT TURN: Context DOM now reflects LLM's writes ``` ## Context DOM Structure ``` context.system.prompt → [aicontext component] → always populated context.system.goal → [aicontext component] → always populated context.user.message → [input component] → current user input context.relevant.code → [SurrealDB live query] → auto-updates context.relevant.history → [SurrealDB live query] → auto-updates context.memory.scratchpad → [LLM-writable component] → LLM sets for next turn context.memory.todo → [LLM-writable component] → LLM sets for next turn context.tools.available → [tool definitions] → always populated context.data.* → [datagrid viewers] → NoSQL-backed interactive data ``` ## Data Flow: Spreadsheet → NoSQL → Viewer ``` User loads spreadsheet ↓ Parsed into NoSQL records (each row = a record) ↓ LLM opens datagrid viewer (j-component) ↓ LLM can: - Filter: "show me rows where status = 'error'" - Sort: "sort by date descending" - Search: "find records containing 'authentication'" - Select: "select all records from last week" - Edit: "update the status field of record X" - Navigate: "go to page 5" ↓ Each interaction is a NoSQL query, not a full dump ``` ## Technology Stack | Component | Technology | Role | |---|---|---| | Server | Total.js v5 | Persistent server framework | | AI Module | Native AI() fluent API | Multi-provider LLM calls (OpenAI, Claude, Gemini, Ollama) | | Database | NoSQL (Total.js embedded) | Immutable ATIF conversation storage, LLM state, parsed data | | Semantic DB | SurrealDB (embedded) | Vector search, graph traversal, FTS, live queries | | Context Assembly | FlowStream | Visual pipeline for building LLM context | | Flow Editor | j-Flow | Visual workflow composition | | Context Components | j-components (aicontext, datagrid, objecteditor) | Reactive context DOM | | Transport | WebSocket (jSocket pattern) | Real-time sync between server and client | | Tool System | NEWACTION | File, shell, web, data tools | | Desktop | Tauri (optional) | Desktop shell | ## Why This Is Better Than Pipali | Pipali | DOMContext | |---|---| | Replay full conversation history every turn | Fixed-size state tree, updated in place | | Raw output passthrough grows unboundedly | LLM writes only what it needs to remember | | Context = everything that ever happened | Context = what's relevant right now | | Platform-side compaction (reactive, lossy) | LLM-controlled state (proactive, intentional) | | 50K-300K tokens per turn after 10 iterations | Potentially 10-20K tokens regardless of history length | | Static tool outputs (text dumps) | Interactive data views (datagrid viewers) | | No visual workflow composition | j-Flow for composable workflows | | Platform dependency (billing proxy) | Direct API keys, self-hosted | ## Key Components Explored ### From Pipali (preserve the tool quality) - 11 actor implementations (read_file, edit_file, write_file, grep_files, list_files, shell_command, search_web, read_webpage, generate_image, email_user, ask_user) - Multi-format parsing (PDF, DOCX, XLSX, PPTX, images) - Three-strategy search cascade (mdfind → ripgrep → walker) - LLM-based webpage extraction - OS-level sandboxing (Seatbelt/Bubblewrap) - Confirmation system for dangerous operations - Skills system with progressive disclosure - MCP integration for external tools ### From Total.js v5 - Native AI() fluent API with multi-provider support - NEWACTION for tool endpoints - NoSQL embedded database - FlowStream for visual orchestration - j-component system for reactive UI - SHELL(), RESTBuilder, NOSQL() globals ### From SurrealDB - Vector embeddings (HNSW, DISKANN) for semantic search - Graph relations (RELATE) for entity links - Full-text search (BM25) for keyword matching - Live queries for real-time context updates - Embedded mode (single Rust binary, in-process) - Hybrid queries (vector + FTS + graph in one query) ### From page-agent - Text-based DOM representation (flatTreeToString) - Index-based element targeting - MacroTool pattern (reflection + memory + action) - Agent loop (observe → think → act → repeat) ### From j-components - Path-based state tree (jC.get/set/update) - Observer reactivity (setter/getter lifecycle) - datagrid (virtual-scrolled data grid with filtering/sorting/pagination) - objecteditor (JSON object editing) - aicontext (template-based context injection) - j-Flow (visual flow/graph editor) ## Architecture Diagram ``` ┌─────────────────────────────────────────────────────────────────┐ │ Total.js Server │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ FlowStream │ │ │ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ │ │ │aicontext│ │SurrealDB│ │ Merge │ │ AI() │ │ │ │ │ │(static) │ │(live │ │(context │ │(LLM │ │ │ │ │ │ │ │ queries)│ │assembly)│ │ call) │ │ │ │ │ └─────────┘ └─────────┘ └─────────┘ └─────────┘ │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ NoSQL │ │ │ │ • ATIF conversation history (immutable) │ │ │ │ • LLM state tree (persistent scratchpad) │ │ │ │ • Parsed data (spreadsheets, CSVs, etc.) │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ SurrealDB │ │ │ │ • Vector embeddings (semantic search) │ │ │ │ • Graph relations (entity links, conversation threads) │ │ │ │ • Full-text search (keyword matching) │ │ │ │ • Live queries (auto-updating context) │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ Tools (NEWACTION) │ │ │ │ • read_file, edit_file, write_file, grep_files │ │ │ │ • shell_command, search_web, read_webpage │ │ │ │ • datagrid viewer, object editor │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ │ │ WebSocket (jSocket pattern) ▼ ┌─────────────────────────────────────────────────────────────────┐ │ Client (Browser / Tauri) │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ Context DOM │ │ │ │ • j-components bound to paths │ │ │ │ • Reactive updates via jC.set() │ │ │ │ • LLM reads DOM, writes to paths │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ j-Flow Editor │ │ │ │ • Visual workflow composition │ │ │ │ • LLM can read and modify flows │ │ │ │ • Context assembly pipeline visualization │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ## Status Architecture exploration complete. Ready to discuss implementation plan.
Tags: architecture, design, totaljs, surrealdb, j-components, llm, agent, context-management, project:domcontext, kind:semantic — Source: claude — 2026-07-08 14:50:25 UTC