Memory
DESIGN — MINI-SPHERE GADGET (practicality pass + spike evidence; drafted by the workspace agent with Mike, Sep 4 2026; grounds the contract draft captured alongside). Evidence standard a5e10e10 (as data where implemented; referenced as convention where not). WHAT / WHY: Mini-Sphere is a self-contained, blueprintable Gadget giving each workspace its own memory + map layer — nodes, typed edges, recursive recall-with-hops. It closes a parity gap: the workspace agent has NO in-workspace memory today (env inspection confirms only CONTEXT + SPHERE bindings, both external Fetchers), unlike the think agent which has a knowledge DO (FTS5) + memory store. Mini-Sphere is the workspace-agent-native equivalent: Cap'n Web RPC to a gadget DO with SQLite, no MCP surface, no approval queue. SOURCE BASIS (important correction): the live sphere substrate is D1 SQLite + Cloudflare Vectorize + KV + Workers AI (db/schema.sql + src/db/init.ts) — NOT sphere_schema_v2.surql. The SurrealDB / tri-temporal / T1-T4 / live-queries / entity-topic-keyword-graph in the vision entries (3e1f9f20) are v2 aspirations, absent in the tree. The contract must pin against the REAL db/schema.sql, not the aspirational .surql. Net: Mini-Sphere is a port of the ACTUAL current sphere semantics into a gadget DO SQLite, minus two cloud services (Vectorize, Workers AI), plus optionality — a smaller delta than the vision implied. SPIKE EVIDENCE (throwaway SPIKE gadget, this workspace, hard results): - FTS5 AVAILABLE: CREATE VIRTUAL TABLE ... USING fts5 + MATCH + bm25() + ORDER BY rank returns ranked BM25 results. Keyword arm = BM25, not LIKE. - Recursive CTE AVAILABLE: BFS graph traversal depth 0-3 over nodes+edges returns correct reachable set with paths + edge weights. Hops mechanic ports verbatim to one recursive CTE. - DO alarms FALSIFIED: "Facets currently cannot set alarms." Gadget DOs cannot self-schedule. RESOLUTION: scheduled-tasks gatekeeper fires the maintenance RPC on a cron (governed + audited; matches the gatekeeper-family principle 6ed41ea6), with lazy staleness as read-time fallback. - Minor: some SQL built-ins gated ("sqlite_version() not authorized"); FTS5/bm25 and recursive CTE are authorized. PRACTICALITY PASS (column A of the sphere coder's breakdown): MAPS CLEANLY (pure logic, no deps — keep): nodes/edges/provenance; 8 closed edge types (kind-gated, symmetric relates_to canonicalized by id order); lifecycle (canonical/draft/deprecated); volatility (durable/state/volatile); kind (episodic/semantic); supersession (supersedes edge w1.0 + deprecate+deindex); graph traversal/hops (BFS fanout<=8 maxNodes50 score=seed*0.6^hop*edgeWeight); rerank (time-decay w/ volatility floors 0.9/0.6/0.15, log1p(recall_count), importance, MMR lambda=0.7); RRF fusion; keyword arm (now BM25); snippet budgeting (4000/400/12000 — vital for agent context); temporal query rewrite (queryUsed); stats/brief/tags rollups (pure SQL); staleness regex heuristic (no LLM); crypto.randomUUID ids. NEEDS OPTIONAL AI/EMBEDDING BINDING (pluggable, default off): vector/semantic recall; semantic dedup-on-capture (0.95 block / 0.85 flag); auto-linking (top-3 >=0.78); contradiction/supersession detection; digests/roll-ups; auto-patterns; LLM classify (importance/kind/canonical); LLM query-tag inference. DROP (column B): MCP surface; approval queue; gatekeeper write-aperture; ingestUrl/crawl/document-extraction; cross-workspace; live queries; SurrealDB/VPS. KEY ARCHITECTURAL SUBSTITUTION: in Mini-Sphere the consumer is an agent (the workspace agent), not a human typing. Sphere runs an LLM POST-WRITE to classify importance/kind/canonical + infer query tags. In a gadget the CALLING AGENT supplies those AT WRITE TIME — it knows whether it is storing a fact or an event. This replaces a class of LLM calls with caller-provided metadata (cleaner, cheaper), making LLM-classify an optional fallback binding, not a core dependency. DELIBERATE DEVIATIONS (recorded as decisions, not carried-over incumbent limits): 1. Normalize tags into a tags(entry_id, tag) table + dedicated indexed columns for status/kind/volatility — fixes the /tags count-aggregation bug class (witnessed live this session: all counts 0), faster filters than JSON-LIKE. 2. History via a revisions(entry_id, content, at) table, not sphere's content-string-append — cleaner, queryable, no parsing. (Alt: byte-faithful single-row; pending ruling.) 3. Pluggable retrieval arms — vector arm is a binding, not a Vectorize hard-dep. Zero-binding default (keyword+graph+rerank) is already strong; semantic gains when an embedding binding is wired. 4. Uncertainty-as-native (a5e10e10) — sphere has NO evidence-standard column (it is an outside contract). Mini-Sphere implements the v2 vision: real claim-tag ([VERIFIED]/[INFERRED]/[UNKNOWN]) + evidence ref on entries/edges. Cheap, differentiating. Proposed for v1. 5. No chunking unless a binding is present — store whole content; chunk (1600/200) only matters for embedding. SCHEDULED MAINTENANCE: the scheduled-tasks gatekeeper calls a narrow, named maintenance RPC (e.g. runStalenessPass(limit)) on a cron; audited per run; lazy staleness on read as fallback if the scheduler has not run. Does NOT re-introduce the write-aperture/approval-queue — the gatekeeper here is a scheduler, not an approver. Wiring = a SCHEDULED_TASKS binding in the creation manifest (contract-at-birth). DECISION POINTS (for Mike to rule before ratification): DP-MS1 tags: normalized table + dedicated status/kind/volatility columns? (recommend yes) DP-MS2 history: revisions table vs content-string-append? (recommend revisions) DP-MS3 vector arm in v1: in-scope as optional binding (default off) vs out-of-v1? (recommend in-scope, default off) DP-MS4 caller-provided classification replacing LLM classify — adopt? (recommend yes) DP-MS5 dedup: exact-hash baseline + binding-powered semantic vs drop? (recommend hash baseline) DP-MS6 digests/auto-patterns: drop for v1 (keep tag stubs)? (recommend drop) DP-MS7 uncertainty-as-native (a5e10e10): in v1? (recommend yes) DP-MS8 staleness: scheduled-tasks gatekeeper + lazy fallback (per spike)? (recommend yes) DP-MS9 contract name/placement: MINI_SPHERE_CONTRACT as child of SPHERE_GATEKEEPER_CONTRACT vs profile-of-v2? (recommend separate child contract) DP-MS10 FTS5 confirmed; keyword arm = BM25. (evidence, not a decision) LINKS: mini-sphere work item 9c51aced; sphere substrate 3e1f9f20; three-layers boundary 2144b79b; foundational vision cf00a760; gatekeeper-family principle 6ed41ea6; evidence standard a5e10e10. Contract draft captured alongside (formal link to be created on approval).
Tags: docuclear, mini-sphere, design, practicality, spike, sqlite, graph, fts5, gatekeeper, scheduled-tasks, project:docuclear, status:draft, volatility:state, duplicate-candidate, kind:episodic — Source: mcp — 2026-09-04 20:12:08 UTC