brainstack

Team Brain — Collaborative AI Memory (plan)

Status: P0–P4 + sync mode + Realtime full push (#31) + semantic recall opt-in (#4) shipped (anon rate limits closed — see #32)
Related: #2, team-brain.md, team-brain-onboarding.md, scopes.md

This document captures the original Team Brain intent, what shipped for collaborative AI memory (FTS + optional semantic recall + agent loop), and remaining gaps — without depending on external memory products.

1. Original plan (issue #2)

When engineer-brain was personal-only, Team Brain meant:

Intent Detail
Problem 2–3 engineers on the same spike each cold-start AI → duplicate research and token burn
Product Initiative-scoped shared context (research, decisions, open questions)
Privacy Personal BRAIN.md stays private; team layer is opt-in
Commands init / attach / sync / breakdown (+ team config)
Portability Same cross-platform story as personal brain
Success Less duplicate research, lower tokens before first PR, faster 2nd/3rd engineer onboard, better epic breakdown

Markdown was always a portable surface. The outcome was agents reusing team memory — not a shared wiki.

Early demo slice (superseded)

That thin log is kept as compat aliases. Current product: remember / recall → Supabase SoT + cache/<KEY>.json, MCP, Cursor agent loop, breakdown / metrics.

2. Target product

Not a notebook. Collaborative AI memory between engineers on a Jira key, backed only by Supabase.

Engineer A agent                    Supabase                         Engineer B agent
     |                                 |                                    |
     |-- remember(jira, kind, body) -->| INSERT memory (+ embed async)      |
     |                                 |-- Realtime INSERT event ---------->|
     |                                 |                                    |-- inject into context
     |-- recall(jira, "auth flow") --->| vector / FTS search                |
     |<-- top-k memories --------------|                                    |
Layer Role
Jira Initiative spine (YOU_JIRA_TICKET_HERE)
Supabase Source of truth for memories + membership + realtime
Local cache .team-brain/cache/<KEY>.json for agents (fast read)
Markdown export Optional human/git mirror — not the sync bus

When sync happens

Mode Trigger
Realtime (preferred) Subscribe to inserts for attached initiatives
Periodic / session On attach, skill start, before breakdown, or every N minutes
Write remember when a finding is durable (agent or human)

Re-running sync must not duplicate rows. Local cache/log is rebuilt or merged from server identity (id / source_ref).

3. Core memory bar (Supabase-only)

The product must deliver this core memory loop:

  1. Write once, reuse by others — remember
  2. Retrieve by meaning — recall / search_memories (FTS → optional pgvector)
  3. Live awareness — near-realtime updates for attached initiatives (watch today; push later)
  4. Agent-native surface — MCP / skill tools: remember, recall, list_recent
  5. Dedup / source identity — source_ref (+ content hash) unique per initiative

Product fit (Brain umbrella):

Later polish: version/rollback, snapshots, richer metrics.

4. Phased build

P0 — Stop being a notebook (done)

P1 — Near-realtime watch (done — poll + full-content Broadcast push)

Decision (poll): use authenticated polling (list_recent + p_since), not postgres_changes.

Reason: captures revoke SELECT from anon; auth is custom p_api_key on RPCs. Realtime CDC uses the JWT role and cannot see rows without opening SELECT to everyone (leak). Polling keeps member-key security for content.

Decision (push, #31 — full content): rather than migrating membership to Supabase Auth JWTs to unlock private Realtime Authorization, Team Brain closes the same gap with application-layer encryption. Each team gets a random 256-bit broadcast_key (server-generated; returned only to a resolved member via memory_broadcast_topic). The CLI encrypts the body (AES-256-CBC + HMAC-SHA256, encrypt-then-MAC) before remember, and the DB trigger forwards that opaque body_ct — still on the same public topic team-brain:{team_id}:{JIRA_KEY} — never decrypting it. A peer holding the key decrypts inline in team-brain-realtime.py and writes straight into .team-brain/cache/<KEY>.json: zero extra RPC round-trip. Anyone without the key (or without the optional cryptography dependency) only ever sees ciphertext, and transparently falls back to the original signal + authenticated _pull_signal pull — the same safety net #31 shipped with initially.

P2 — Semantic recall (done — one-command crew opt-in, #4)

Embedding providers (OSS)

Provider Env Notes
none (default) unset FTS-only recall; no API key needed
openai TEAM_BRAIN_EMBED_PROVIDER=openai + TEAM_BRAIN_EMBED_API_KEY or OPENAI_API_KEY Model default text-embedding-3-small with dimensions=768. Cost: ~$0.02 per 1M input tokens (Aug 2026 pricing) — a crew writing a few hundred memories/month costs cents. Body leaves your machine → OpenAI.
ollama TEAM_BRAIN_EMBED_PROVIDER=ollama Default model nomic-embed-text (768-d); TEAM_BRAIN_EMBED_BASE_URL optional. Cost: $0 (local inference). Body never leaves your machine — best choice for sensitive/regulated content.

One-command opt-in (recommended path):

bash core/scripts/team-brain-api.sh enable-semantic openai   # or: enable-semantic ollama
export TEAM_BRAIN_EMBED_API_KEY=sk-...                        # openai only — never persisted to team.yaml
bash core/scripts/team-brain-api.sh enable-semantic openai   # re-run to verify: prints vector dims on success

bash core/scripts/team-brain-api.sh remember YOU_JIRA_TICKET_HERE research "EE schema path lives in …"
bash core/scripts/team-brain-api.sh recall YOU_JIRA_TICKET_HERE "where is decision_environment scaffolded"
# → stderr: "recall mode: vector (openai)"; response body: "mode": "vector"
bash core/scripts/team-brain-api.sh reembed YOU_JIRA_TICKET_HERE        # backfill memories written before opt-in

Manual/legacy path (still works, e.g. for one-off scripting): export TEAM_BRAIN_EMBED_PROVIDER / _MODEL / _BASE_URL directly — enable-semantic is just a documented, persisted shortcut for the same env vars.

When FTS is enough vs when to turn on vectors: FTS (default, zero setup) is fine for exact/near-exact keyword recall — “find the memory where we discussed X”. Turn on vectors once a crew is asking conceptual questions (“did anyone already figure out auth for this service?”) where the right memory doesn’t share vocabulary with the query.

P3 — Agent parity

Agent loop (what makes it a shared brain)

Engineer A laptop                         Supabase                         Engineer B laptop
     |                                       |                                    |
     |-- recall(KEY) ----------------------->|                                    |
     |<-- crew memories ---------------------|                                    |
     |-- (research) --- remember(KEY, ref) ->|                                    |
     |                                       |<-- recall(KEY) --------------------|
     |                                       |--- memories (incl. A's) ---------->|
     |                                       |                                    |-- (reuse, then research)

Humans can still run CLI manually; agents must not skip the loop.

P4 — Beat on workflow

bash core/scripts/team-brain-api.sh breakdown YOU_JIRA_TICKET_HERE
bash core/scripts/team-brain-api.sh metrics YOU_JIRA_TICKET_HERE
# → initiatives/YOU_JIRA_TICKET_HERE-breakdown.md + metrics.json (gitignored)
bash core/scripts/team-brain-api.sh metrics --team   # crew coverage + reuse (#35)

Team aggregation v1 signals (#35)

In scope (opt-in Team Brain activity) Out of scope for v1
Member display_name × memory kind counts Personal BRAIN.md / career skills
Member × initiative (jira_key) memory counts Memory bodies / source_ref text in aggregate payload
Memories per initiative + ISO-week activity GitHub/GitLab review collaboration graph
Local metrics.json recall overlay (this machine only) Uploading local metrics to the server
Crew members with a valid team api_key Org-wide dashboards beyond the crew

CLI: metrics --team or aggregate → prefers RPC team_aggregate_metrics (migration 20260807000001_…); falls back to list_recent with bodies stripped.
Privacy: same boundary as list_initiatives — never uploads personal BRAIN.md; aggregate response has no bodies.
Parent: #2 · issue #35 · roadmap: roadmap.md

5. Working TODO checklist

Use this as the build board (check off in PRs):

ID Phase Task Owner notes
M1 P0 Migration …_team_brain_memory.sql Apply after v1 (SQL Editor)
M2 P0 remember RPC (+ dedup) ✅ in migration
M3 P0 search_memories + list_recent ✅ FTS
M4 P0 CLI remember / recall + cache write ✅
M5 P0 Skill/docs language shift ✅
M6 P1 watch (authenticated poll) ✅
M7 P2 pgvector + embed path ✅ — apply …embeddings.sql
M8 P3 MCP server tools ✅ mcp/team-brain/
M9 P4 breakdown ← recall + metrics ✅

6. Privacy & security

7. Success metrics (from #2, updated)

7b. Tombstone delete + peer cache eviction (#66)

When poisoned or stale team context must be removed (not just corrected), members and admins call delete_memory — viewers cannot.

Layer Behavior
Server Sets captures.deleted_at; appends memory_deletions + capture_revisions audit rows
Read RPCs list_recent, search_memories, team_aggregate_metrics exclude tombstones
Undelete remember at the same source_ref clears the tombstone (undeleted: true)
Local cache Authoritative poll replace each sync cycle; realtime push sends deleted: true so peers purge without manual cache wipe 
# Tombstone (member/admin)
bash core/scripts/team-brain-api.sh delete YOU_JIRA_TICKET_HERE --source-ref "YOU_JIRA_TICKET_HERE#bad-claim"

# Admin audit crew tiers
bash core/scripts/team-brain-api.sh list-members

Workshop Part 5 flow: A stores poison → B recalls → B deletes → A’s cache evicts on next poll or realtime tombstone signal → A no longer recalls the deleted finding.

Smoke tests: tests/team-brain/cache-purge-smoke.sh (local), tests/team-brain/governance-smoke.sh (live RPC, optional env).

7c. Redundant-memory feedback + admin approval queue (#67)

Feedback engine: remember detects near-duplicates (FTS; vector when embeddings enabled) and cross-author source_ref overrides on research/decision. Returns redundant_candidate: true — nothing is stored until the author fixes the slug or queues for review.

Detection thresholds (v1)

Path Trigger Notes
Vector (when embedding supplied) cosine distance ≤ 0.12 (~similarity ≥ 0.88) Issue #67 draft suggested 0.08 (≥0.92); v1 uses 0.12 to reduce false blocks on paraphrases
FTS (fallback / no embedding) ts_rank ≥ 0.05, query body ≥ 8 chars Uses plainto_tsquery('english', …) on search_tsv
Exact hash identical content_hash Auto-dedupe (unchanged)
Cross-author source_ref research/decision only note/learning still auto-update same source_ref (author merge)

Response contract

Outcome Key fields
Success (inserted / updated / deduped) id, initiative_id, content_hash, has_embedding, author_member_id, author_name, created_at, updated_at, plus result
Blocked redundant_candidate: true, matches[], conflict_reason, suggested_action
Queued pending_submitted: true, pending_id, conflict_reason

suggested_action values (v1): recall_and_merge_same_source_ref_or_queue_for_review, use_same_source_ref_after_recall_or_queue_with_p_queue_for_review, await_admin_approval.

Admin approve: promotes queued body to live memory and attributes author_member_id to the submitter (credit for the improved finding).

Situation Default Member escape hatch Admin
Identical body Auto-dedupe — —
Same source_ref, same author Auto-update + archive — --force immediate
Same source_ref, different author (research/decision) Block → redundant_candidate remember … --queue pending approve <pending-id>
New body, semantic/FTS match Block → redundant_candidate + matches[] remember … --queue pending approve <pending-id> 
# Member blocked — see matches, reuse source_ref:
bash core/scripts/team-brain-api.sh remember DEMO-1 research "…" 
# → redundant_candidate (not stored)

# Member queues override for admin:
bash core/scripts/team-brain-api.sh remember DEMO-1 research --source-ref "DEMO-1#auth" --queue "…"

# Admin reviews + approves (promotes to live memory):
bash core/scripts/team-brain-api.sh pending list DEMO-1
bash core/scripts/team-brain-api.sh pending approve <pending-id> --note "Better finding"

# Admin rejects (keeps existing live memory):
bash core/scripts/team-brain-api.sh pending reject <pending-id> --note "Duplicate of Alice's note"

Workshop Part 5: B tries to re-store A’s topic → feedback engine blocks → B recalls A’s source_ref OR queues → admin keeps only the improved version.

Admin review dashboard: dashboard/ — local /admin/review UI (same RPCs as CLI). Run npm run dev in dashboard/, open /admin/review, connect with admin tb_… key. GitHub Pages demo does not include live review (credentials stay local).

8. Future hardening (roadmap)

Capability Today Next
Recall before research ✅ Rule/skill/MCP Keep mandatory
Remember after findings ✅ Direct save Keep mandatory
Dedup (source_ref) ✅ —
Redundant guard + pending queue ✅ CLI/MCP (#67) ✅ Admin dashboard UI (dashboard/admin/review)
Correction / learning ✅ correct + learning kind —
Version/history/soft rollback ✅ history / restore + capture_revisions Optional snapshots / UI
Tombstone delete / governance ✅ delete + list-members; peer cache eviction Per-initiative ACLs (deferred)
Semantic search ✅ One-command opt-in (enable-semantic); FTS default (#4) —
Live push into other agent context ✅ Full-content encrypted Broadcast + poll fallback (#31) Optional key rotation command if a member is offboarded
Model compliance ✅ Stronger prompts + soft session gate (compliance / prepare_research; CLI not hard-blocked) Optional hard gate / metrics later
Metrics Local metrics.json + crew metrics --team (#35) Optional team dashboard UI
Request/body hardening ✅ remember 20000-char body cap; curl --max-time on all RPCs; doctor readiness preflight —

9. Out of scope (for now)

References

This site is open source. Improve this page.