Case study
A self-hosted AI agent platform running across WhatsApp, Telegram, Discord, and Slack.
Sole architect and maintainer · Active · since
tests
commits
lines of Python
MCP server modules
LLM providers routed
concurrent burst, exactly-once
external contributors
Test count is measured on origin/develop (11 commits behind origin/main, not yet merged). Every other figure above is measured on origin/main.
Personal AI agents that live in a single chat app are a demo, not infrastructure. The moment you want the same agent reachable from WhatsApp, Telegram, Discord, and Slack, with one memory and one retrieval layer behind all four, you're building a message router, a deduplication layer, and a knowledge store that has to survive concurrent writes from channels that were never designed to talk to each other. Off-the-shelf agent frameworks assume one interface and one conversation at a time. I wanted an agent that remembered a conversation started on Telegram when it continued on Slack three days later, retrieved context through a pipeline that didn't depend on a single index being right, and never dropped a message under load. That meant owning the full stack: async ingestion with bounded queues, a fusion-based retrieval pipeline, and a routing layer that didn't go dark when one LLM provider had an outage.
Four channel adapters: WhatsApp (through a supervised Node/Baileys subprocess), Telegram, Discord, and Slack, plus a CLI and a browser playground, all funnel into one persona_chat() pipeline. Ingestion runs every message through a FloodGate (a 3-second batch window), a deduplication pass (5-minute TTL), and a bounded asyncio FIFO queue (100 deep, two workers): configuration constants read from that same ingestion module, not a benchmark, and the path the burst-ingestion test below exercises directly.
The agent core routes each request across 17 LLM providers with fallback chains, and retrieves context through a fusion pipeline: FastEmbed embeddings (nomic-embed-text-v1.5-Q, 768-dimensional) go through LanceDB's cosine ANN search with a hemisphere prefilter, then a four-factor score fusion (semantic 0.35, recency 0.20, importance 0.20, affect 0.25) with a fast gate that returns early above a 0.80 fused score and a FlashRank (ms-marco-TinyBERT) rerank pass otherwise.
Long-term memory lives in a SQLite knowledge graph in WAL mode, with a node/edge schema kept API-compatible with the NetworkX object it replaced (see the decision record below), so most call sites never had to change even though the query layer underneath did. A separate behavioural-profile system maintains eight JSON layers per user, rebuilt every 50 messages with the last 30 versions kept. 9 MCP server modules (among them browser, calendar, gmail, memory, slack, and tools) expose the agent's tools and memory to external clients.
ADR-001
I held Synapse's knowledge graph entirely in a NetworkX object in process memory. Peak RAM scaled directly with graph size, and a restart lost in-flight state unless a separate serialisation step ran first. My prior attempt tuned NetworkX's own memory footprint and added manual garbage collection after large graph operations. I reverted it once the RAM curve kept tracking graph size almost linearly.
I moved the graph into the existing SQLite store as a node/edge schema, reusing the storage layer already in place for other state rather than adding a new dependency. The rewrite touched every call site the query layer had, and each one needed a test before I could trust the migration.
add an in-memory LRU cache in front of hot subgraphs if query latency becomes the bottleneck instead of memory.
ADR-002
Channel adapters for WhatsApp, Telegram, Discord, and Slack all feed the same agent core. A synchronous, per-channel handler serialises unrelated channels behind one request, and an unbounded queue turns a traffic burst or a retried webhook into unbounded memory growth and duplicate deliveries.
I built one async ingestion layer, shared across every channel adapter, with bounded queues and deduplication ahead of the agent core.
expose queue depth as a metric in the observability layer instead of checking it manually during load tests.
The knowledge graph migration wasn't my first attempt at fixing memory growth. My first attempt tuned NetworkX's own memory footprint and added manual garbage collection calls after large graph operations: a small, inconsistent improvement that added complexity without solving the actual problem, which was structural: an in-memory object graph has no way to page state to disk short of full serialisation. I shipped that approach, ran it for a period under real usage, and reverted it once it was clear the RAM curve was still tracking graph size almost linearly.
The actual fix (moving the graph into the existing SQLite store as a proper node/edge schema, in WAL mode, while keeping the external API compatible with the NetworkX surface it replaced) took me longer to build, because the query layer underneath still needed a full rewrite, and every call site needed a corresponding test before I could trust the migration. That rewrite is where a large share of the 3,897 tests, on origin/develop, came from. The lesson: a memory problem that looks like a tuning problem is often an architecture problem, and the honest fix usually touches more surface area than the first attempt wants to admit.
Four things below are directly reproducible from the repository. Two are not: see the note beneath the commands.
# Full test suite: 3,897 tests, on origin/develop (11 commits
# behind origin/main)
git checkout develop
pytest -q
# Burst-ingestion test: asserts exactly-once processing of a
# 500-message concurrent burst (zero duplicate drops, zero
# full-queue drops, drained inside 60s). This is what the test suite checks;
# no separate run log is published.
pytest workspace/tests/load/test_pipeline_burst.py::test_no_dropped_under_burst_500 -v
# MCP server module count (9)
ls workspace/sci_fi_dashboard/mcp_servers/ | grep -v '__init__.py\|base.py'
# Commit count (835), origin/main
git rev-list --count HEADRetrieval-latency and knowledge-graph memory benchmarks exist as scripts (scripts/bench_retrieval_latency.py and scripts/bench_kg_memory.py, both runnable end to end via bash scripts/run_all_benchmarks.sh), but neither has a committed, dated output on any branch, so I haven't published either figure above. I'd rather ship no number than one I can't point you to.
| Technology | Role |
|---|---|
| Python | Primary implementation language. |
| FastAPI | HTTP and API layer for the agent core and channel adapters. |
| LanceDB | Vector index: cosine ANN search over FastEmbed embeddings. |
| FastEmbed (nomic-embed-text-v1.5-Q) | Generates the 768-dimensional embeddings LanceDB searches. |
| FlashRank (ms-marco-TinyBERT) | Reranks candidates that don't clear the fast-path fusion-score gate. |
| SQLite FTS5 | Full-text search index. |
| SQLite (knowledge graph) | Node/edge storage in WAL mode backing long-term memory: see the decision record above. |
| asyncio | Bounded FIFO ingestion queue: see the decision record above. |
Committing dated output from the existing benchmark harness (scripts/bench_retrieval_latency.py and scripts/bench_kg_memory.py) so the retrieval-latency and knowledge-graph memory figures can be published with evidence instead of staying withdrawn. Merging origin/develop into origin/main so the test count stops needing a branch caveat, and documenting the MCP server modules well enough that an external client can use them without reading the source first.