The pattern, in one paragraph
A 10-second voice note in Telegram becomes a transcribed message, an entity-extracted JSON blob, a node in a Neo4j knowledge graph, an embedding in a Qdrant collection, and — if it implies a follow-up — a Lead in ERPNext and a draft email in Listmonk. Total elapsed time, voice to CRM: about a minute. I run this for myself every day and adapt it for clients who want institutional memory without an MBA project.
Why I built it
I lose context faster than I can write it down. The names of three people I met yesterday, the open thread on a quote I sent two months ago, the half-formed idea for a calculator I had in the car — all of it evaporates if I don't capture it within minutes.
CRM tools assume you'll sit at a desk and type a structured Activity. I won't. Notes apps assume you'll come back later to organise. I don't. The only capture surface that survives contact with my actual life is the chat app I already have open — Telegram. Anything that requires switching apps loses.
So the architecture had to start from "voice note in Telegram" and work backwards. Everything else — the graph, the vectors, the CRM — is downstream of that decision.
What's in the stack
Five open-source pieces, plus a small custom orchestrator:
Telegram bot front door — captures voice or text, drops the audio into a watch directory, queues the message ID for the rest of the pipeline.
Whisper, on a local GPU — transcribes the audio to text. I run it on a Proxmox container with ROCm-compiled PyTorch on a Radeon GPU. Twelve seconds of audio takes about two seconds to transcribe. The first time I tried this on CPU, it took thirty seconds and I almost gave up on the whole pattern.
Ollama with a small instruct model — extracts entities (people, organisations, projects, follow-ups, dates) into a structured JSON blob. Local model, never leaves the network. The schema is fixed enough that I can validate it before persisting anything.
Neo4j as the knowledge graph — every entity becomes a node, every relationship an edge, with provenance back to the original transcript. "Who introduced me to X" and "what was the last thing I told Y about the calculator" are real Cypher queries against this graph months later.
Qdrant as the vector store — the same notes are embedded with a 1024-dim local model and stored with a hard cross-reference invariant: every vector points to a graph node, every graph node has at least one vector. Semantic recall and graph traversal stay in sync, no orphan vectors.
ERPNext as the CRM — when an entity extraction implies a follow-up ("John asked for the spec sheet by Friday"), the pattern creates a Lead with provenance back to the note.
Listmonk for outbound — drafts a campaign with the suggested copy, parked unsent.
The whole thing is orchestrated on Windmill — a code-first workflow engine I migrated to from n8n once the flows started needing strong typing, real testing, and long-running suspend/resume semantics.
The two non-negotiables
There are two invariants I will not break, and I would not advise anyone else to break them either.
Cross-reference between graph and vectors
Every Qdrant point carries the Neo4j node ID it came from in its payload. Every Neo4j node has a property listing the Qdrant point IDs that reference it. They drift if I'm not careful — and when they drift, I get a vector hit that points to a graph node that no longer exists, or a graph node I can't recall semantically. So there's a reconciliation job. It's two SQL-flavoured queries and a diff. It runs every hour. It is the most boring code in the entire stack and I would not remove it for anything.
Human approval on every outbound
Nothing in this pattern sends an email by itself. Ever. When the pipeline drafts a Listmonk campaign or wants to update a CRM field that a person will see, it pauses the workflow with Windmill's suspend/resume primitive and sends me a Telegram prompt with the full context. I tap yes, the workflow resumes, the email goes. I tap no, it's discarded.
This is the only reason I trust the pattern. The cost of a wrong outbound — a hallucinated commitment, a bad date, a name swap — is permanent. The cost of a five-second approval tap is nothing. The asymmetry is overwhelming.
What I learned migrating from n8n to Windmill
I started this on n8n because n8n is where my brain goes for "wire these services together". For a while it was fine. The trigger was a Telegram webhook, the steps were Function nodes, the persistence was HTTP Request nodes against the Neo4j and Qdrant REST APIs.
It started fighting me when:
- The transcript step needed to retry with backoff against Whisper.
- The entity extraction step needed schema-validated output, and "JSON in a Function node" is a long way from real validation.
- The approval step needed to suspend the workflow for hours, sometimes days, and resume cleanly.
- I wanted to run the same flow against historical voice notes to backfill the graph, and "n8n's manual execution semantics" don't quite fit that.
Windmill solves all four. Steps are TypeScript or Python files. Validation is just types. Suspend/resume is a built-in primitive that doesn't burn a worker. Backfills are scripts that call the same flow.
I still use n8n for things n8n is great at — Telegram receive, simple scheduled triggers, integration glue with services that have an n8n node and no decent API. The Telegram bot front door for the Second Brain is still an n8n workflow that just forwards the payload to Windmill. Use the right tool per workflow.
What this pattern is not
It is not magic. It will not surface the right note at the right moment unprompted. It is not a personal AI assistant in the sci-fi sense.
It is a capture and recall layer. The recall is via deliberate query — Cypher against Neo4j, semantic search against Qdrant, dashboards against ERPNext. The pattern's job is to make sure that when I do go looking, the thing is there, with provenance.
Anything that promises more than that — anything that proactively sends emails, updates CRMs, or makes commitments on my behalf — is, in my experience, a bug factory. The interesting design work is in the boundary between automation and approval, not in pushing further into automation.
Adapting it for a client
The pattern generalises. The substitutions I've made for client deployments:
- Telegram → Slack, Microsoft Teams, or a custom mobile-friendly web form.
- Whisper local → Whisper API (where data residency allows) or a self-hosted model on the client's infrastructure.
- Local Ollama model → a model the client already runs internally for other AI workloads.
- ERPNext → whatever CRM the client uses (HubSpot, Salesforce, a homegrown one).
- Listmonk → the client's existing email platform.
The two non-negotiables stay non-negotiable. The cross-reference invariant and the human approval gate are the difference between a useful tool and a liability.
If you're considering this for your own organisation, the gating question is whether you have someone willing to act as the human gate — to triage approvals reliably, fast enough that the pattern stays useful. If yes, the rest is implementation.