pablan/docs/notes.md

Implementation notes

Durable engineering learnings — decisions, calibrations and gotchas that are still true of the system as it stands. Grouped by theme rather than by when they were learned. The roadmap says what exists; this says why it is the way it is and what will bite you. For formal decisions see decisions.md; for the as-is contract see architecture.md and data-model.md.

Retrieval & embeddings

• One permission filter, evaluated live. rag/permissions.py is the single source both the API and retrieval use, and it runs against the current tables — never against the denormalized visibility/department copy stored in chunk meta (that copy is display-only and can lag a reindex). This is what makes "a chunk the user may not read is structurally unreachable" true rather than aspirational.
• Authors vs retrieval. An author sees their own drafts through the API (readable_documents_filter), but retrieval is published-only (searchable_documents_filter), so a draft never reaches an LLM prompt. Admins get no read-everything bypass — an admin only sees what the normal rules grant.
• Hybrid search is one SQL statement. pgvector top-20 + German FTS top-20 fused with Reciprocal Rank Fusion, RRF_K=60. Keeping it one statement is what lets the permission CTE gate everything before ranking.
• Thresholds are calibrated against bge-m3 on the fixture corpus, and must be re-measured if the embedding model changes. No-answer fires at vector distance ≥ 0.45 (matched hits land ~0.34–0.45, unrelated ~0.50+). The similarity path (similar_chunks) uses max_distance 0.45 for capture-context.
• A chunk is embedded WITH its heading path (rag/indexing.embedding_text). A section reads "Solldruck 180 bar" and never repeats which machine it belongs to, so on its own it is unreachable by the name the asker uses. Measured on the dev corpus: "Welcher Solldruck gilt für die Hydraulikpresse?" went from a no-answer (best distance 0.492, above the 0.45 gate) to a confident hit (0.375, and the section that literally answers it at rank 2). The genuine no-answers moved too little to matter (Deutschlandticket 0.503 → 0.488), so the gate keeps separating them. What is STORED as content stays the raw section: the path is context for the vector, not part of the document, and the excerpt a reader sees must not grow a header. Tests that search for "the exact chunk text" have to build it through embedding_text or they land at distance ~0.96.
• The chunk tsv covers the heading path too (generated column over content || ' ' || meta->>'heading_path'). Same reason on the keyword side: "Solldruck Hydraulikpresse" matched no chunk before, because only the document title carries the machine name. Note what it does NOT fix: websearch_to_tsquery is AND over every surviving lexeme, and German stopword lists keep verbs, so "Welcher Solldruck gilt für die Hydraulikpresse?" still has no keyword match. Treat fts_match as "the keywords really are in there", not as "this was a keyword question".
• Calibrate on what the engine renders, not on what a human writes. Real capture drafts are compressed notes and sit further from their source than a hand-written paraphrase, so thresholds tuned on paraphrases miss real matches.
• rag/similarity is deliberately not the hybrid path. RRF produces a fusion rank, not a similarity, and its vector distance is null for FTS-only hits — a threshold needs a comparable number, so similarity is a pure-cosine query with the threshold applied in Python after ORDER BY … LIMIT (a distance predicate in WHERE fights the HNSW index).
• HNSW post-filtering can under-fill. For a heavily permission-filtered user, the fixed candidate fetch can return fewer rows than expected; keep candidate counts generous.
• Hybrid retrieval is robust to statement-vs-question phrasing; the pure-cosine path is not. A declarative statement ("Die Zentralschmierung … muss aufgefüllt werden.") retrieves the same top document as the equivalent question, because the German FTS component matches the keywords regardless of phrasing — guarded by declarative- statement cases in golden_queries.yaml. The pure-cosine similarity path (similar_documents/similar_chunks, used by the capture picker and refinement grounding) has no FTS to lean on and is therefore more sensitive to how a thought is phrased. It stays pure-cosine on purpose (a comparable distance for its threshold), so a phrasing-sensitive picker is a known trade-off, not a bug.

Permissions & auth

• FK delete semantics encode the product. Authored documents survive their author (ON DELETE SET NULL) — "an employee leaving is exactly the case the product exists for". The same holds for a document's department.
• The auth boundary is a full-page navigation. Login and logout do a real document load (window.location.assign), never a client goto. Module-level runes singletons (conversation list, chat state, resolved locale) live for one page load, so reloading at the boundary guarantees the next user starts clean. A client navigation kept the previous user's conversation titles on screen — an information disclosure. Logout also clears the PARAGLIDE_LOCALE cookie so a shared terminal does not pin one user's language for the next.
• A private helper imported across modules is a design smell, not a shortcut. Four of them had grown (_readable_document, _role_config, _excerpt, _HEADING_RE): each marked a place where one module had quietly become part of another's interface without saying so. Splitting the big API modules surfaced all four at once. When a second module wants an underscore name, that name is the interface and should be renamed, not imported.
• A test double that patches by module name silently rots. The fake-embedding fixture patched indexing and retrieval; the moment the similarity path moved into its own module, an "offline" test would have called a real endpoint. A fixture that names the modules it patches has to be updated with every split, so it says why in a comment right there.
• A history snapshot is written AFTER its event. So the content stored on an edited event is the state that edit produced, not the one it replaced. Diffing a snapshot against the live document therefore shows the change of the NEXT entry, and the newest entry shows nothing at all. What a timeline entry has to answer is "what did this one do", so the version endpoint returns the pair (content_md + previous_content_md) and the dialog diffs those.
• The product degrades to a search engine, not to an error page. Postgres is already an inverted index (chunks.tsv, a stored to_tsvector('german', …) column with a GIN index), so "no model reachable" costs the generated answer and the semantic half of retrieval, nothing else. The turn ends with the full-text hits as a list, marked as a search without a model, and is persisted like a normal reply so a reload replays it. Worth remembering when adding any other model-dependent surface: ask what remains without the model.
• Classify an endpoint failure once, in LLMError.code. "Not reachable" and "busy, try again" need different words to the user, and a timeout means the second, not the first: the endpoint took the request and never came back. Retrieval embeds before the model is ever called, so the failure often happens in search(), not in chat_stream — the conversations router catches LLMError centrally rather than every mode remembering to.
• Enumeration-resistant login. An unknown email still runs a constant-time dummy verify, so timing does not reveal which accounts exist.
• Admin CRUD guardrails. An admin cannot delete themselves or change their own role (409 self_modification); a password change or reset revokes all of that user's other sessions.
• Sharing is grants, not new permission logic. Multi-department sharing plugs straight into the EXISTS doc_permission branch that was already in searchable_documents_filter — PUT /documents/{id}/departments just manages the rows. No reindex: grants are evaluated live against the table, not denormalized into chunk meta. Any per-department count therefore measures REACH, not a partition: a shared document counts under every department it reaches, so such counts can exceed the distinct document count.
• Self-lockout only bites admins. readable_documents_filter grants the author read access unconditionally, so a normal author can never edit themselves out of their own document; the guard (_guard_self_lockout, a Python mirror of the read rules) only ever fires for an admin changing a document they do not own, who must resend with confirm_lockout. When mirroring a SQL filter in Python for a pre-commit check, keep the two in the same file so they cannot drift.
• A bool = False Pydantic field serializes as required in the generated TS client (openapi-typescript), which breaks every existing caller that omitted it. Use bool | None = None (like the other optional fields) and bool(...) at the use site — then it stays optional in schema.d.ts.
• The audit trail is where "who checked it" lives. document_events (authoring/history.py::record_event) appends who-did-what at every transition; review_resolved writes the answering colleague as the event actor — once a request is answered, the review_requests row only says that nothing is open any more, so the trail is the only place that identity survives. Content-bearing events (created, edited) snapshot the Markdown, never the chunks, so per-event snapshots of the resulting state chain into a version diff. Gotcha: a freshly created document must be flush()ed before recording its created event — the UUID PK default lands at flush, not at object construction, so document.id is None until then. History is served through the document's own read gate, so it cannot leak to a user who may not read the document.

LLM & prompts

• chat_template_kwargs.enable_thinking = false is the big latency win. On a reasoning-capable local model it drops section refinement from ~10s to ~1s with no quality loss, because refinement is a mechanical rewrite. Passed through extra_body; ignored by endpoints that do not support it.
• Prompt shape for caching. Keep the system prompt byte-identical and the history stable, and put the volatile part (retrieval excerpts, the grounding block) last — the endpoint then reuses the cached prefix and only reprocesses what changed.
• Content never reaches logs. LLMError is raised from None so a model response cannot ride out in a chained traceback; SQL bind params are stripped; the openai SDK's DEBUG body-logging is capped unless PABLAN_DEBUG_LOG_PROMPTS is set (never in production). This is canary-tested.
• One SDK-level retry covers the occasional llama.cpp APIConnectionError; chat_json additionally validates and retries once on a schema mismatch.
• Prompts are admin-editable, defaulting to code. Every system prompt renders through app/prompts/overrides.py::get_prompt(key), which returns a DB override (prompt_settings) or the code default (app/prompts/defaults.py). This mirrors the LLM-settings cache but is simpler: the reset target is the code default (prompts have no .env), and there is no bootstrap — a missing row just means "use the default". Keep the module-cache clear in the test teardown, or an override leaks into the next test. Prompt-cache byte-identity (D22) still holds because a value changes only on an admin write.
• The context inspector needs the FULL retrieval, so a no-answer keeps its results. Query mode used to discard the low-confidence hits (results = []); now it keeps them, sends them in the sources event marked used=False, and passes only used ones to the prompt. That is what lets the "?" inspector explain a no-answer ("these were close but too weak") instead of showing nothing. The cited-source badges filter to used.
• The shipped system prompt was picked by measurement, not by taste. A bench over 15 objective checks (grounding phase, the load-bearing fact, an honest "not documented", answer language) on the dev corpus: the long rules-list version scored 42/45 at 486 chars mean answer, a four-line version 45/45 at 279 chars. Everything the list spelled out, the model already did; what it was missing was the one instruction that carries its weight — say when something is not documented — without which the three no-answer cases all produced plausible filler. Shorter prompt, shorter answers, better score. The bench lives in the session scratch, not in the repo: it needs a seeded instance and a live endpoint, which is what make eval is for — port a case there when it earns a permanent floor.
• Asking about the product is a retrieval question, not a special case. The help/ pages are indexed like anything else, so "how do I share a document?" is answered through the same path — and tests/evals/test_self_knowledge_eval.py measures it (10/10, and asserts the help pages do NOT outrank company documents for a company question, which is the failure mode that would matter).
• Cloud validation is still owed. Every shipped prompt (query, refinement, grounding, topic summary, title) has only run against a local Gemma-class model. The first cloud-model eval pass is roadmap epic 15 — expect to fix behaviour a 26B model tolerated.

Jobs & runtime

• The claim transaction stays open for the whole handler. Claiming a job (FOR UPDATE SKIP LOCKED) and completing it happen in one transaction, so a crash mid-handler rolls the claim back and the job is retried — at the cost of one pinned connection per in-flight job.
• Backoff is 30s·2^n over 5 attempts; failure bookkeeping runs in its own transaction. Retention cleanup reschedules itself daily.
• The endpoint gate is per base_url, not per role (llm/gate.py). chat and utility point at the same server in the shipped config, so a per-role limit would let one server take twice its slots. The waiting is bounded twice (a wait timeout and a queue-length cap) because an unbounded wait in front of a saturated endpoint is the same failure as no gate at all, just quieter. Both refusals reuse llm_busy, so the UI has one sentence rather than two. The background job worker is strictly sequential, so a big reindex contributes ONE concurrent embedding call, not a burst.
• Exactly one uvicorn worker. The job loop, the metrics registry and the LLM client cache are per-process; a second worker would double-run the queue and split the metrics. The compose file pins --workers 1.

Frontend & i18n

• Citations are snapshotted into messages.meta. Chunks are disposable and get reindexed, so an answer freezes the exact passages it cited at answer time; the excerpt is re-cleaned to plain prose on the way out (cleaning is idempotent).

• SSE via fetch + ReadableStream, not EventSource. The turn is a POST that must be abortable (the stop button), which EventSource cannot do. A client abort still persists the partial assistant message (asyncio.shield on the backend).

• Theme per device, language per account, both applied pre-paint. Theme lives in localStorage and is stamped onto <html> by a boot script before first paint; language comes from users.locale and is baked into the first SSR byte — otherwise the page flashes the wrong theme/language on every load.

• Paraglide pitfalls. Source is de (informal "du"), en written in the same change; two lint rules fail the build (a missing en key, an em/en dash). There is no url locale strategy (locale is not in the path). Intl formatters must be passed the resolved locale explicitly or they default to the server's. ICU plurals have their own syntax trap. Recompiling messages under a running dev server can wedge it — restart after adding keys.

• hooks.server.ts needs an absolute backend URL. A relative event.fetch never reaches the vite proxy, which silently breaks server-side auth calls.

• The chat renders Markdown LIVE while streaming. Rendering plain-text tokens and then re-rendering to Markdown on completion read as a snap; the turn now goes through the sanitizing Markdown component every token, so the formatted output is unveiled progressively. Partial constructs (an unclosed ** or $) render as literal text until closed — a small local flicker, far better than the whole-reply snap.

• The local model emits real LaTeX, inline $...$ and display $$...$$ with \frac, \sqrt, superscripts. $lib/markdown.ts runs marked-katex-extension + KaTeX BEFORE DOMPurify, and the KaTeX CSS/fonts are bundled locally (imported in the root layout — no CDN, the artifacts self-host). DOMPurify keeps <span class style> by default, so the typeset math survives sanitizing; throwOnError: false degrades a malformed formula to text instead of throwing mid-answer.

• Prefer SvelteSet over a plain Set for reactive membership state — eslint (svelte/prefer-svelte-reactivity) enforces it, and mutating it (add/delete) is reactive without reassigning.

• The refine suggestion is an inline CodeMirror block widget, not a side pane. It appears right under the section being edited (WritingEditor.svelte): a WidgetType wrapping a persistent DOM element (built once, eq() compares by element identity so CodeMirror reuses it) placed by a StateField + StateEffect. Tokens stream by mutating that DOM directly; CodeMirror does not learn a widget's height changed on its own, so call view.requestMeasure() (rAF-throttled) after each update or the lines below overlap. ignoreEvent() returns true so the accept/dismiss buttons handle their own clicks, and their mousedown preventDefault keeps the editor from blurring first.

• The editor is not continuously autosaved (that would empty the diff you are shown before saving). A beforeNavigate guard instead discards an abandoned, never-filled template draft (only headings/blank lines) and saves an edited-but-unsaved draft on the way out — fixing both the data-loss trap and the empty-draft clutter without touching the save flow. Skip it once the document was published from the editor, or it would re-save over the publish.

• Seeded documents are built through the real process, not inserted finished (app/seed.py). Each one gets the history it would have had: the empty draft, one edited snapshot per section as the author works down the page, the publish, and the questions colleagues asked afterwards — plus a few documents left as drafts and one archived, so every screen has something to show. Without it the history view, the version diffs and "recently changed" are empty or lie in every dev stack. Order the corpus deliberately (the overdue document has to be an old one, drafts have to be the newest), and give document_events and review_requests explicit timestamps: server_default=now() would stamp the whole life of a document at seed time.

• Bits UI keeps inactive tab panels mounted, just hidden. On a page of independent panels (/admin) that means all four fetch their data on arrival, and controls nobody can see sit in the DOM where a click never reaches them (Playwright finds the element, then times out waiting for it to be visible). lib/components/Tabs.svelte renders only the open panel.

• A grant that ends needs its own access reason. Answering a review request on someone else's draft takes the access away again — correct, and it dropped the reviewer on a 404 for doing what they were asked. AccessReason.review names the case ("the only thing letting you in is the request"), so the UI can say thank you instead. It mirrors readable_documents_filter, where the visibility rules only apply to a PUBLISHED document: on a draft, public never explains access, the request does.

• The classic-search fallback ORs its terms; the hybrid path ANDs them (rag/retrieval._any_term_tsquery). websearch_to_tsquery builds an AND over every lexeme, which is fine when the vector half carries the recall — but alone it answers a typed-out question ("Wie läuft die Qualitätsprüfung im Wareneingang?") with nothing at all, because no single chunk contains every word. Rewriting the operators between the groups (' & ' → ' | ' on the tsquery text) keeps quoted phrases and exclusions intact and leaves the ranking to ts_rank_cd. NULLIF the result: a query of nothing but stop words rewrites to an empty string, and to_tsquery('') is a syntax error rather than an empty match.

• The fallback needs chunks, so it needs a past embedding run. chunks.tsv is generated from chunks.content, and a chunk row is only written when reindex_document succeeded — which calls the embedding endpoint. A document published while the embedding endpoint is down has no chunks at all, so the "classic search" fallback cannot find it either. It keeps an already-indexed knowledge base usable without a model; it does not index without one.

• The e2e suite pins English with Accept-Language, which an ACCOUNT language outranks — that is the product rule (a language a person picked follows them to every device), so a dev stack where somebody switched the admin to German failed every spec that reads a label. The login helper hands the account back to "follow the browser"; specs that run as that user select tabs by testid rather than by label.

• npm run check compiles Paraglide with the CLI, which used to write the DEFAULT strategy chain over the one vite.config.ts configures — leaving the dev server (and the e2e suite, which pins Accept-Language) resolving German for everyone until the next vite restart. The messages script now passes the same --strategy flags. The other half of the trap stands: recompiling while vite dev runs breaks its module graph until restart, which is why make lint skips the recompile when :5173 is listening.

History

A dialogue-driven "interview" capture engine (a state machine that interviewed the employee and assembled a document from the answers) was built first, then removed and replaced by the writing-first editor that exists today. Nothing from it remains in the code path; the roadmap describes building the current system directly. The only ideas that carried across are generic: template import/upsert by config id, and the pyyaml dependency.